Research:Patrolling on Wikipedia
Editors patrol recent pages and edits to ensure that Wikimedia projects maintains high quality as new content comes in. Patrollers revert vandalism and review newly-created articles and article drafts. Patrolling of new pages and edits is vital work. In addition to making sure that new content conforms to Wikipedia project policies, patrollers are the first line of defense against disinformation, copyright infringement, libel and slander, personal threats, and other forms of vandalism on Wikimedia projects.
This research project is focused on understanding the needs, priorities, and workflows of editors who patrol new content on Wikimedia projects. The findings of this research can inform the development of better patrolling tools as well as non-technological interventions intended to support patrollers and the activity of patrolling.
The goals of the project are:
- Understand the general workflows of editors who engage extensively in patrolling (at the edit/revision level and the article level; in official or unofficial capacities; at a high and low levels of activity) on different languages of Wikipedia and on non-Wikipedia projects such as WikiData and Wikimedia Commons.
- Understand the tools and techniques that these editors use in their work
- Understand the social and technical structures of the MediaWiki platform (e.g. anti-vandalism wikiprojects, Special:RecentChanges, special user-rights, Huggle) that support patrolling activities.
- Identify limitations of current patrolling tools techniques, and structures that may create vulnerabilities to misinformation, disinformation and other forms of individual or coordinated vandalism in multiple languages of Wikipedia.
This research will be conducted in several stages, with each stage building upon the findings from the previous one.
Stage 1: Literature reviewEdit
Status: completed In this stage, we will analyze the existing research literature on the activity of patrolling, including but not limited to patroller experiences, workflows, use of supporting tools, and the impacts of patrolling. This research will be used to inform the development of interview protocols and (potentially) survey questions in later research stages.
Stage 2: Patroller interviews and surveysEdit
- In this stage, we will interview individuals who are engaged in patrolling activities such as new page patrol and recent changes patrol. We may also interview individuals who use assistive tools such as Huggle and Twinkle to perform edit review activities (whether or not they consider their activities to be "patrolling").
For this research, we interviewed 4 people who had been involved in patrolling activities on different Wikimedia projects. Participants were recruited through purposeful sampling, with the aim of selecting a set of interviewees who represented diversity in terms of their patrolling role and focus, and the size of the Wikimedia project(s) they work on.
Interview participant information (click to expand)
|p1||Dutch Wikipedia, English Wikipedia, Wikimedia Commons|
|p2||Greek Wikipedia, English Wikipedia, WikiData|
|p4||Italian Wikipedia, Wikimedia Commons, WikiSource|
Stage 3: Cross-wiki analysisEdit
- August 6 - 9
- Conduct literature review (including a review of previous research conducted by the PI, e.g. Flow workflow interviews) and write up key findings.
- Use key findings to draft interview protocol.
- Elicit suggestions from Community Liaisons for interview candidates (English speakers, not En-Wiki based)
- Submit interview protocol for Legal review and work with Legal to draft consent form.
Collaborators: WMF Community Liaisons; WMF Legal
Deliverables: Literature review summary (wiki page); draft interview protocol (wiki page); list of interview candidates (spreadsheet)
- August 12 - 23
- finalize interview protocol based on Legal input
- send out invitations to interviewees (Target: 3 interviews)
- conduct interviews
- loosely transcribe interview data (doesn't need to be word for word)
- summarize key points
Collaborators: WMF Legal
Deliverables: Final interview protocol (wiki page); Interview recordings (audio and/or video files); interview transcripts (Google doc); consolidated interview notes (Google doc)
- August 26 - 30
- complete data analysis, publish findings, recommendations, and suggest next steps
Deliverables: Report of findings, recommendations, and implications (wiki page)
- September 2019-?
- (TBD) Potentially: surveys, additional interviews (e.g. with new article and article draft patrollers), quantitative cross-wiki comparative analysis, development of machine learning models to identify threats and aid patrolling
Policy, Ethics and Human Subjects ResearchEdit
Interviews and surveys will be conducted according to the Wikimedia Foundation's policies for informed consent. All non-public data gathered during this research (including interview recordings and notes) will be shared and stored in accordance with the Wikimedia Foundation's data retention guidelines.