Research:Wikipedia Dispute Resolution

Contact

Steven Zhang

English Wikipedia

Research:Projects

This page documents a completed research project.

Key Personnel

Steven Zhang

Project Summary

No study on Wikipedia dispute resolution has ever been undertaken. There is a relatively small core group that is active in dispute resolution, and we have our own views on what works well within dispute resolution, and what can be improved. The main problems that we've observed (there are about a dozen of us that have been in discussions) is a low amount of active users in dispute resolution (compare it to very high doctor-patient ratios, we can't look after everyone well) and inexperienced volunteers (a first year med student doing open heart surgery). This has led to people leaving the project. I have a small handful of examples in my talk page archives but I have seen examples of people leaving, and the fallout from a dispute due to an inexperienced mediator. I just haven't collected results of it. No study on DR has been done in the past, and that's kinda the whole point of this. We as a collective have our views, concerns and ideas, but we want to get the views of the community on the matter, and that is the purpose of the survey. I hope people will point out aspects of dispute resolution that they think work well and don't work so well, offer suggestions on ways the dispute resolution processes could be altered to improve the success rate of dispute resolution, and hopefully get involved in the process. I see the end result as proposed process/policy changes on the English Wikipedia, and training workshops similar to (and including) the one proposed for Wikimania see link to increase the number of users that are active in dispute resolution. We have a pretty full lesson plan, what we have listed there is just an outline.

Methods

My main focus here is the survey of the Wikipedia community. Jonathan Morgan has generated a list of users to offer the survey to: all users that have edited any of a list of dispute resolution pages, for a total of 10 or more edits within the past two years. This query yields a sample of approximately 2,000 users who we will offer the survey to via a talk page notice, which will be delivered by a robot. The survey will be conducted with Google Forms, and can be found here. I've discussed the survey with several people, including Oliver Keyes (who did the recent new pages patrol survey) and used the same demographics buckets that they used, as well as the Wikimedia Editor survey. This allows for easier comparison between editors overall, and editors that participate in dispute resolution.

As for the reason for the sample size, I've been told that on average, one in ten respond to the survey, so from an offering of just under 2,000, I would hope for about ~200 responses.

Dissemination

The results of the survey will be collated, with sensitive information de-identified. To ensure privacy, survey data will be de-identified in the following manner:

all demographic data will be presented in aggregate (i.e. "25% of survey respondents were between the ages of 20 and 30")
Once the desired number of respondants have filled out the survey and before any data analysis is conducted, the survey data file will be downloaded and each survey response will be assigned a unique alphanumeric key (e.g. dr1, dr2, dr134). Personally identifiable information (PII), which in this survey consists of real names, user names and contact email addresses solicited directly from respondents, will be removed from the archival data file and stored in a separate 'lookup' file which will contain no other survey data, in which each respondent's PII will be identified by the same unique alphanumeric key.
no copies of the original downloaded survey datafile that contain PII and do not contain the numeric key will be retained.
The lookup file will be stored offline in a secure location, separate from the archival survey datafile. The data in the lookup file will only be used to contact respondants who have indicated in their survey responses that they would like to be contacted.
if "short answer" responses to survey questions (e.g. "Describe a positive experience you had with DR") are published as a batch, each respondent's responses be identified by their unique alphanumeric key only, not the respondent's user name or other PII.
if "short answer" responses to survey questions are published individually--for instance, if they are directly quoted or used as illustrative examples in a conference presentation or a summary report--all real names, Wikipedia usernames or email addresses will be redacted from the quotation.

Once this is completed I would present the findings in a report similar to the one that was submitted for the Wikimedia Editor Survey. I would also use the results in a presentation at Wikimania, to detail the problems we found with dispute resolution through the survey, the amount of people that participate and use dispute resolution, and so on. I would hope that this would increase the awareness of dispute resolution on our projects, as well as get more users involved in the process.

Benefits for the Wikimedia community

In the long term, if all goes to plan, I envision a DR process that works well and most of the time, doesn't leave editors fuming or storming away from Wikipedia in frustration. In the short term, I see a greater understanding of the DR processes and where we stand with everything at present. What works. What we need to fix, and how we can fix it. No study or survey has ever been done on Wikipedia dispute resolution before and I feel this is a necessary first step in my proposed fellowship.

Time Line

The survey is ready to be offered to the community, pending approval by RCom. I'd give a month or so to get responses, and then take a month or two to collate the results. I would then prepare the report, as well as use some of the information in the survey to fully prepare my proposed presentation at Wikimania and use the information that I gather from the survey to move forward ideas on-wiki as well as techniques to teach users to improve the skill of mediators (improving the resolution of disputes and lessening the amount of users leaving over inadequate dispute resolution)

Contacts

I can be reached via email, Skype or by phone, and can provide the details of the latter two if required. My email is cro0016 gmail com. I apologise if this is a bit short, and am happy to elaborate or fill in any gaps as needed.