Research:Wikipedia Editors Survey 2011 November

This page is about the November 2011 Editor Survey. For the preceding April 2011 Editor Survey, see here; for the subsequent 2012 Editor Survey, see here

This page presents a list of frequently asked questions about the Wikipedia Editors Survey November 2011 that has been conducted by the Wikimedia Foundation. If you have any questions and concerns about the survey, please share your feedback with us.

Contact

Mani Pande & Ayush Khanna

Wikimedia Foundation

Research:Projects

This page documents a completed research project.

General questions

Why is the Wikimedia Foundation conducting a survey of Wikipedia editors?

The Wikimedia Foundation conducted the first ever survey in April 2011, which helped us gather some baseline data about the community. The Wikipedia Editors Survey, which will be conducted in November - December 2011, is the second semi-annual survey of the Wikipedia Editors community. This edition of the Editor Survey will serve as a longitudinal study, and help us understanding the evolving needs and experience of Wikipedia Editors.

The key objectives of the study are as follows:

Understand how editor experiences have changed over the past 6 months
Gather data about any changes in the editor demographic
Understand editors' perception of the community (and how it has evolved) using metrics such as the Wikipedia Editor Satisfaction Index
Understand the experiences of new editors
Understand the motivations behind contributing to Wikipedia
Understand what kinds of interactions editors have with others within the community

What does the survey cover?

The survey is limited to Wikipedia projects. The questionnaire has been written in English, and we are going to employ the help of the community to translate the survey into other languages. The translation process should begin shortly, in end of Oct or early November. Our apologies in advance for running it so close to the fundraiser, but we would like to field the survey before the end of year.

When, and how often will the survey be conducted?

This survey will be fielded in end of November and early December, for about 5-7 days depending upon the response rate to the survey.

How can I take the survey?

Every registered user (editor) will see a notification once to participate in the survey. Anyone may click on this link and participate in the survey. We are also looking into the possibility of providing a link to take the survey if respondents miss the notification the first time.

How will the survey data be handled? What about privacy?

The data will be anonymized before conducting analysis to ensure that individual responses cannot be associated with specific respondents. The data will be analyzed collectively and responses will not be associated with any one respondent.

Will the report from the editor survey be shared with the community?

The report from the survey will be shared publicly, and placed into the public domain (Creative Commons Zero).

How can I provide feedback?

You can provide feedback about the survey on the feedback page.

In which languages will the survey be translated?

Last time, we conducted the survey in all languages for which we had a translation. Please see below the data on languages (completed surveys) in which respondents took the survey. To ensure that we are spending our resources wisely, we will accept surveys in all languages where we had at minimum 10 respondents who took the survey.

Row Labels	startlanguage	Percents	startlanguage2
ar	39	0.74%	Arabic	*
ca	26	0.49%	Catalan	*
da	19	0.36%	Danish	*
de	670	12.69%	German	*
en	2395	45.35%	English	*
es	333	6.31%	Spanish	*
fi	55	1.04%	Finnish	*
fr	232	4.39%	French	*
he	57	1.08%	Hebrew	*
hr	15	0.28%	Croatian	*
hu	53	1.00%	Hungarian	*
it	195	3.69%	Italian	*
ja	205	3.88%	Japanese	*
mk	6	0.11%	Macedonian
nl	52	0.98%	Dutch	*
pl	149	2.82%	Polish	*
pt	112	2.12%	Portuguese	*
ru	481	9.11%	Russian	*
zh-Hans	93	1.76%	Chinese Traditional	*
zh-Hant-HK	53	1.00%	Chinese Simplified, HK	*
(blank)		0.00%	—
NA	41	0.78%	—
Grand Total	5281	100.00%	—

Survey design

Who designed the survey, and how?

The survey is a revised version of the April 2011 survey. It was designed by the Wikimedia staff. Several staff members, including editors, provided input into the design of the survey.

How can I help design or improve the survey?

You can provide us your feedback on the feedback page we have set up specifically for the survey.

How is this survey different from the April 2011 survey?

This survey is shorter. It is designed to take no longer than 10 minutes. We received a lot of feedback from community members about the length of the April 2011 survey. Since it was our first attempt to measure the community, we needed to establish a baseline.

About 90% of the questions continue from April 2011, since we would like to conduct a longitudinal analysis, and look for any shifts around participation and gender since April 2011. We also used inputs from the Wikipedia Summer of Research project to better understand experiences of new editors.

Survey implementation

What software will be used? Will it be hosted by the Wikimedia Foundation?

We are still looking at different tools -- both open source and proprietary. We used Lime Survey, an open source tool, for the last Wikipedia Editor Survey, but we are re-evaluating its use since we found critical security flaws. Some of the proprietary tools we are considering are Qualtrics and Survey Gizmo. Once we have made a final decision, we will update this page.

Update: We have decided to move forward with Qualtrics for now.

Will the referring site be tracked?

No.

Will unfinished surveys be saved? Will the answers be counted?

Yes, unfinished surveys will be saved so people may resume them later. We will compare unfinished responses with the finished ones after the survey is taken offline. However, the final analysis will only include complete responses.

Can the respondent "skip" questions?

Based on feedback from the community, we have decided to make all questions non-mandatory.

Why will a user see the link to the survey only once? How?

This will ensure that all users (editors) have an equal probability of participating in the survey, and the survey is not biased towards those editors who edit more frequently. The foundation is interested in eliciting responses from all types of editors. We'll set a cookie within the Central Notice banner for the survey to guarantee that it will only show up once per user.

In case you missed the banner, you may send us an email at survey@wikimedia.org, and we can generate a custom survey link for you.

Privacy & transparency

Is the survey covered by the Wikimedia Foundation's privacy policy?

We worked with our legal team, and have crafted this privacy policy for the survey:

By answering these questions, you permit us to record your responses and agree to donate them to the public domain. This allows us to freely share your answers with others for the purpose of open analysis, research, and study. We will not publicly share your name, address, phone number, or email unless the law otherwise requires or you chose to include that information in answers to questions not specifically asking for your name, address, phone number, or email."

Who will have access to the raw data? Will the raw "anonymized" results be shared publicly?

Researchers working at the foundation will have access to the raw data. We will share data with other researchers for analysis, but we will take steps to account for outliers where identity can be revealed. For example, if only one woman editor from Macedonia takes the survey, we will anonymize the country data for her response before it's shared to protect her identity.

To learn more about the process we followed to anonymize Editor data and protect their identities for the April 2011 Editors Survey, read about it here.

How long will the data be kept?

Since we will use data from the survey to perform longitudinal analysis, we will keep the data for about 2-3 years.

What do you mean by "anonymized"?

Individual responses will not be associated with any user.

Results

The results have been published as a summary report (PDF) and in a series of blog postings.

The raw dataset of responses has been released (in anonymized form) here.