Research:New editor experiences, 2017

This page will describe a design research project done in South Korea and the Czech Republic in 2017. Our report can be cited as Reboot and the Wikimedia Foundation (2017), "New Editor Experiences: Summary of Findings from the Korean and Czech Wikipedias".

Preparations

We announced our plans to the Korean Wikipedia and to the Czech Wikipedia.

Research team

The research was conducted by the following team:

Lucia Cizmaziova, Researcher in the Czech Republic
Lauren Gardener, Reboot
Hyun Ju Lee, Researcher in South Korea
Panthea Lee, Reboot
Soohyun Pae, Participant Recruiter in South Korea
Catherine Park, Researcher in South Korea
Neil Patel Quinn, Product Analyst, Wikimedia Foundation
Abbey Ripstra, Lead Design Researcher, Wikimedia Foundation
Ivana Lukeš Rybanská, Researcher in the Czech Republic
Václav Zouzalík, Participant Recruiter in the Czech Republic

Recruitment

We found editors to interview by searching the MediaWiki databases for new users who had given an email address. We then sent them emails which explaining our research, inviting them to participate, and asking those who were willing to fill out a survey. The survey asked for information including the respondent's demographics, contact information, neighborhood where they wanted to meet, and times during the week they were generally available to meet, so a survey response reflected a concrete willingness to participate. The survey was covered by this privacy statement, which we also provided in Czech and in Korean.

A recruiter on our team then contacted respondents whose location and availability fit our schedule to arrange an interview. We did not aim for a random or strictly representative group of interviews; instead we prioritized those respondents who (based on their survey responses, initial communications, and editing history) seemed to have experiences that were less familiar to us or which we had not heard in previous interviews. For example, during most of our recruitment, we prioritized female respondents, while during the later stages of our research, we avoided interviewing users who seemed to have edited for promotional reasons.

We sought to interview users in their homes or workplaces, for additional insight into their background, but most users opted for interviews in a neutral public location. For example, in the Czech recruitment survey, 16% selected "home", 21% selected "workplace", 45% chose "cafe", and 24% chose "other" (often a public place like a restaurant or library).

In addition, we directly recruited few experienced editors who were already known to WMF as program or affiliate leaders.

South Korea

Initially, we contacted all the users who had (1) registered in the previous 6 months, (2) made at least one edit, (3) given an email address when registered, and (4) if their last edit was in the past 90 days, had not made it from an IP address that geolocated to outside South Korea. This gave about 2 600 users. We emailed all the users with a standard invitation, and sent those who had not responded a standard reminder 11 days after the initial message was sent. Of these 2 600 email address, about 150 (6%) returned as undeliverable. Out of these 2 600, about 380 (15%) started the survey, and 140 (5%) completed it.

From this initial group, we had many respondents who had made only a few edits, but none who had edited more heavily. To attempt to drive more respondents from this group, we expanded our window to registrants from the previous year and looked at those editors who had at least 3 hours of editing session time (about 310, out of 7 600 total). We targeted these prospects with emails acknowledging them as a top new editor or (for roughly 30 of the most active) with a personalized email or talk page message. We received about 10 responses from this group.

Demographics of survey respondents

Unfortunately, we forgot to retain summary demographic data before deleting the survey responses.

Czech Republic

During our second research sprint, we improved our recruitment in two main ways.

First, we expanded our main recruitment pool to editors from the previous year. Second, we came up with a more useful scheme for segmenting our potential respondents based on their patterns of editing activity and tailored our invitations to each group. We considered new editors who had edits spanning at least 30 days and at least 1 hour of editing session time as retained and the rest as non-retained, and from our retained editors we also split out a top retained group of editors who had at least 10 hours of editing session time and who had edited at least 10 different pages.

With our non-retained group, our invitations emphasized that we could learn a lot from an interview with them even if they had only made one or two edits. With our retained and top retained groups, our invitations praised them for being in the top 10% and top 1% of new editors, respectively; this emphasis seemed to be much more effective than our generic invitations during our first round of recruitment in South Korean. We sent one reminder 8 days after our initial email for our survived and top survived groups.


Group	Invitations sent	Responses started	Responses submitted
Non-retained	3 479	355 (10.2%)	102 (2.9%)
Retained	295	67 (22.7%)	17 (5.8%)
Top retained	34	21 (61.8%)	10 (29.4%)

Demographics of survey respondents

Preferred interview language
Answer	%	Count
Czech/Slovak	69.34%	95
English	1.46%	2
I am comfortable speaking either.	29.20%	40

Primary occupation
Answer	%	Count
Working at a job	56.20%	77
Attending school or university	15.33%	21
Looking for a job or unemployed	1.46%	2
Raising children	0.73%	1
Retired	16.06%	22
Other	10.22%	14

Gender
Answer	%	Count
Male	76.47%	104
Female	22.79%	31
Other	0.74%	1
Prefer not to say	0.00%	0

Age
Answer	%	Count
16–17	5.88%	8
18–24	9.56%	13
25-34	16.18%	22
35-44	28.68%	39
45-54	14.71%	20
55-64	10.29%	14
65-74	11.03%	15
75-84	2.94%	4
85 or older	0.74%	1

Methods

The topics of the interviews included other activities the users participated in (particularly ones that were voluntary, Internet-based, or related to knowledge-sharing), their technology ecosystems, their attitudes towards and awareness of Wikipedia, and their experiences while contributing.