Research:Perception on rules, values and motivation structure of Wikidata contributors

Created

02:50, 15 April 2020 (UTC)

Contact

Charles Chuankai Zhang

University of Minnesota

Collaborators

Loren Terveen

University of Minnesota

Duration:  2020-April – ??

---

Research:Projects

This page documents a research project in progress.
Information may be incomplete and change as the project progresses.
Please contact the project lead before formally citing or reusing results from this page.

Structured data peer production (SDPP) platforms like Wikidata and OpenStreetMap play important roles in knowledge production. Compared to traditional peer production platforms like Wikipedia, data produced from SDPPs are more structured and machine-readable. We propose to do an interview study of Wikidata editors to learn their perceptions of this community's quality control rules and regulations. We also want to learn what motivates editors to edit Wikidata (compared to their motivations for editing Wikipedia if they edit there as well), especially as it relates to how Wikidata is used by various applications. We hope our results can be used to help the Wikidata community develop apps and widgets to improve the Wikidata editing experience.

Context

About Wikidata

Basic information about Wikidata can be found here.

Anticipated Contribution

First, we would like to hear from the editors what motivates them to make the contributions. There has been researchers conducting studies with Wikipedia editors to summarize their incentives of contributing to Wikipedia. Yet considering the unique fact of Wikidata that data is mainly used or externally queried by APIs like SPARQL that are untraceable for most editors, we are interested in whether Wikidata editors have knowledge about whether their contribution is going to be used outside Wikimedia community and how knowing(or not knowing) the information will influence their editing behavior. We would also like Wikidata editors share their stories about their editing experience, how they start editing and what motivates them do the edits.

Secondly, through this interview study, we aim to identify editor's explicit invocation of rules and guidelines in Wikidata. There are a large number of edits that are being reverted or reedited immediately, with most of them are vandalism. Yet some of these edits may come from good faith editors' that try to contribute to community but did not do it correctly. We want to find out whether there's a gap between the rules and editors' perception of rules that is causing this issue. If there's a potential disconnection between them, not only the quality of data may be affected, but also it will diminish the editors passion to make contribution. In the meantime, we would also like to investigate whether the rules themselves are fully functional. If specific rules that are causing frustrations or confusion to the editors are identified in this study, it could help improving the community and editors' editing experience.

Methods

Through a qualitative interview study, we aim to understand the motivation structure of Wikidata editors. We would also like to identify if there are specific rules in this structured peer production platform, compare them to Wikipedia if they do exist, and study how contributors perceive and follow these rules. We would like to recruit 20~30 editors who have experience editing Wikidata for a 30-minute semi-structured interview. Questions will focus on editors' editing activities, perception of their contribution reuse outside Wikimedia community and their perspectives on rules and contributions. Wikipedia editing experience is not mandatory in this research.

Recruitment

We planned to recruit Wikidata editors in following ways:

1. Post invitation message on Wikidata:Project_chat.

2. Send out individual invitation message on users User:Talk page. We plan to recruit 20~30 editors so we expect to send out around 100 invitations.

Anyone who is interested in working on this project with us could fill in a brief questionnaire here or leave a message on https://www.wikidata.org/wiki/User_talk:Chuankaz so that we could set up an interview.

individual invitation message template:

Hi Username,

I noticed your editing stats in Wikidata, which led me to look up your profile. Thank you for all the great work!

I’m reaching out to you because I’m working on a research project about understanding what motivates editors like you to contribute to Wikidata. We’re also interested in learning about how you feel your contributions are being used outside of Wikidata. Since you are such an active community member, I thought you might also be interested in helping to build the broader community’s knowledge about Wikidata, and why it matters.

If you’re interested, let’s schedule a time to talk over Zoom, or whichever platform you prefer. If you are interested, please fill in a questionnaire. The conversation should take about 30 min.

Hope you have a great day,

Chuankaz (talk) 20:43, 27 May 2020 (UTC)

Interview Structure

1. Ask basic information about editor's background and editing experience on Wikidata.

2. Ask about details of contribution (e.g. last editing activity on both platforms).

3. Gather editor's thoughts and knowledge of how their contribution(s) will be used.

4. Ask questions about perception of rules on Wikidata.

The consent form and interview question draft is available here.

Timeline

2020.4 IRB review

2020.4 - 2020.5 Receive input and feedback from the community, project modification

2020.5 - 2020.7 Recruit participants and conduct interview

2020.6 - 2020.8 Data analysis

2020.8- TBD

Policy, Ethics and Human Subjects Research

After submitting this study(IRB ID: STUDY00009161) to University of Minnesota's Institutional Review Board, the IRB determined that this study meets the criteria for exemption from IRB review on April 8, 2020. To arrive at this determination, the IRB used “WORKSHEET: Exemption (HRP-312).

Confidentiality

The records of this study will be kept private. In any sort of report we might publish, we will not include any information that will make it possible to identify a subject. Research records will be stored securely in a University of Minnesota owned GitHub repo and only the contact of this project in this project will have access to the records. The recording data(only audio of the interview will be kept) will be used for data analyzation and only students evolved in this project will have access to the data. The data will be deleted as soon as the transcription has been done.

Voluntary Nature of the Study

Participation in this study is voluntary. If you decide to participate, you are free to not answer any question or withdraw at any time.

Participation

If one is a Wikidata editor and would like to participate in this study, he/she could fill in this questionnaire here to provide a little bit of background information and set up an interview.