Grants:Project/nschwitter/The Role of Offline Ties of Wikipedians

This project is funded by a Project Grant

proposal

people

timeline & progress

finances

midpoint report

final report

statusselected

Project Grants

The Role of Offline Ties of Wikipedians

summaryThis project aims to assess the effect offline meetups have on online behaviour on Wikipedia.

targetGerman Wikipedia

type of grantresearch

amount$26000

grantee• ASociologist

contact• Nicole.Schwitter.1warwick.ac.uk

created on14:13, 9 March 2021 (UTC)

Friendly space expectations

Project idea

What is the problem you're trying to solve?

What problem are you trying to solve by doing this project? This problem should be small enough that you expect it to be completely or mostly resolved by the end of this project. Remember to review the tutorial for tips on how to answer this question.

Problem description

Wikipedia is one of the most visited websites worldwide and functions as an important source of information for the general public in today’s world; The German Wikipedia in particular is one of the largest and most active language versions. Besides its rich online component, Wikipedia is further characterised by regular local offline meetups, which give editors a time and place to get to know each other personally.

While these meetups are generally open to all, a certain reluctance to join them is observable on the organisation pages: In many cases, editors that are or consider themselves to be in a minority on Wikipedia – e.g. newcomers, young editors, women – are hesitant to join local meetups, while other active users have attended hundreds of meetings and can be considered “stars” in the network of Wikipedians. Reading the discussion pages of meetups further reveals that meetups are not neutral spaces but that tensions and cliques can develop. This project asks the question of how important the offline ties created at those meetings are for the online community of Wikipedia. The goal is to understand what effects attendance at offline meetups will have on contributors and to what extent these might be a decisive factor in editor retention and online engagement. It will shed light on the question whether offline meetups are a negligible factor or whether these need to be designed in a more inclusive fashion to foster a more diverse and fair community, and enhance sustainability of the Wikipedia project.

Research questions

The overarching research question of this project asks the following: To what extent is online behaviour on Wikipedia affected by offline meetings between Wikipedians? Three areas which all deal with the problem of creating and sustaining the public good shall further be explored: Productivity, norms, and voting behaviour. The project will focus on 1) how the participation in offline meetups influences an editor's contributing behaviour to articles (i.e. number of contributions), 2) how the participation in offline meetups influence an editor's norm-relevant behaviour (i.e. frequency and target of reverts), and 3) how the participation in offline meetups influences an editor's voting behaviour in RfA processes.

Previous literature

Previous research of other online communities has shown that there is generally strong evidence to support online communities with offline gatherings, as stronger relationships develop. However, these meetings can also bring along new challenges. For example, Sessions (2010) finds that having offline relationships enhances a user's engagement with the online community as a whole and also strengthens ties to other attendees of offline meetings. However, weak relationships with non-attendees dissolve to an extent . While offline meetings can thus be beneficial for the individual, they can have detrimental effects to the online community as a whole. These previously analysed online communities are quite different from Wikipedia, as most are concerned with forums or web-blogs based on discussion and common interests. Offline meetups of Wikipedians have previously only been researched by Stegbauer (2009). This project will extend previous research by analysing more dimensions of online behaviour, applying more advanced statistical methods and assess a longer time frame.

What is your solution to this problem?

For the problem you identified in the previous section, briefly describe your how you would like to address this problem. We recognize that there are many ways to solve a problem. We’d like to understand why you chose this particular solution, and why you think it is worth pursuing. Remember to review the tutorial for tips on how to answer this question.

Evidence-based research will be conducted which allows to conclude whether offline meetups are a relevant factor for contributing towards Wikipedia. To assess the effect that offline meetups have on online behaviour, data on all meetups in the German Wikipedia has been collected and will now be analysed with statistical methods. This will allow to conclude whether offline meetups are a relevant factor for contributing towards Wikipedia. These conclusions will be made on empirical facts instead of speculative thoughts.

The outcomes and results will then be made accessible to a wider audience. On one hand, it will be shared with the Wikimedia community where problematic issues can be flagged. While this research project will not yet directly suggest implementable solutions, it will allow for informed decisions regarding the organisation of offline meetups. On the other hand, the research will be written up as a PhD thesis and academic journal articles. The conclusion drawn there can be transferred to those who organise meetups. In practice, it can help them to better understand what they need to pay attention to in order to organise meetups that are beneficial to the community as a whole.

Project goals

What are your goals for this project? Your goals should describe the top two or three benefits that will come out of your project. These should be benefits to the Wikimedia projects or Wikimedia communities. They should not be benefits to you individually. Remember to review the tutorial for tips on how to answer this question.

1. Identify and characterise all past offline meetups of the German Wikipedia community
2. Understand the role of offline meetups for the Wikipedia online community
   1. Understand their effect in regard to contributing behaviour
   2. Understand their effect in regard to reverting behaviour
   3. Understand their effect in regard to election participation
3. Publish the results in Wikimedia community discussion forums to allow for reflection upon the necessity to create more inclusive offline meetups
4. Create documents/guidelines documenting the data collection and analysis to allow others to apply and reproduce the analyses conducted in other contexts (e.g. other language versions of Wikipedia)

Project impact

How will you know if you have met your goals?

For each of your goals, we’d like you to answer the following questions:

1. During your project, what will you do to achieve this goal? (These are your outputs.)
2. Once your project is over, how will it continue to positively impact the Wikimedia community or projects? (These are your outcomes.)

For each of your answers, think about how you will capture this information. Will you capture it with a survey? With a story? Will you measure it with a number? Remember, if you plan to measure a number, you will need to set a numeric target in your proposal (i.e. 45 people, 10 articles, 100 scanned documents). Remember to review the tutorial for tips on how to answer this question.

The main goal of this project is to conclude whether offline meetups are a relevant factor for contributing towards Wikipedia. To achieve this goal, statistical methods of inferential statistics are employed to assess the effect of offline meetups on online behaviour. For example, methods of social network analysis will allow to describe and map the network of Wikipedians that has developed through offline meetups. Simple comparisons or more complex time series regression models can be conducted to assess whether Wikipedia users begin to contribute more after having taken part in an offline meeting. Panel models, which take into account that one user is observed at multiple moments in time, can be used to analyse the election participation of Wikipedians and find out which factors are most relevant to explain whether a user runs for administrator, votes supportively, votes opposingly, or does not participate at all. These models will allow to conclude the importance of offline meetups on online behaviour as they numerically estimate how much the likelihood and/or extent to contribute, to revert an edit, to run for administrator or to vote on an election changes as Wikipedians become more involved in the offline side and take part in more meetups and meet more other editors.

This output will subsequently positively impact the community of Wikipedia. The research results can be published in Wikimedia community forums such as Der Kurier, to raise community awareness of the role of offline meetups. This will allow for the community to reflect on potential unintended consequences of offline meetups and organise them in a way that can address the issues flagged and allow for more justice and fairness.

The project will also result in a PhD thesis which will be submitted in 2022. The research will also be submitted for presentation at a Wiki Workshop to present it to a wider audience specifically interested in research on Wikipedia. The goal is also to publish two datasets in an anonymised format to allow other (community) researchers to review the research as well as conduct their own. This will be 1 dataset containing all 4403 meetups and 1 dataset containing all 1213 admin elections of the German Wikipedia. Additionally, it is also planned to write up and document the data collection and the analyses that were conducted in the fashion of hands-on tutorial. This will allow other researchers to apply and reproduce the analyses conducted to other contexts.

This project will not result in any major Wikipedia mainspace edits. It has already resulted in some minor edits in which errors of meetups archives were fixed. The description and characterision of offline meetups in the popular Stammtischguide (only with consent of creator) might be extended.

Project plan

Activities

Tell us how you'll carry out your project. What will you and other organizers spend your time doing? What will you have done at the end of your project? How will you follow-up with people that are involved with your project?

This grant will fund the final year of a postgraduate degree (PhD) at the University of Warwick. Many time-intensive activities of this research project have already been conducted throughout the past 2 years. Activities conducted so far:

1. Take part in introductory modules and training for doctoral students at the university.
2. Conduct literature review of relevant research.
3. Definition of scope of the project.
4. Gaining ethical approval of the ethical advisor of the Department of Sociology at the University of Warwick.
5. Receiving necessary training in statistical methods at the University of Warwick/University of Michigan.
6. Manual and automated collection of all offline meetups in the German Wikipedia up to March 2020 (over 4000).
7. Manual and automated collection of all elections for administrators in the German Wikipedia up to April 2020 (over 1000).
8. Collection and processing of user data, using the Media Wiki API and the XML data dumps.
9. Pre-processing part of data.
10. Descriptive analysis of pre-processed data (frequency distributions, contingency tables, spatial and temporal distributions, visualisations).

In the 12 months funded by this project grant, the following activities will be conducted in the following time frame:

Timeline of project

Task	Days required	Time period (ca.)
Finish pre-processing and cleaning of data	20 working days	October 2021
Conduct descriptive analysis of data	10 working days
Conduct inferential analysis of voting behaviour	20 working days	November 2021
Write up of results and creating documentation for reproduceable analysis of voting behaviour	20 working days	December 2021
Conduct inferential analysis of productivity and collaboration	20 working days	January 2022
Write up of results and creating documentation for reproduceable analysis of productivity and collaboration	20 working days	February 2022
Preparation of presentation of research at WikiWorkshop	10 working days	March 2022 (date depends on CfP)
Conduct inferential analysis of reverting behaviour and write up of results	20 working days	April 2022
Write up of results and creating documentation for reproduceable analysis of reverting	20 working days	May 2022
Attendance of local meetup / offline community engagement	5 working days
Write up of research as submittable PhD thesis and write up of documents/guidelines for reproducible analysis	30 working days	June 2022 - July 2022
Preparing data to make it openly sharable and researching repositories for data	10 working days
Spread outcomes to the Wikipedia/Wikimedia community	20 working days	August 2022
Write up of end report for WFM and preparation of academic research articles	20 working days	September 2022

The major activities of the project are the research and labour required to so that the researcher can qualify for their doctoral degree from the University of Warwick. To complement that research, and beyond the scope of academia, the researcher and avid Wikipedia user will present the results to the Wikimedia community to transfer the insights into practical use.

Budget

How you will use the funds you are requesting? List bullet points for each expense. (You can create a table later if needed.) Don’t forget to include a total amount, and update this amount in the Probox at the top of your page too!

• $26000: research sponsorship

The project grant is used to fund the research labour during the final year of the researcher’s postgraduate research degree. As only this final year needs additional funding, no other sponsorship opportunities are available as other funding bodies tend to fund only whole projects. The project requires fulltime dedication.

The annual salary as a research assistant at the University of Warwick (working full time, 36.5 hours) starts in the first year at £26700 ($37500). This project requires fulltime dedication for a year. However, the project will also result in an academic degree (PhD). To account for this, the budget has been reduced accordingly and is comparable to stipends as offered by the Economic and Social Research Council (£21000 = $29300). There is no budget allocation necessary for management.

The Wikimedia project grant would be particularly well-fitting as this research aims to closely follow Wikimedia's mission of making and keeping knowledge public and accessible. Further, it aims to honour Wikimedia’s goals of creating an inclusive and diverse space.

Ethics statement

• This research project has been reviewed by the ethical advisor of the University of Warwick, Department of Sociology.
• The content and outcomes will conform to the wmf:Open access policy.
• This project only uses public data which anyone can access and does not seek any confidential Wikimedia user information.
• Even though this project will only use public data, it is possible to surface information which could be harmful to individuals. Because of this, and balanced against the open access policy, this project will not share all of its data openly and will withhold identifying information of Wikimedia user accounts, but will make anonymised data available. The intent here is user safety.

Community engagement

How will you let others in your community know about your project? Why are you targeting a specific audience? How will you engage the community you’re aiming to serve at various points during your project? Community input and participation helps make projects successful.

In the beginning phase, this project has been discussed with Wikipedia administrators at an edithaton to gain more informal knowledge and an inside view. Also, this research project has and will be presented at conferences and sessions with other researchers which are other PhD students or scientists also focusing on Wikipedia research, network analysis, big data or computational social science as this allowed/allows for an exchange of ideas and improved analysis. The project and Wikipedia as a whole have been and will be presented by the researcher as teaching example for successful open-source collaboration projects to students of diverse backgrounds (e.g., in continuing education modules at the University of Bern, Switzerland).

During the project period, the researcher plans to attend meetups (if the situation permits) and is available for input and discussion. As monthly and midpoint reports will be shared, the input of the community is welcomed. At the end of the research project, outcomes will be shared with the Wikipedia community to allow for further engagement. As the data will also be published, interested community researchers can further engage with it.

While the researcher will aim to be in an exchange with the community throughout the project, she will particularly aim to reach the community at the end of the research project. The research results can be published in Wikimedia community forums such as Der Kurier to raise community awareness of the role of offline meetups. The research will also be submitted for presentation at a Wiki Workshop to present it to a wider audience specifically interested in research on Wikipedia. Additionally, it is planned to attend local meetups (once they can safely take place again) to discuss the research more directly. The researcher will also get in touch with WMDE to ideally discuss the research at a Wikipedianischer Salon.

Get involved

Participants

Please use this section to tell us more about who is working on this project. For each member of the team, please describe any project-related skills, experience, or other background you have that might help contribute to making this idea a success.

• Nicole Schwitter / User:ASociologist
  • I am a PhD student at the University of Warwick. I will be starting my final year for which I require this project grant.
  • I have a background in Sociology and computer science, contributed towards Wikipedia online and taken part in offline events of Wikipedia.
  • I have received extensive training in data science and data analysis which give me the necessary skillset to undertake this project. I have demonstrated my ability to undertake research projects: https://orcid.org/0000-0002-3837-680X
  • I have previously collaborated with the non-profit organisation Oxfam and have written a guideline on web data collection which has been extensively used by them.
• Prof Ulf Liebe is advisor on this project with profound knowledge on data analysis. He is Professor of Sociology and Quantitative Methods at the University of Warwick and Director of the Q-Step Centre which provides social science students with state-of-the-art training in social data science and quantitative methods in general.
• There are other expert advisers available which have done research in online user interaction and big data analysis. They will get credit for their contributions to the project.

Community notification

You are responsible for notifying relevant communities of your proposal, so that they can help you! Depending on your project, notification may be most appropriate on a Village Pump, talk page, mailing list, etc.--> Please paste links below to where relevant communities have been notified of your proposal, and to any other relevant community discussions. Need notification tips?

The research was discussed at an offline edithaton with the organisers (and administrators of the English Wikipedia) in the beginning of the project. A user that has organised some of the largest and popular meetups in Germany has also been informed.

Endorsements

Do you think this project should be selected for a Project Grant? Please add your name and rationale for endorsing this project below! (Other constructive feedback is welcome on the discussion page).

• I think the offline interaction in internet communities and its influence back on the online actions and relationships of community members is an interesting topic both for wikipedia, and for internet communities in general, therefore I would very much like to see this research pursued further. 195.144.34.162 17:22, 12 May 2021 (UTC)
•  Support I support this project. I think it would be really helpful to understand the functioning of the community and the synergy between online and offline meetups. This project can help to improve wikipedia and make it a more diverse and fair place — ZeenahB (talk) 18:56, 12 May 2021 (UTC)
•  Support I strongly support this project. Due to the ongoing pandemic, the balance between offline and online work is being reassessed across many industries. Studying the impact of offline ties between Wikipedians on online behaviour is highly relevant at this critical time, both for Wikipedia and beyond it. Given the extent of research already conducted by the researcher on the topic, I have no doubt that the project will achieve its goals in the proposed timeline. Ofyalcin (talk) 03:43, 13 May 2021 (UTC)
•  Support Strongly support. Seeing the efforts of the Wikimedia community in community-building events such as online and offline meetups and workshops on the one hand, and the well known but not so well understood problems of Wikimedia projects in terms of diversity and gender bias on the other hand, I think this project hits an important spot and asks the right questions to shed more light on the role of offline and online ties and consequences to the community. I specifically advocate the "research first" approach here, as I have seen many actions being taken without a good backup by research to indicate that these actions are effective or even the correct ones to take. Here I can see how the results of the research can lead to much more informed actions in the community concerning offline and online meetups. The proposal is well written and provides a conclusive idea of scope and feasibility for the given time frame. With the grantee being a researcher experienced in network analysis and statistics as well as with practical applications of their research, I have no doubt the project will be conducted successfully and will provide answers with impact to the Wikimedia community. --IdS (talk) 09:25, 13 May 2021 (UTC)
•  Support Very useful to find out how to support and diversify the Wikimedian community.--Flor WMCH (talk) 14:08, 19 May 2021 (UTC)