Research:Guiding and Retaining Volunteers Using Social Media Bots
Helping to Retain Newcomers and Bring More Diverse Topics to Wikipedia
editRecruiting and retaining volunteers can be difficult, especially volunteers who have different expertises and can provide diversity to a project. In the case of Wikipedia, research shows that although readership has been steadily growing, the number of active contributors started declining in recent years [4]. The difficulty of engaging, and finding tasks for new contributors has been one the main challenges faced by the project [2].
In this project, we will investigate systems that guide the contributions of newcomers to retain them longer term in a project. Our insight is that by guiding newcomers to provide contributions that they can do in short bursts of time, and also providing them with micro-socializations opportunities with experienced Wikipedians we will be able to retain newcomers longer term and also be able to include more diverse topics in Wikipedia.
In particular, our objective is to investigate:
- Can systems that guide newcomers to do micro-contributions and also provide them with micro-socializations be used to retain newcomers longer term ?
- Can systems that guide newcomers to do micro-contributions and also provide them with micro-socializations be used to bring more diverse topics to Wikipedia ?
- What are the opportunities and limitations of systems that guide newcomers to micro-volunteer on Wikipedia?
- What are the opportunities and limitations of systems that provide newcomers with micro-socializations from experienced Wikipedians on Wikipedia?
- What are the best ways to guide newcomers to help retain them and also help Wikipedia get more diverse articles?
- What are the best micro-socializations to give newcomers to help retain them and also help Wikipedia get more diverse articles?
- Are there differences between guiding newcomers who joined Wikipedia during an offline event (e.g., an Edit-a-thon) than newcomers who joined on their own online?
We build our system on research findings that identified that pre-selecting tasks can dramatically increase contributions [2]. By providing guidance, we hope that our system will help people to avoid the hurdles of finding tasks where they can volunteer; or understanding the whole editing process to know where they can best use their knowledge. Our hypothesis is that via these micro-participations, we will generate a large specialized volunteer workforce that can, in seconds, help the Wikipedia community when needed.
Broader Impact
We will publish all results and findings in open access spaces, and share all materials and results for public use. All software produced for the research will be open source.
Participation Time
We expect newcomers to spend from 1-5 minutes in each micro-task; and participate no more than an hour total to the study. Participants will be able to withdraw and stop participating whenever they want without any penalization. We will only recruit 1-5 participants directly from the Wikipedia community.
We expect experienced Wikipedians to to spend from 1-5 minutes in each micro-socialization; and participate no more than an hour to the study. Participants will be able to withdraw and stop participating whenever they want without any penalization.
Related Work
Several platforms have tried to design approaches to recruit volunteers [1, 3, 6]. Others have focused on creating workflows that encourage new volunteers to stay. Such platforms generally implement sandboxes where newcomers can make safe contributions, as well as learn from more experienced volunteers about the community [6]. However, the approach requires experienced citizen volunteers to invest a great amount of time providing assistance, which can limit and affect their own contributions. Other approaches have engaged new citizen crowds with simple lightweight feedback processes [1, 6]. Note, however, that these techniques operate only with the volunteers who have arrived to the platform by themselves. This can limit the type of people who initially decide to take part and influence the amount and type of people who are continuously active in the effort.
To leverage more diverse participants, some approaches [9] have gone outside Wikipedia to obtain contributions. There are bots that tweet each time a new Wikipedia article is created to request help expanding it. However, to date a systematic study on the design of these social media bots to recruit and engage participants in Wikipedia does not exist. However, the bot only targets its Twitter followers and does not offer any guidance, likely limiting the amount and quality of the participation. A different study used bots to recruit volunteers [7]; however, it focused on encouraging participation from the general population instead of experts. We will study how via online bots we will be able to guide the contributions of newcomers experts in diverse topics. We hope that with our approach we will be able to get more diverse topics covered on Wikipedia, and we will be able to make it easier for newcomers to contribute longer term to the effort.
Low Barriers to Participation.
To enable lower barriers of participation for editing Wikipedia, we will recruit volunteers from social media and wikipedia edit-a-thons, and ask them to contribute to Wikipedia. We hope that via this process we can enable a larger number of people to continue contributing to Wikipedia.
Study the effects on asking users edits random articles vs articles in which the user shows an interest in a specific topic.
We anticipate that the users who are asked to edit an article in which the users has knowledge or is interest in, it will increase the willingness of the user to participate in the edition of Wikipedia.
Promoting Strongly Support for Sharing
We will use marketing techniques to create supportive audiences that will encourage people to contribute content to particular topics in Wikipedia. We will study different ways of creating supportive audiences, and also different ways of requesting contributions to those audiences, to identify the most effective techniques. We expect that by creating supportive audiences we will retain volunteers longer term, and volunteers will contribute higher quality work.
Enabling Informal Mentorship
We will create informal mentorship between experienced Wikipedians and new volunteers recruited from social media, by enabling micro-socializations between the two on social media. We believe that these micro-socializations will minimize the burden imposed on Wikipedians, while helping newcomers to become familiar with the Wikipedia community and become core contributors in the near future. We hope that by actively prompt a participatory culture, we will empower Wikipedia to have a larger more reliable volunteer workforce. We focus on four aspects of Wikipedia work that we expect our approach to affect.
- Retention (survival) of new volunteers.
- The productivity & survival of new volunteers.
- The time and burden that current Wikipedians have to invest in recruiting and guiding newcomers.
- The ease of contributing to Wikipedia.
Methods
editTo test the effectiveness of our bots we will run a controlled test on the Spanish Wikipedia. During the test, we will use our bots to harvest a participatory culture, and guide the contributions of volunteers on social media to half of Wikipedia's Wikiprojects. The other half of the projects will receive the current experience (control).
- experimental: volunteers recruited and guided by bots that harvest a participatory culture.
- control: no change.
We will analyze how each of the variables used to create a participatory culture affect the retention of volunteers and the quality of work that volunteers produced. For each participatory culture variable we will study how much it affects the retention of volunteers and quality of work volunteers produce. In specific, we will explore the following metrics related to retention and Wikipedia labour.
Retention
editTo look for evidence on how our bots may help in the retention of newcomers, we measure different aspects related to retention:
- short-term survival rate: based on surviving new editors, we use trial & survival periods of 3 and 4 days to look for evidence of short-term retention effects
Work
editTo look for evidence that our bots transform the work produced by new volunteers, we examine:
- participation per newcomer: median # of contributions per newcomer.
- productive contributions per user: based on productive new editors
Policy, Ethics and Human Subjects Research
editIt's very important that researchers do not disrupt Wikipedians' work. Please add to this section any consideration relevant to ethical implications of your project or references to Wikimedia policies, if applicable. If your study has been approved by an ethical committee or an institutional review board (IRB), please quote the corresponding reference and date of approval.
Results
editThe results were presented as a poster at CSCW 2016. Flores-Saviaga et al. (2016) doi:10.1145/2818052.2869106.
References
[1] G. Ciampaglia and D. Taraborelli. MoodBar: Increasing new
user retention in Wikipedia through lightweight socialization.
In CSCW’15.
[2] D. Cosley, D. Frankowski, L. Terveen, and J. Riedl. SuggestBot:
using intelligent task routing to help people find work in
wikipedia. In IUI’07.
[3] A. Halfaker, O. Keyes, and D. Taraborelli. Making peripheral
participation legitimate: reader engagement experiments in
Wikipedia. In CSCW’13.
[4] Aaron et al. Halfaker. 2012. The rise and decline of an open
collaboration system: How WikipediaâA˘ Zs reaction to pop- ´
ularity is causing its decline. American Behavioral Scientist
(2012), 0002764212469365.
[5] B. Mako Hill and A. Shawand. 2013. The Wikipedia Gender
Gap Revisited: Characterizing Survey Response Bias with
Propensity Score Estimation.
[6] J. Morgan, S. Bouterse, H. Walls, and S. Stierch. Tea
and sympathy: crafting positive new user experiences on
wikipedia. In CSCW’13.
[7] S. Savage, A. Monroy-Hernandez, and T. Hollerer. Botivist:
Calling Volunteers to Action using Online Bots. In CSCW’16.
[8] C. Toxtli and S. et al. Flores-Saviaga, C.and Savage. BotViz:
Data Visualizations for Collaborations With Bots and Volunteers.
CSCW’16, posters.
[9] E. Zangerle, G. Schmidhammer, and G. Specht. Wikipedia
on Twitter: analyzing tweets about Wikipedia. OpenSym’15.