Wikimedia Meta-Wiki

Grants:IEG/Reimagining Wikipedia Mentorship

< Grants:IEG

status: completed

Individual Engagement Grants
Individual Engagement Grants 		Review grant submissions
review
grant submissions 		Visit IdeaLab submissions
visit
IdeaLab submissions
eligibility and selection criteria

project:

Reimagining Wikipedia Mentorship


project contact:

ijethrobotgmail.com

participants:


grantees:

  • I JethroBT will lead project management and assist with content development / on-wiki organizing.
  • Soni will primarily focus on developing program content and on-wiki organizing, and will also compile and organize reports from grantees.
  • Gabrielm199 will be a researcher on the team. He will prepare background literature, develop predictions for the pilot based on existing adoption programs, and perform qualitative evaluations of learner's experiences in the pilot.

volunteers:

advisors:

summary:

Creating a lightweight model of mentorship to match up learners with experienced mentors to help accomplish editing goals.

engagement target:

English Wikipedia, with possibility to expand to other languages

strategic priority:

Increasing Participation

total amount requested:

22,600 USD


2014 round 1

Project idea edit

What is the problem you're trying to solve? edit

The existing approach to mentorship, while thorough, is impractical due to its commitment length and the overwhelming and comprehensive nature of many "adopt-a-user" programs that exhaust both learners and mentors alike. This approach, therefore, shuts out both learners who want to learn a specific set of skills and potential mentors who can provide guidance in specific areas. Furthermore, research in other online communities has suggested that approaches which isolate newcomers "out of the mainstream" for training purposes actually demonstrate negative results regarding quality of work and connection to the community.[1]

What is your solution? edit

We will create a broadly approachable space (in the spirit of spaces like the Teahouse and IdeaLab) and pilot an a la carte or choose-your-own-adventure approach to skill learning that incorporates the 1-to-1 interactions of mentorship. This approach would allow learners to chart their own course through the program and match them to mentors with the experience to teach those skills. In short, we want to create a space where learners get to decide what kind of editors they want to become. We hypothesize that this approach will attract more participation than traditional mentorship affords.

Project goals edit

Reimagined mentorship will be a less onerous, easier to bite-off program than is currently available. We will emphasize learning specific skills, both fundamental and complex, rather than engaging an entire curriculum. Our hypothesis is that skill-focused learning which meets an individual’s needs will be more effective and attractive than longer commitments required by traditional mentorship. We also predict that this approach to mentorship will be less of a burden on mentors generally.

We want learning a new skill on Wikipedia to be easier and more enjoyable. In our space, we will incorporate strong visual cues that provide recognition of skill proficiency to motivate learners and encourage meaningful interactions between them and our mentors. Providing visual recognition to learners and mentors will help address incentive factors for knowledge sharing that are often found to encourage similar activities in formal organizational settings.[2]

We also wish to reduce the frustration that new editors face. We want to improve new editor engagement to keep more people with us longer from the beginning, and we want to motivate and deepen the engagement of experienced editors so that we retain their valuable presence in the community.

Project plan edit

Scope edit

Project Map edit

Step 1: Background Research edit

The first step of the project will perform background research to check our hunches about the existing adopter program. We will:

  • Survey the research literature (e.g. median time to adoption in 2011 was 4 days[3])
  • Perform a longitudinal analysis of the newcomer performance (e.g. surviving new editor, how long do people wait for a mentor today, how long does mentorship generally last, how often and what is edited after completing mentorship, ratio of adoptees who complete a mentorship program)
  • Interview and survey adoptees/adopters about their satisfaction level with the whole program as well as other mentorship programs, their frequency of meeting, motivation to participate, whether they learned what they wanted to, the quality of the mentor, the quality of their tests or teaching approach, etc.
    • Protocol for interviews with learners and mentors that have participated in existing mentorship programs: Interview Protocol

Step 2: Design a new process edit

  • Project members will start working on content development. We will generate and define a set of skills that learners could acquire through this program. Skills will be broad in scope. Several skills will be broadly applicable to article creation, others will be more specific to skills helpful on discussion pages, templates, or organizational tools common on Wikipedia. We will also define skills that are more fundamental and apply broadly to editing (e.g. referencing, article layout), and also define more complex skills that require proficiency in multiple fundamental skills (e.g. creating a new article).
  • The new mentorship space would be created in partnership with a contracted visual designer (while the current adopt-a-user process remains active). The designer’s help would create a clear, inviting, and consistent look and feel throughout. Components would include learner and mentor profiles, a 'menu' of skill and learning pathways, and features for visually recognizing and acknowledging participants that can be given and received.

Step 3: Design a social technology for matching mentors and learners edit

Technical support to streamline the process: We will build lightweight technical support to streamline the whole process. Profiles with simple skill-seeking and skill-teaching capabilities will allow for effective matching. We also want to craft a well-designed interface for mentors as well to keep an eye on the learners so they can monitor users and help them them effectively. At its heart, mentorship is about people helping people individually, but a little tech will go a long way towards making that more robust and user-friendly.

  • A programmer will be contracted to aid in creating ways to match up learners to mentors (based on interests, skills they want to learn, location, mentor workload).
    • Code from Campus Neighbor, an online skills-to-need matching website, is being made available to this project to potentially expedite the development process, saving time and money.

Step 4: Develop tools to onboard and support mentors edit

  • Create a mentor resource center
  • Create simple mentor expectations, such as those used for Teahouse Hosts
  • Develop processes to screen out and deal with problematic mentors

Step 4: Trial the system edit

We'll deploy a trial version of the new program and use a mixed-methods approach to measure engagement with current mentors as well as the impact the program has on newcomer experience in Wikipedia:

Experimentation edit

  • A system of semi-automated invitations to this program will be developed to recruit learners, preferably with integration into existing systems such as those used for Teahouse, Snuggle, Wikipedia Adventure, etc.
  • We will invite potential mentors to join the mentorship hub, and to fill in basic info about themselves and the skills they can teach learners. We will directly approach established Teahouse hosts, traditional mentors, and those involved with certain project that fit with some of our tasks (e.g. recruiting regulars from Articles for Creation to aid with skills related to creating a new article).
  • Controlled experiment
In order to look for measurable effects that the project has on newcomer outcomes, we'll perform a controlled experiment. We'll randomly sample from a subset of newly registered editors to invite to the program. These users will be considered part of the "test" condition. Users who were not invited to participate will be considered to represent the "control" condition. This test will help us answer the following questions:
  • Quantitative measurement
To measure engagement with the program, we'll use database queries (via tool labs) and the api to gather information about mentors and learners to answer the following questions?
  • What proportion of eligible new editors participate in the program? ...start/complete a lesson?
  • How many mentors contribute lesson material? ...interact directly with a new editor in the program?
  • Qualitative measurement
We'll use surveys and interviews to explore mentors and learners' perception of the program and to identify changes that should be implemented.
  • Do participants feel like they learned something? ...was it valuable to them?
  • How do learners feel about the short lesson format?
  • Did learners feel like they were matched with an appropriate mentor?
  • How do mentors feel about participating in the program? ...does the short lesson format make sense?
  • What aspects of the program design motivate mentor participation?
  • What processes in the program are unnecessarily difficult?

Budget edit

Total amount requested edit

$22,600.00 USD

Budget breakdown edit

Number Category Item description Unit Number of units Cost per unit Total cost Currency Notes
1 Programmer JavaScript/Bot Programmer compensation for creation of profile matching, interface development, and page curation tools Contract 1 3,000.00 3,000.00 USD
2 Graphic Designer Graphic Designer compensation for working on theme of the site, page layout, and profiles. Contract 1 4,000.00 4,000.00 USD
3 Program Developer Compensation for Soni, committing to 10/hrs. per week in a program development role at USD $20/hr. over the six-month period (26 weeks). Stipend 1 5,200.00 5,200.00 USD
4 Project Manager Compensation for I JethroBT, committing to 15/hrs. per week in a project management role at USD $20/hr. over the six-month period (26 weeks). Stipend 1 7,800.00 7,800.00 USD
5 Researcher Compensation for Gabrielm199, committing to 5/hrs. per week in a research and analysis role at USD $20/hr. over the six-month period (26 weeks). Stipend 1 2,600.00 2,600.00 USD

Intended impact edit

Target audience edit

In this initial pilot, the entire English Wikipedia community could see benefits. Inexperienced users will benefit from more light-weight training and one-on-one guidance as needed, with recognition added to boost morale. Experienced editors who are currently serving as mentors will benefit from a reduction in workload and required commitment. Experienced editors who would like to teach new editors but cannot commit to comprehensive, long-term mentoring can become mentors in this program. Editors proficient in only some areas of the Wiki can take up teaching only their preferred courses in the program. The program has the potential (if more new editors jump on board) to create higher quality edits and more editors.

Community engagement edit

The community we wish to aid are any editors who want to learn specific editing skills from a mentor. We plan to engage existing projects that already bring in many new editors who need help, such as the Teahouse, the Help Desk, and AfC participants who can refer users to our project. We also want to implement a feedback tool to give learners an opportunity to evaluate the program based on its interface, the quality of mentorship, and how well they learned the desired skill. We may also ask what learners what tasks they want to take on as a result of mentorship that they could not do before.

The following are the locations from where we primarily intend to recruit mentors for the project -

Other locations through which we plan to inform and engage the community and any other interested mentors would be through -

Fit with strategy edit

Attrition is a major concern of the Wikimedia movement, until recently, we struggled to understand the constant editor decline. With initiatives like the Visual Editor being rolled out, we are slowly removing the "it's too hard barrier" to editing. By creating a more light-weight approach to mentorship, new users can learn the skills they feel they need (and receive some general support from a more experienced editor) without a requirement to sign up for a long, formal agreement (It's too long, it's too hard), over time creating more editors that contribute more often, with higher quality. Over time, these editors could train others, making the project a sustainable one.

Although this grant will help all English Wikipedia editors, it will be especially effective for newer editors. Research suggests that frustration among editors from having their edits reverted leads to newcomers leaving Wikipedia.[4] Helping new editors experience more success in their on-wiki work will help address a key aspect of editor retention.

Sustainability edit

By making this project more lightweight, with some automation in the process, the project has long term prospects of being self-sustainable. Volunteer burnout for both mentors and learners is minimized by the fact that skills are taught with little long term-commitment. When mentors take a break from editing, their adoptees can continue their program from another mentor with minimal transition problems. Additionally, learners over time may become mentors and teach skills they have learned to others, creating a virtuous repeating cycle that sustains itself. Automated invites can feed learners to the portal, as can links from existing help spaces such as Teahouse.

Furthermore, if the pilot is successful, many of the tools we use in the pilot (e.g. a matching system, profile builder, mentoring space) could be packaged and appropriated for similar mentoring programs in other Wikipedia projects. One advantage is that many of these tools do not require intensive translation, and could be implemented with minimal difficulty.

Measures of success edit

  • Improve editor activity and retention by 50% compared to the existing adoption program (Editors that go through the new mentorship program go on to edit longer, more often, and more diverse contributions than the established baseline for previous learners. This would be established first by analysing editors that have gone through adoption--we can check the transclusions of the {{adopted}} or {{adoptee}} templates for potential candidates to get an idea of their edits before adoption and after. We can then experiment with an alternative model in parallel with the existing process through a sample of say 10-20 editors with the old process, and the same using the new process and compare results to measure how effective the changes have been).
  • 50 learners in the trial period (More newcomers seeking and attain mentorship)
  • 30 mentors in the trial period (More mentors will participate in teaching).
  • Cut the time to find a mentor in half (Fewer learners waiting for a mentor due to a more lightweight design)
  • High satisfaction ratings in surveys of mentors and learners (4 out of 5, 5 out of 5)
  • Higher qualitiative satisfaction when directly comparing between the new and old models (Did you prefer...?)

Participant(s) edit

  • I JethroBT: Greetings! I'm Jethro, and I have some experience in project management in the realm of publishing, which has led me into this project management role here, and have an M.A. from the Univ. of Illinois at Chicago in cognitive psychology for my work in memory. I have worked as a host at the Teahouse extensively and take a strong interest in new editors, their initial experiences with Wikipedia, and what their editing goals are. I also field a variety of questions from new editors through OTRS system about images and general how-to. I see this project as a means by which to help new editors efficiently complete their editing goals and more easily furnish them with a sense of accomplishment.
  • Soni: Hi. This is Soni here, also known by my previous username TheOriginalSoni. I've been editing the English Wikipedia for over two years now in a variety of locations. So far, I've been a host at the Teahouse, a member of the WikiProject Editor Retention and its Editor of the Week program and have provided Aaron with feedback and onWiki help on Snuggle. I also am an OTRS volunteer and a helper at the IRC help channel, and am familiar with Articles for Creation Wikiproject. For a brief while, I also had my adoption programme (apart from being adopted myself) so I know about the current adoption systems too. I'm mostly here because I love helping newcomers and because I think the proposal will benefit from someone well versed with the locations only.
  • Steven Zhang: Hi, I'm Steve. I wrote one of the first adoption programs which is still in use today, and I've mentored editors both voluntarily and as part of mandatory arrangements. I am a previous Wikimedia fellow and worked on reforming the English Wikipedia's dispute resolution processes, so I'm here in this project to help with analysis of metrics and stats, and as an advisor.
  • Gabrielm199: I am a PhD candidate at the Syracuse University School of Information Studies. My research looks at how newcomers learn how to participate in open collaborative projects like the Zooniverse and Wikipedia. I have been a member of the Campus Ambassador program since its initial pilot in 2010 where, in addition to working with college students on how to contribute to Wikipedia, I train new campus ambassadors.

References edit

  1. ↑ Farzan, R., Kraut, R., Pal, A., & Konstan, J. (2012). Socializing Volunteers in an Online Community: A Field Experiment (pp. 325–334). Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work., Seattle, WA.
  2. ↑ Davenport, T. H., & Prusak, L. (1998). Working Knowledge. Boston, MA: Harvard Business School Press.
  3. ↑ Musicant, D. R., Ren, Y., Johnson, J. A., & Riedl, J. (2011, October). Mentoring in Wikipedia: a clash of cultures. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (pp. 173-182). ACM. http://www.tc.umn.edu/~chingren/pdf/papers/wikisym-2011-Mentoring.pdf
  4. ↑ Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2013). The Rise and Decline of an Open Collaboration System: How Wikipedia's Reaction to Popularity Is Causing Its Decline. American Behavioral Scientist, 57(5), 664–688. doi:10.1177/0002764212469365

Discussion edit

Community Notification edit

Please paste a link below to where the relevant communities have been notified of this proposal, and to any other relevant community discussions. Need notification tips?

Endorsements edit

Do you think this project should be selected for an Individual Engagement Grant? Please add your name and rationale for endorsing this project in the list below. Other feedback, questions or concerns from community members are also highly valued, but please post them on the talk page of this proposal.

  • A previous version of this proposal, led by Steven Zhang and the late Jackson Peebles in 2013, received five (5) endorsements which can be found here. I JethroBT (talk) 06:15, 26 March 2014 (UTC)
  • I supported this project last time around and support this incarnation as well. Keilana|Parlez ici 06:16, 30 March 2014 (UTC)
  • Like Keilana, also happy to re-iterate my support of this important project. Orderinchaos (talk) 16:30, 31 March 2014 (UTC)
  • Endorse--I very much support the goal of developing a mentorship process that is both intermediate between the informal/formal dichotomy options of today and something that is more individualized one-on-one, more bottom-up and less top down/formal with respect to rules and format. I don't believe I'm a good judge of whether or not this particular proposal will get us there or not, but generally speaking, if what you are doing is not getting you where you want to go, try something else. And I believe the $20,000+ of Foundation funds would be a small price to pay to "try something else." Cheers. N2e (talk) 23:46, 8 April 2014 (UTC)
  • Endorse I have spent the past year mentoring Miss Bono on English Wikipedia who lives in Cuba and has to deal with extremely problematic internet access. My mentorship has been highly informal and oriented to her specific needs at any given time, rather than following an established curriculum. Our collaboration resulted in a good article, Morleigh Steinberg, and significant contributions to several other articles. Due to her inability to access Wikipedia at this time, she cannot now often edit Wikipedia directly. However, we are in daily communication by email, and I tutor her on colloquial English language usage, again, on an informal ad-hoc basis. She will return to active Wikipedia editing instantly if her access issues are resolved. My experience with this mentorship has convinced me that highly flexible mentorship arrangements focused on the new editors immediate daily needs, rather than hitting structured curriculum benchmarks, is most likely to be successful. I support this venture strongly, and would be happy to share my experiences in detail. Cullen328 (talk) 04:35, 12 April 2014 (UTC)
  • Endorse. This is a sensible and feasible plan for learning more about how to do wiki mentorship effectively — we know very little about that right now — and the people involved are trusted and capable.--Ragesoss (talk) 14:05, 21 April 2014 (UTC)
  • Endorse - This is a well-though-out proposal which will create some much needed structure around the current mentoring process. MrX (talk) 19:40, 26 April 2014 (UTC)
  • Community member: add your name and rationale here.
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Grants:IEG/Reimagining_Wikipedia_Mentorship&oldid=12690626"