Learning and Evaluation/Plan your program metrics/Retain community

Part of the Wikimedia Resource Center

Learning and Evaluation

• Start
• Plan
• Report
  • Overview
  • Setting goals and measures
  • Plan your program metrics
  • Global metrics
  • Capturing your metrics
• Contribute
• Past News and Events

Back to Measures for Evaluation Retain Community Goal: Increase the number of active editors Example Programs, Projects, or Events Invite long-time editors to come back, like On-wiki Editing Contests Invite existing editors to contribute in a new way, like Wiki Loves Monuments or through Hackathons Invite existing editors to collaborate and continue editing, like WikiProjects

More Measures |1 |1 |1 Recruit New Users Awareness of Content Needs Awareness of Wikimedia Convert New Users Diversity of Participation Access to Wikimedia Retain Community Participation of Content Experts Reputation of Wikimedia Quality Productivity per Editor References and Sources

	Measurement	Tool
Inputs

Number of organizing and supporting staff and volunteers (ambassadors, mentors etc.)

Count of supporting staff and volunteers

Meeting Records, Logs, Reported estimates

Hours invested by staff and volunteers organizing and executing

Estimate of hours invested

Calendar Records, Logs, Reported estimates

In-kind donations (prizes, etc),

Estimated value in currency (comparable cost)

Funds or money used

Budget accounting

Budget spreadsheet

Participant pool to draw existing editors from

Target usernames

Project or event page or other username list

	Measurement	Tool
Outputs

Number of participants involved in the project or program event

Count of total number of participants (Organizers and attendees)

Project or event page or other participant tracking

Number of users

Count of participants with usernames

Project or event page or other username list

Number of new account users participating

Number of participant usernames with less than 10 edits

Wikimetrics, Threshold (from account registration forward) or Edits (from event date forward)

Number of existing editors participating

Count of participants with 1 or more edits, under their username, in their lifetime

Wikimetrics, Threshold (from account registration forward) or Edits (from event date forward)

Amount of text improved during event

Number of bytes added or removed

Wikimetrics - Bytes Added

	Measurement	Tool
Short term outcomes

Participant motivation to contribute

Participant self-report

Poll or questionnaire

Participant skills (editing skills, policy knowledge, process awareness)

Participant self-report AND/OR direct observation

Poll or questionnaire AND/OR quizzes/tasks with monitoring (i.e., human observer or mechanical observation via Wikimetrics/Mediawiki)

Number of new institutions/partnerships

Count of new institutions and partners working to implement Wikimedia programs

Direct observation/Program documentation

	Measurement	Tool
Intermediate outcomes

( Productivity )

Number of participants who became contributors (10+ edits over lifetime)

Number of Contributors (10+ edits over lifetime)

Wikimetrics, Threshold (from account registration date forward) or Edits (from event date forward)

Amount of text improved since the program event

Number of bytes added or removed

Wikimetrics, Bytes Added

( Retention )

New Editor retention

Number of new editors (less than 10 edits) survived (i.e., 1+ edit in past 30-days) at 30 days and 3, 6, and 12 months

Wikimetrics, Survival (from account registration date) or Edits (from event date)

New Editor Conversion to Active Editors

Number of new editors (participants who had only 1-9 lifetime edits) became active editors (i.e., 5+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling New Active Editors (from account registration date forward) Rolling Active Editors (from event date forward), or Edits (from event date forward)

Number of Survived Editors

Number of existing editors (participants who had 10 or more lifetime edits) which survived (i.e., 1+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling Active Editors (from event date forward), or Edits (from event date forward)

Number of Active Editors

Number of existing editors (participants who had 10 or more lifetime edits) became active editors (i.e., 5+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling New Active Editors (from account registration date forward) Rolling Active Editors (from event date forward), or Edits (from event date forward)

Number of Very Active Editors

Number of existing editors (participants who had 10 or more lifetime edits) became very active editors (i.e., 100+ edits in past 30-days) at 30 days and 3, 6, and/or 12 months

Wikimetrics, Rolling Active Editors (from event date forward), or Edits (from event date forward)

Number of repeat participants

Number of individuals (on or off-wiki) who are returnees

Project or event page or other participant logs

Number of repeat contributors

Usernames with 1 edit during a program or event who are returnees

Wikimetrics, Edits (from event date forward)

Number of participants who increase participation in administrative roles, committees, chapters, etc.

Participant self-report, online username tracking

Questionnaire and/or manual tracking