User:Shouston (WMF)/Sandbox/Grant Metrics

People and organizations receiving Project and Annual Plan Grants awarded by the Wikimedia Foundation are required to define measures of success and program objectives using metrics that capture the outcomes of their grants. Rapid Grants, Conference & Event Grants, and Travel & Participation Support have their own program-specific requirements. If you have any of these grants, the information on this page does not apply to you; please check with your program officer for more information.

They aren't defining them so much as they are choosing or selecting them. KLove (WMF) (talk) 18:36, 9 August 2016 (UTC))

re:defining vs choosing / selecting. I think we can stick with define and add a sentence afterwards explaining what this means: to choose from a set of stablished metrics, and clarify that there is also room to suggest the grantee's own metrics. María (WMF) (talk) 22:54, 11 August 2016 (UTC)

breaking these groups into bullets rather than continuous text. María (WMF) (talk)
For the following groups:

• Project Grants submitted before September 2016; and
• Simple Annual Plan grants and applications for grants with start dates before January 2017,

grantees and applicants should report on the program objectives, measures of success, and Global Metrics included in their applications. Information and resources for using Global Metrics are in the Global Metrics section of this page.

For these other groups:

• Project Grants and applications submitted after September 2016; and
• Simple Annual Plan and FDC Annual Plan Grants and applications for grants with start dates after January 2017,

applicants should continue to set and report on their own program objectives and measures of success as requested in the application forms, but. Note that you should use the new grant metrics requirements rather than the previous Global Metrics requirements.

The first grant rounds to fall under this new requirement are as follows:

• Project Grants: Grant proposals submitted in September-October 2016 open call
• Simple Annual Plan Grants: All grant applications for grants with start dates after 1 January 2017
• FDC Annual Plan Grant: Applications submitted by 1 October 2016, for grants with start dates after 1 January 2017

For all PEG and IEG grants: Grantees should report against the project metrics and Global Metrics as outlined in their proposal. Information and resources for Global Metrics can be found in the Global Metrics section.

New grant metrics requirement for Project Grants and Annual Plan Grants

For those receiving Project Grants: Grantees are required to include the following metrics.

• 3 shared metrics, which are reported across all Project and Annual Plan grants.
  • If any of these metrics are not relevant to your grant, they may be removed or labeled "Not Applicable" and substituted with an appropriate additional metric.

suggest adding:

• If any of these metrics is relevant to your grant, but you do not have a specified target (either because you have no baseline or because it is not your core target) you may note that you will measure and report it.

This was another type of N/A for proposals folks didn't understand was possible JAnstee (WMF) (talk) 20:45, 9 August 2016 (UTC)

• In addition, at least 2 grantee-defined measures that are relevant to the goals of the grant.

This is a total of at least 5 metrics.

For those receiving Annual Plan Grants: Grantees are required to set program objectives for each program, and to include the following metrics aggregated across all programs.

• 3 shared metrics, which are reported across all Project and Annual Plan grants.
  • If any of these metrics are determined not relevant to the grant in consultation with your program officer, they may be removed or labeled "Not Applicable" and substituted with an appropriate additional metric.
• In addition, at least 2 grantee-defined measures that are relevant to the goals of the grant; you are free to include more if desired. These grantee-defined measures should be relevant to one or more of your programs.

This is a total of at least 5 metrics.

Three shared metrics

	Metric	Definition	Tools & documentation
	Total participants	The number of people who attend events, programs or activities, either in person or virtually. This definition is not as broad as the previously used "Individuals Involved" metrics, and does not include people organizing activities, social media followers, donors, or others not participating directly. See the #What is a "participant"? section for more detail.	This metric may be calculated manually by headcount or a sign-in sheet.
	Number of newly registered users	The number of participants that create new accounts on a Wikimedia project. These include users who register up to two weeks before the start of the event.	Recommended tool & resource: Magic Button Other tools & resources: Wikimetrics Wikimetrics Training Overview
	Number of content pages created or improved, across all Wikimedia projects	A content page is an article on Wikipedia, an item on Wikidata, a content page on Wikisource, an entry on Wiktionary, and a media file on Commons, etc. See the section on defining and counting content pages for more detail.	Recommended tools & resources: For all projects except Commons: Magic Button For Commons: GLAMorous 2 Other tools & resources: Wikimetrics: Content pages created, disaggregated by Wikimedia project

+ at least two grantee-defined measures

These measures (whether quantitative or qualitative) may be used to highlight outcomes not captured adequately by the shared metrics. These measures could capture the outcomes of a single program (e.g. one edit-a-thon), or a set of programs (e.g. a series of edit-a-thons); there is no preset list to choose from when selecting these measures. See the additional resources for grantee-defined measures for help in finding and selecting measures that are relevant to your goals and activities, or consult your program officer.

Resources

Whom can you contact for help?

Please ask your program officer if you need help defining the appropriate metrics for your grant application, or if you need help reporting on your metrics. They may help you directly, or refer you to the Learning & Evaluation team for additional help.

What is a participant?

THIS SECTION IS GOOD. CAN WE ADD A SIMILAR SECTION DEFINING "CONTENT PAGES"? MAY ALLOWS US TO SHORTEN THE INFORMATION INCLUDED IN THE TABLE ABOVE WITHOUT LOSING NUANCE. Winifred Olliff (WMF Program Officer) talk 19:06, 8 August 2016 (UTC)]

+1 Alex Wang (WMF) (talk) 03:38, 9 August 2016 (UTC)

I DON'T THINK THE QUESTION 'WHAT IS A PARTICIPANT IS ANSWERED HERE. INSTEAD IT'S TALKING ABOUT WHAT IS NOT A PARTICIPANT. WOULD SUGGEST ANSWERING THE QUESTION IN ONE BOLD LINE RIGHT UNDER THE QUESTION BEFORE GOING INTO ALL THE OTHER CAVEATS AND DETAILS. SOMETHING LIKE "A PARTICIPANT IS SOMEONE WHO ATTENDS OR PARTICIPATES IN YOUR ACTIVITY." KLove (WMF) (talk) 19:04, 9 August 2016 (UTC))

As noted, the definition of participants used in the new grants metrics is less broad than the previously used previous metric used, called «Individuals Involved» metric, . and Participant is limited to those people participating directly in an event. Based on interviews and feedback from grantees, while the following two groups of people WHAT FOLLOWING TWO GROUPS? I SEE THREE GROUPS LISTED AT THE BOTTOM BUT THAT'S NOT 'FOLLOWING' KLove (WMF) (talk) 18:53, 9 August 2016 (UTC) are important to some activities, they play a role that is distinct from «participants». For this reason, we think it is helpful to capture participants as distinct from these other groups. If particularly relevant, other groups may be reported separately from «participants», perhaps in the grantee-defined measures. If you are unsure of how to determine participants under this definition, please consult your program officer for advice specific to your projects and programs.

Here are some examples of other groups that may be involved with your work, that may not count as participants under this definition: THEY MAY NOT COUNT? DOES THAT MEAN THEY CAN BE INCLUDED OR THEY SHOULDN'T BE? I SUGGEST BEING CLEAR ABOUT WHAT FALLS INTO THE DEFINITION, AND IF FOLKS REPORT ANY OF THE PEOPLE BELOW THEY SHOULD MAKE A NOTE OF THAT. 19:06, 9 August 2016 (UTC))

+1, the definition is not clear. María (WMF) (talk)

• Organizers. These are people who plan, coordinate, and execute an event, activity, or program.
• People who are reached by mass communication, such as:
  • Social media (e.g. Facebook follower and likes, Twitter followers and re-tweets, page views of Vine or Youtube videos)
  • Mailing list recipients
  • Newsletter recipients
  • People who received emails as a part of an outreach or advocacy campaign
  • The audience of TV or radio shows, interviews, presentations, or advertisements
  • The audience at a presentation or lecture given at an external (i.e. non-Wikimedia movement) conference
• People who donate money or in-kind resources to support your activities.

How do I count content pages added or improved?

For the purpose of defining this metric, a "content page" is a page in the Main namespace (namespace 0), except on Commons where a "content page" refers only to pages in the File namespace (namespace 6). This metric should be reported as a single total; if the desired, this metric can also be reported disaggregated by Wikimedia project. However, this disaggregation is not required.

For example, let's say participants at an edit-a-thon create 4 Wikipedia articles, improve 12 Wikipedia articles, upload 4 images to Commons, and add properties to 18 items on Wikidata.

• The grantee should report 38 content pages new or improved across Wikimedia projects.
• The grantee can also (but is not required to) report 16 created or improved articles on Wikipedia, 4 media uploaded on Commons, 18 items created or improved on Wikidata.

Additional resources for defining your metrics

AGAIN, IS THIS ABOUT DEFINING THE METRICS? KLove (WMF) (talk) 19:06, 9 August 2016 (UTC))

• If you are considering collecting qualitative data through a survey, visit the Survey Support Desk.
• To find out which metrics are typically used for common Wikimedia programs (e.g. Education program, GLAM, editathons, competitions), explore the program toolkits, or ask your grant program officer.
• To find general information about measures for reach, participation and content, visit measures for evaluation.

Should we link to Measures for evaluation here? María (WMF) (talk)

Global metrics resources and tools

This is like a total dump at the end, would be good to break it apart moreJAnstee (WMF) (talk) 20:56, 9 August 2016 (UTC)

+1 I suggest making a three column table for the three types of resources, and a button to learn more / contribute. María (WMF) (talk)

The Learning pattern on Global Metrics contains more information, including additional learning patterns on how to collect the data. We recommend using the Magic Button, GLAMorous 2 and PETscan to collect this data, but Wikimetrics (and the Wikimetrics Training Overview) and other tools are available (e.g. the Tool Directory and more).

Sample of boxes I made for the Global Reach page. María (WMF) (talk)

Governments We will be looking to build partnerships with government entities that are keen on promoting educational and open-source projects. We will collaborate with Ministries of Education, Technology, and ICT to increase Wikipedia’s footprint across our regions. Learn more

Nonprofits Mission aligned local nonprofits and UN organizations that have the potential to help us increase reach will also be approached by our regional managers. Many of these organizations already have successful and mature projects that we would highly benefit from. Learn more

Educational institutions Wikipedia is a valuable tool for students and educators around the world. We believe we can leverage our existing mobile partnerships to support schools and universities have adopted technology. Learn more