Wikimedia Conference 2016/Program/15

15: Global Metrics Retrospective and Revision

How to move forward

• 
  Sati presenting preliminary findings from interviews and survey on Global Metrics

What was this session about?

A retrospective on Global Metrics and review of preliminary findings from community interviews & survey. Gathering ideas for "Global Metrics 2.0".

What are the next steps to be taken?

• Additional information will be posted on the retrospective project page.
• On July 30, 2016 the Resources team has published a proposal for new so-called "Grant Metrics" (instead of "Global Metrics")

Who is the person to reach out to?

Sati Houston & Jaime Anstee

Photos

see the Commons category

Slides

Slides

Original Session description

• Part 1: Presentation (1 hour)

A retrospective on Global Metrics and review of preliminary findings from community interviews & survey. Will cover the current use of Global Metrics, the benefits and issues reported by grantees, Committee members and WMF Grant staff.

• Part 2: Roundtable Discussions

Small group roundtables focused on gathering input on three areas:

• The ways to measure specific outcomes, e.g. What metrics are people already using to measure retention? Quality? Partnerships? Other things that can be surveyed?
• The positives/negatives about current resources that support this data collection, as well as suggestions for new resources to be created.

Desired Outcome

• New or improved resources to support grantees and Committee members in using qualitative and quantitative data effectively
• Feedback on what Global Metrics should turn into.

Session Format

Working

Speakers

Sati Houston, Jaime Anstee

Summary of the session

Sati and Jaime present the review with the help of the slides.

A participant asks how global metrics has led to numeric outcomes. Have people focused on short term outcomes instead of longer term outcomes because of these metrics? Sati answers that even before the GM, 70% of PEG/IEG already reported on such kinds of metrics. Longer term outcomes are still in the proposals.

A participant says that not all photos on Commons should be used in articles. Usage should not be the ultimate goal. Jaime replies that it depends on what the project goals are. The goals should be reflected in appropriate metrics. The participant asks about cultural metrics (gender gap etc). Jaime says that projects are judged in a context, never de-contextualized.

A person asks who was interviewed for compiling the review. Were people within WMF interviewed? Sati answers that initially a few PEG-grantees/staffers/FDC+GAC-committee members/organizations were interviewed. For most IEG-s it is not relevant to use GM.

A participant says that communicating through GM was not good. It was not said that GM is only about Wiki-projects and not about re-use etc. Another participant adds that many APG applicants feel that GM is mandatory but not very useful. They feel that they have to squeeze their projects into GM or else they don’t measure at all. The communication of GM does not provide any alternatives.

Another participant says that almost everything that their chapter does is outside of GM; for example partnerships with institutions.

A person says that collecting data is a huge drain and writing reports are seriously difficult as it takes a lot of time. Sati answers that they want to talk about not only metrics but the whole eco-system (tools etc). The Grants program is going to be restructured. Katy from WMF adds that of course GM is time consuming to run. But one thing that is missing from the story is the point of view of the committees and staff judging the projects to fund. Before GM there was nothing. The participant raising the problem agrees that the issue of using donors’ money is important but the reporting process should be made easier.

A participant points out that GM are not useful for communicating with and reporting to partners outside our movement. They have different goals and they are not interested in our content measurements. Sati agrees that partnerships are currently the biggest area that GM does not cover.

Sati says that a participant has asked them why GM are called Global Metrics. And to be honest, they do not know. A person says that they are supposed to be for internal use, not global. Sati adds that the broader shift for storytelling is happening and the evolution is towards combining the numbers and stories.

Sati says that a good balance needs to be found and then iterated. A person says that iteration should be done in a smaller group instead of the whole community because that is a heavy burden on staff and volunteers. Sati says that maybe we need to find different metrics for different target groups or different sizes/capacities/needs.