User:Shouston (WMF)/Sandbox/Grants Impact Analysis FY13-14

Overview of data and analysis

• This report covers grants awarded in Wikimedia Foundation (WMF) fiscal year 2013-2014 (referred to as “FY13-14”), with a specific focus on the Project and Event Grant (PEG) program and the Individual Engagement Grant (IEG) program.
  • The Annual Plan Grants (APG) program has been excluded.

• At the time of this report, a new grants structure was under discussion. As such, analysis of spending was done by both the Current grants program structure (referred to as “Current structure”), and the Proposed grants program structure (referred to as “Proposed structure”).

• For FY13-14, Global Metrics was a not required part of grantee reporting. As such, the aggregated metrics in this report are likely lower due to under-reporting.
  • A small number of these grants were either not complete (set to end in 2015) or did not have an “accepted” final report. For these grants, metrics and outcomes were aggregated from their most recent interim report, draft final report, or grant talk page discussion.

• Analysis of outcomes was meant as an exploration into grantee-reported qualitative outcomes. As such, how often an outcome was reported or the “number of mentions” is excluded from the presentation of each outcome. Instead, quotes from grant reports are offered as examples of each qualitative outcome.

Executive Summary

• $1,322,348 was granted in FY13-14 over all non-APG grant programs, 75% to PEG, 14% to IEG, and 11% to TPS + Wikimania Scholarships

• The Proposed grants program structure leads to an approximately even distribution of total non-APG spending, across Projects, Events, and Simple Annual Plans
  • Growth projects are a small portion of overall non-APG spend in FY13-14 (10% of total non-APG spend; ~30% of total Project spend)
  • Growth projects do not seem to cost more than Seed projects; 86% of FY13-14 Growth projects are under $11K

• PEG and IEG grantee outcomes focused primarily on Content, Participation, and Quality
  • Reporting of Global Metrics (before it was required for all grants) reveals varied levels of reporting: “Total Individuals involved” reported in 70% of PEG & IEG grants, whereas “Absolute bytes” reported in only 8%
  • Only 62% of grants reported content quantitatively; other outcomes reported qualitatively

Recommendations

• Additional reporting or follow-up may be necessary for tool building grants. Current grant reporting period is seems too short to to observe and capture outcomes.

• Offer Event grants a different set of required metrics / outcomes, given content is not the focus (and therefore Global Metrics is only partly applicable)
  • All FY13-14 PEG, IEG Event grants seemed to have some component of the following: Experience sharing (incl. sharing and gaining knowledge), Networking (e.g. making contacts) , Creating/starting new collaborations or projects

• Additional discussion or investigation of the following:
  • Applicability and usefulness of “Absolute bytes”, to understand if a different quantitative metric is more appropriate.
    • Additionally, assess the usefulness of the learning question on motivation - Does it elicit and capture the intended information?

(Embed the graph about GM usage)

• • Indicators of “Increasing awareness”, particularly whether media mentions & social media following should be an indicator.
  • Storytelling as an effective way to communicate outcomes in annual plans; activity-focused reporting format for annual plans does not seem effective

Analysis of Spending

Analysis of Spending included all non-APG grant program spend, i.e. spending for PEG, IEG, TPS, and Wikimania Scholarships awarded in FY13-14.

Key insights:

1. Proposed grant program structure results in an even distribution of FY13-14 non-APG spending
2. Majority of Project spending is on Seed projects (70% of total spending & # of grants)
3. ~$30K limit characterizes all Project grants
4. Most Project grants are completed in a year. Extensions have large impact in IEG; minimal in PEG

Highlighted insight: (Choose top insight to expand upon and include graphs)

Analysis of Outcomes

(State Grants programs included in analysis).

Overview

(Re-create slide 12)