Grants:APG/Learning/Post-Decision Survey/2015-2016 Round 1

APG Application Process

Post-Decision Feedback Survey


Round 1 2015-2016 Results

Summary

edit
The goal of the FDC is to make decisions about how to effectively allocate
movement funds to achieve Wikimedia’s mission, vision and strategy.
Following board Annual Plan Grants (APG) funding decision for 2015-2016 Round 1, Community Resources surveyed participants of the Funds Dissemination Committee (FDC) process from February 11-24 2016 with the purpose of identifying what is working well and what is not working well for the different participants in the process.

This page highlights the key findings of the survey based on 17 responses comprising 10 Organisations, 4 FDC Members and 3 WMF staff.

  1. Overall, 93% of participants are Satisfied or Very Satisfied with this rounds’ APG / FDC process. While all stakeholders are Satisfied or Very Satisfied at the process, 12.5% of applicants are Dissatisfied.
  2. The FDC increased overall spending over 2014–15 R1, and increased the average award size.
  3. Participants reported spending Less Time on Average on the FDC / APG Process than the previous year.
  4. 75% of applicants familiar with other funding sources thought that the FDC process required significantly more investment in time and money.
  5. Majority of applicants agreed that their participation in the FDC process benefited their organisation in some way.

Context of 2015–2016 Round 1

edit
  • This round had eleven compliant applicants including one chapter participating in the FDC process for the first time.
  • The FDC awarded funds to all eleven entities, with amounts ranging from $75,000 to $1,346,000.
  • One entity received 36% of total FDC funding recommendations.
  • FDC approved a total of 90% of requested funding.
  • All entities received 80% or more of their requested funding.
  • On average, organisations received 96% of requested funding; 64% of whom received their full requested amount.
  • Entities that received APG funding the previous year were granted an average 11% budget increase.

Results

edit

Time investment

edit
Participants are spending less time on Average on the FDC/APG process
  • Participants spent 16% less time on the process, an average of 79 hours spent this round.
  • Organisations specifically however, are spending more time on the application: 38% of whom feel they have devoted Too much time to the process.
  • Average time spent by survey respondents in this round was 78.75 hours, 33% of whom felt this was too much, and 60% felt it was about right. 7% felt it was too little time.



  • Individual applicants in the organisation's self reported spending an average of 112 hours participating in the process.
  • While most respondents spent between 70 and 100 hours, 2 people indicated they spend 200 hours and 1 person reported that they spent 16 hours in total on the application process.

The FDC assessed eleven dollar-allocated proposals each first round between 2012 and 2016.

Other investments

edit
Majority of applicants who have sought funding from other sources found that the FDC process required significantly more investment

Basically we ask external donors to fund a given set of programs which they look into. The FDC, on the other hand, looks at all of our programs no matter the amount of money we're asking for. This is highly disproportionate.

— Applicant


  • Half of the entities that responded had never sought funding from other sources (e.g., the Project and Events Grants Program, external donors).
  • Three-quarters of respondents who had engaged in similar levels of funding thought the FDC required significantly more investment in time and money.
  • Probed about why it required more investment, excerpts include: “Bilingual texts, translations, increased expectations concerning design, layout” and “a very detailed methodology”.

36% of organisations spent money on outside help, reporting consultancy or translation services costs associated with their application; an average of $963 ranging from $280 to $2,000.

Allocation size

edit
The FDC increased overall spending over 2014-15 R1, and increased the average award size
  • The FDC reviewed applications from 11 compliant organisations, one of whom was a first time applicant.
  • The FDC awarded funds to all 11 organisations, approving amounts ranging from $75,000 to $1,346,000, granting a total of $3,770,000 to organisations.
  • On average, the FDC approved 9.7% more of proposed budgets from each organisation than 2014-15 R1.
  • Excluding WMDE and WMUA (the new entrant), the FDC increased spending on Annual Plan Grants by 0.33% and approved 11% more of proposed budgets from 2014-15 R1.

It is worth noting that WMDE requested for a budget increase of 79% to undertake software development activities for the Wikimedia movement.
*Average as used in the chart does not include WMUA who are a new applicant to the APG process.

Benefit to organisations

edit
88% of Applicants agreed that their participation in the FDC process benefited their organisation in some ways

Members and Wikipedians really don't care about Foundation processes, which are seen as cumbersome and nittpicky. They're happy to let us deal with it. The real beneficiaries of our funds are non-Wikipedians (either institutions or newcomers) and they really have no idea how things work internally (nor should they).

— Applicant


  • Clarity about Goals and plan: 80% Agreed or Strongly Agreed that ‘The FDC process helped them achieve greater clarity about their annual goals and plan’.
  • Stronger Plan with Potential for Impact: 80% Agreed or Strongly Agreed that ‘The FDC process resulted in a stronger annual plan with greater potential for impact’.
  • Transparency about Annual Plan and Budget: 50% Agreed or Strongly Agreed that ‘The FDC process increased transparency within their organization and with members about their annual plan and budget’. 30% disagreed, one of whom wrote that “None of our volunteers reads exentsive meta pages in english, the level of detail and strategic planning is beyond their interest”.




Impact on Community and the Movement

edit
Majority of respondents agreed that the APG / FDC allocations process has positive impact on Community and the Movement



  • 87% Agreed or Strongly Agreed that APG/FDC process ‘produces greater movement-wide information about what it takes to achieve impact’. 13% respondents Disagreed or Strongly Disagreed.
  • 87% Agreed or Strongly Agreed that APG/FDC process ‘is putting movement funds to the most effective and efficient use’. 13% respondents Disagreed or Strongly Disagreed.
  • 73% Agreed or Strongly Agreed that APG/FDC process ‘is increasing community voice in how movement funds are spent’. 20% respondents Disagreed or Strongly Disagreed.

Communicated Expectations

edit
Majority of respondents agreed that this rounds’ process was Transparent with clearly communicated Expectations


  • 87% respondents Agreed or Strongly Agreed that at this rounds’ APG/FDC process “Participants received responses on their questions”.
  • 87% respondents Agreed or Strongly Agreed that at this rounds’ APG/FDC process “Expectations and deadlines were clearly and effectively communicated”. 7% Disagreed.
  • 80% respondents Agreed or Strongly Agreed that at this rounds’ APG/FDC process was “Transparent” and “Application requirements of participants were reasonable and achievable”. 13% Disagreed to both assertions.


FDC deliberation resources

edit
Three out of the Four FDC members who responded found each of the resources provided them to be Important or Very Important
Please rank the importance of the following Unimportant Neutral Important Very Important
Questions/comments on proposal talk pages by FDC members, staff, community - - 50% 50%
Financial analysis (Google spreadsheet with lots of numbers, private to FDC) - 25% 25% 50%
Staff proposal assessment on Meta - - 75% 25%
Financial report (narrative document private to FDC) - - 75% 25%
Achievement reports (private to FDC, traffic lights on programmatic impact of applicant to date, looking at potential for impact in proposal) - 25% - 75%
Early proposal assessment tool, & FDC Skype call on survey results (April) 25% - 25% 50%
  • All FDC respondents found “Questions/comments on proposal talk pages by FDC members”, “Staff proposal assessment on Meta” and “Narrative Financial report” to be Important or Very Important.
  • 3 out of 4 found the “Financial analysis spreadsheet”, “Achievement reports” and “Early proposal assessment tool, & FDC Skype call” to be Important or Very Important.
  • One FDC Member thought that the “Early proposal assessment tool, & FDC Skype call” was Unimportant.
  • Asked about what other document might be useful, one respondent wanted to see “a comparison of the ‘value’ of the money in each of the applicant countries”; adding that “Exchange rates/inflation are one thing, but the cost of living is important to know what "good value" actually means in their context.”

Themes and recommendations

edit

Improve Communication

edit
  • Increase communication between staff to avoid sending entities conflicting information
  • Provide avenues for facetime contact between entities and staff
  • Increase frequency of check-ins using multiple mediums
  • Delegate an entity member to be on standby during FDC deliberation days

Application process

edit
  • Set up model application templates
  • Make application multilingual
  • Use of plain simple language
  • Lessen time required to create the application (especially in calculating the metrics)
  • Adjust global metrics for the different projects
  • Prioritize application requirements for grantees

Other themes

edit
  • Improve clarity about details required in Budget and Reporting templates

Appendix

edit

What was done well?

edit

Increased focus on the past and anticipated outcomes of organizations.

— Response from 1 WMF staff
  • Support received:
  1. 3 grantee respondents indicated that FDC staff were supportive
  2. 2 organisations and 1 FDC member wrote that WMF staff were supportive
  • The following were each expressed once by grantee respondents:
  1. Regular Check-ins
  2. Advanced feedback on draft proposal
  3. Relatively quick turnaround
  4. Clear expectations and procedure
  5. Democratic process, involves community
  6. FDC were accessible
  7. FDC understanding of local context
  8. Clear assessments by staff and FDC
  9. FDC staff have developed a good sense of role in supporting the committee

What can be improved?

edit
  • Three respondents wrote about ‘Clarity of Expectations’
  1. “… to know what programs are priorities, which aren't” — Grantee
  2. “… to have a better sense of what the Foundation expects from the chapters” — Grantee
  3. “Understand which pieces of information are ‘need-to-have’ versus ‘nice-to-have’ for the FDC” — WMF staff
  • Two grantees wanted to see changes made to templates.
  1. Reporting templates: “Sometimes you don't know which is the best way to tell your story or how is it expected to be told”
  2. Budget templates: “It was not clear to what extent will the budget need to be detailed”
  • These opinions were each expressed once by grantees
  1. Introduce multilingual process
  2. Follow up on multi-year funding circles as intended in 2014–15 round 1
  3. More proactive feedback on what works and what doesn't across chapters.
  4. Many chapters have done great submissions in the last years; to take them as a role models to help other chapters to not get lost in the process would be great.
  5. Very limited opportunities to discuss and explain in depth with WMF staff and FDC
  6. Global metrics could be better adjusted to different projects.
  7. Greater consensus between staff and FDC volunteers, to create a more coherent message to the grantees.

How can WMF staff support you as you implement your APG in 2016?

edit

Two to three times a year check in calls

  • 4 out of 8 grantees mentioned check-ins
  • Other suggestions expressed once by grantees
  1. More workshops for staff during Wikimedia Conference
  2. More face time with people with cross movement knowledge of activities.
  3. Pings, copy edit and quick tips
  4. More communication between foundation staff to avoid receiving conflicting messages [paraphrased]

For the full results, see full survey results