Grants:APG/Proposals/2013-2014 round2/Wikimédia France/Staff proposal assessment

Translate this page

Wikimédia France received a funds allocation in the amount of €600,000 (~US$826,062) in 2013-2014 Round 2. This funds allocation was recommended by the Funds Dissemination Committee on 24 May 2014 and approved by the WMF Board of Trustees on 28 June 2014.

Applicants should read this document to familiarise themselves with the assessment criteria; however, it should be edited only by FDC-related staff. The staff will use the document to set out their objective assessment of each funding request from eligible entities – specifically, to determine the extent to which plans align with the mission goals of the Wikimedia movement, and the extent to which the organization is well-placed to execute those plans effectively. The staff will base this report on proposals from entities and input from the community and other stakeholders (including, where required, deeper due-diligence on the organization's previous track record within the Wikimedia movement). The staff will provide completed reports to the FDC, which will then make a recommendation to the Board of Trustees on funding allocations.

Methodology behind the 2013-2014 Round 2 FDC staff assessments

The FDC staff assessments provide a snapshot of the staff review of Annual Plan Grant (FDC) proposals, and identify both strengths and concerns with the work proposed.

We did this through both portfolio analysis - looking across proposals and calibrating across different organizations - as well as assessing the context of each individual organization and its proposal.
Inputs into the staff assessment process this round include:
- The proposals themselves, with the attached documents (strategic plans, annual plans, detailed budgets, supplementary documents)
- Past and current reports (quarterly progress reports on annual plan grants, other grant reports, annual financial and narrative reports)
- A portfolio view of financials across proposals for both rounds this year Grants:APG/Proposals/2013-2014_round2/Financial_overview and Grants:APG/Proposals/2013-2014_round1/Financial_overview.
- Internal financial, program evaluation, and grants compliance inputs from the WMF Finance, Programs, Legal, and Grantmaking teams.
- Detailed comments and questions collected during the Community Review period, which are also summarized in these staff assessments.
We used a few key principles for the assessment, across all proposals. These are not new, and are detailed in the narrative and scoring sections of the assessment. They include:
- Impact on Wikimedia projects and the global movement, commensurate with funds requested. We looked for the rationale behind the work organizations do, aligning them to the Wikimedia strategic priorities and goals. We looked particularly for direct and indirect impact of this work on Wikimedia projects; for example, growth in contributors and content donation. Most importantly, we looked at whether the funds requested were justified in the context of current and potential impact of the proposed work.
- Organizational strategy, leadership, and governance. We looked for evidence that an organization had a good understanding of its own context, and the needs, opportunities and challenges of the communities it seeks to serve or partner with. We looked for board, staff and volunteer leadership that is committed, effective and engaged with these communities and practising movement values of openness and transparency.
- Rates of growth and financial sustainability. In this round, no applicants asked for funds within the recommended maximum growth rate in FDC funding of 20%, which is itself a significant rate in many contexts. The minimum growth rate in the FDC/movement funding proposed is 22.7% and the maximum proposed is 216%. Given the challenges faced by some organizations with raising funds externally, there are significant concerns about the long term financial sustainability of this level of dependence on movement resources in areas where other funding may be available.
- Program design, learning, and evaluation. We recognize and appreciate that organizations in this round aim to link their work directly to outcomes on the Wikimedia Projects they are targeting and to work toward aligning their high-level goals across programs; at the same time, this work does not always coincide with clear links between these high-level targets and the direct outcomes of program activities, which may make it difficult to understand the effectiveness of some program activities. We appreciate the good work that organizations have done in this area so far, and urge organizations to continue to work toward making the links between program activities, objectives, and high-level targets clear.
- Inclusion of exceptional organizations. Two organizations were included in this round that required exceptions. The Centre for Internet and Society is the first non-Wikimedia organization to participate in the FDC process, and was made eligible through a resolution by the WMF Board of Trustees. As a result, detailed attention was paid to the financial data presented by CIS to understand the work specific to the Wikimedia movement that will be funded by the FDC, in addition to understanding their broader budget context, and only this work was described in detail in the CIS proposal and analyzed in the staff assessment. Other than this, the process for assessing the CIS proposal has been the same as that for other proposals. The Wikimedia Foundation also participated in this round of the FDC process; at the FDC’s request we did not complete a staff assessment for the Wikimedia Foundation.

Note: We shared our nearly final draft assessments with all FDC applicants a day ahead of publishing them, so that organizations had some lead time for sharing these with their Board and staff. We then took all responses into consideration where possible, making factual changes and clarifying our statements where necessary. We encourage organizations to note all substantive comments and concerns on the discussion pages of these assessments.

Proposal overview

Name of eligible organization	Wikimédia France
Summary staff assessment [substantial concerns: none/some/many]	Some concerns
Total amount proposed to the FDC for the upcoming year annual plan in currency requested	600,000 EUR
Estimate of total amount proposed to the FDC for the upcoming year annual plan in US dollars	826,062.00 USD
Exchange rate used to calculate estimate in US dollars and date accessed (mm/dd/yy)	1.37677, 1 EUR to USD (2014-03-01)
Date of proposal submission	2014-04-01
Adherence to proposal process [yes/no]	yes
Adherence to legal, regulatory, and policy regulations [yes/no]	yes
Adherence to US anti-terrorism laws [yes/no]	yes

Financial summary

A detailed summary of the financial information from all proposals is here: financial overview document.

	Projected 2013	Proposed 2014	Proposed change (as a +/− %)
FDC or WMF funding	US$550,921.40	US$826,062	49.9%
Overall budget	US$895,336.94	US$1,322,942.42	47.8%
Staff costs	US$616,717.24	US$616,717.24	0

Annual plan

Overview of staff assessment

Strengths

What are the top strengths of the plan?

Wikimédia France (WMFR) is testing innovative programs and approaches (for example, its regional approach), while also building on past successes and expertise in areas in which it has an established track record (for example, institutional partnerships).
WMFR is aiming to focus on the Wikimedia mission by directly linking offline work to online impact, and is currently measuring and tracking direct results of its program activities and plans to align these programs across broad indicators related to the Wikimedia Projects; in addition, it is mapping its costs to its programs through the budget and its quality framework.
WMFR is a movement leader in actively and openly sharing learning with the movement through detailed reports and frank conversations, including discussions about failures and challenges.

Concerns

What are the top concerns with the plan?

WMFR is planning numerous activities and designing its new and tested programs on a significant scale; as a result the plan may not be feasible, especially if WMFR’s revised staffing structure does not address past inefficiencies as expected.
WMFR acknowledges that volunteer engagement is an ongoing and significant challenge, while at the same time presenting a plan that relies on significant volunteer resources; it is unclear how WMFR will find the volunteer resources needed to execute this plan.
The funds requested are not commensurate with the results proposed, and WMFR has not made a clear case that more funding will lead to improved results: while the scope of this proposal is very ambitious, examples of successful past activities described in this proposal are all lower-cost small-scale programs.

Strategic alignment

To what extent does the plan address the strategic priorities of the movement?

Alignment with the strategic priorities

WMFR’s programs, especially its work on institutional partnerships (Axis1) and its work to support contributors (Axis 4), are strongly aligned with the strategic priorities of Increasing participation and Increasing quality, and its programs are also aligned with Increasing reach and Encouraging innovation.
Significantly, WMFR’s overall strategy is to focus on the Wikimedia mission through linking its offline work to online impact.
Some program activities do not have a clear enough rationale for strategic alignment: particularly, WMFR’s international work (Axis 3). For example, it is not clear how offline work will lead to Increasing participation, since those reading offline Wikipedia may not make significant contributions to the Wikimedia Projects. At the same time, it is important to mobilize local volunteer leadership (a strategy that may directly contribute to Increasing participation), but WMFR does not explain how it plans to do this. With other international activities, the proposal does not make it clear how the activities will achieve any direct impact on the Wikimedia Projects.

Program objectives and measurable outcomes

We appreciate that WMFR is being innovative and focused in defining targets and understanding results that are directly related to results on the Wikimedia Projects, and we appreciate that WMFR is creating a system to show how it plans to achieve specific results related to each program and to measure them. As WMFR implements this system, this may become a key learning for other Wikimedia organizations that wish to establish a clear link between program objectives and online impact.
Details about WMFR’s plan for tracking direct outcomes of its program activities are not provided in this proposal: WMFR’s detailed plan for tracking these outcomes is to be developed in the new fiscal year. However, in its recent progress reports, WMFR is making good progress tracking both outputs and outcomes resulting from some of its program activities.
WMFR has made great progress in aligning its measurable targets across programs and in assessing the costs of each program to the organization (organizational resources) through its quality approach. Sometimes, this global approach to targets makes it difficult to understand the link between objectives and specific program activities. For example, it is difficult to understand how a Training session and Community management will lead to 5,000 edits in the main namespace without more explanation, and it is difficult to estimate whether or not this target is realistic.
Some programs do not include SMART objectives: particularly, WMFR’s advocacy work (Axis 5) lacks specific targets. This may be understandable as the results of some activities, like raising awareness, can be inherently difficult to measure, but it is not clear from the proposal how WMFR will know if its work in this area is successful or not.

Strategic planning

WMFR has a brief strategic plan that guided the development of this proposal. As part of this strategic plan, WMFR is implementing a quality approach that assesses WMFR’s programs with respect to impact on the Wikimedia Projects, cost to the organization (WMFR), and social impact (for example, media attention). WMFR has worked diligently to align its new proposal with the new strategic plan in this proposal cycle, using the methodology of its quality approach to define program objectives and targets.
We need more insights into WMFR’s methods for discovering and responding to the needs of its community. Many programs are intended to support contributors directly, but there is no clear rationale or needs assessment. We encourage WMFR to share more information about how it understands community needs and how it decides to start new programs or continue others based on these needs. This would not only strengthen the rationale for these program activities, but could also provide valuable learning for other Wikimedia organizations.

Context of the organization and community

To what extent does the context of the organization (e.g., community participation, fundraising context, sociopolitical environment) enable or inhibit the organization's potential for impact? To what extent does the organization's experience enable or inhibit the organization's potential for impact?

Online focus and potential for global impact commensurate with requested resources

WMFR’s programs that target French Wikipedia and Wikimedia Commons have potential for global impact if executed successfully, since these programs are well-aligned with Wikimedia strategic priorities and could benefit large numbers of contributors and readers. French Wikipedia is one of the top ten largest Wikipedias at 1.5 million articles, and French is spoken by about 338 million speakers worldwide.
WMFR’s work in Africa, if executed effectively, may have the potential to lead to impact in a region in which the Wikimedia movement has not yet been active.
WMFR’s programs span an impressive array of French-language and global projects, encompassing all of the different Wikimedia Projects and leveraging them in creative ways. This attention to a diverse group of projects may be a good opportunity for movement learning on different strategic entry points for growing our community and expanding our content.
WMFR seeks to engage in innovative approaches that can be adapted by others in the movement if successful. To this end, WMFR is building tools and documenting its learning carefully. For example, WMFR’s localised strategies for building community may lead to important learning for the movement, as many organizations struggle with similar challenges of broadening their outreach.
While WMFR’s work has great potential for global impact, the actual amount of FDC funding and the total budget planned is significant; more results on a global scale may be needed to justify this significant investment.

Financial sustainability and proposed growth

WMFR plans to spend 650,317 EUR (~US$895,337) out of its current budget of 697,008 EUR (~US$959,620), and is requesting an increase of 263,895 EUR (~US$363,323) in its overall budget to 960,903 EUR (~US$1,322,942), or a growth rate of 47.8% based on projected spending, WMFR is requesting an increase from 400,155 EUR (~US$550,921.40) to 600,000 EUR (~US$$826,062) in FDC funding, or a growth rate of 49.9%. This indicates that WMFR plans to grow the percentage of its expenses covered by FDC funding from 57% of its total expenses in the current year to 62% of its total expenses in the upcoming year, and so is marginally moving away from rather than towards its goal of diversifying funding sources. WMFR’s explanation is that with the reorganization process that took place over the past 18 months, their budget levels have actually stayed stable. In addition, they are planning to cap budget growth for the next two years at 6%.
WMFR’s proposed staff costs at 447,945 EUR (~US$616,717) are 46% of total expenses. After staff expenses, WMFR’s travel costs across programs are 117,130 EUR (~US$161261, or 12% of its overall budget), and its next largest areas of spending across programs are office equipment at 76,734 EUR (~US$105,645) and communications costs at 63,806 EUR (~US$87,846).
WMFR’s highest program costs are for its regional program (Axis 2) at 163,813 EUR (~US$225533) including staff and 92,000 EUR (~US$126,663) for costs other than staff, and its international programs (Axis 3) at 121,358 EUR (~US$167,082) including staff and 68,660 EUR (~US$94529) for costs other than staff.
WMFR reorganized its staff structure in order to address inefficiencies and plans to substantially increase program activity without an increase in staff.

Feasibility and risks for the annual plan

How feasible is the annual plan? Specifically:

To what extent are the budget, staffing, availability of volunteers, and other resources realistic for achieving the anticipated outcomes?
Are the timelines realistic for the proposed initiatives and associated activities?
Are there acknowledged and/or unacknowledged risks in the proposal?

Feasibility strengths

WMFR’s board has addressed leadership transitions and challenges energetically in recent years, and has been open in sharing experiences with the movement. WMFR’s leadership (both staff and board) is highly motivated and actively seeking to improve its ability to have and to measure impact on the Wikimedia Projects. WMFR has successfully hired a new Executive Director, with a positive impact on the organization, corresponding to increased program activity.
WMFR is building on its past work with successful GLAM partnerships, and has the proven expertise to continue pursuing this work and to begin to develop models to be used by others in the movement doing similar work.
WMFR has shown willingness to adapt and change when current strategies are not working, and is agile enough to take advantage of new opportunities.
Through its past work and reports, WMFR has shown that it is a movement leader by candidly sharing both its successes and failures with the broader movement and seeking continuous improvement in its reporting processes.

Feasibility concerns

The scope of this plan, including the number of activities proposed, may make it difficult to execute: each of the six programs proposed includes many sub-program groups and WMFR’s online areas of focus encompass the entire spectrum of Wikimedia projects. At the same time, WMFR intends to continue to improve its systems for monitoring and evaluating its work, which also takes significant time, resources, and focus.
WMFR has revised its approach to staffing: rather than requesting new staff, it has reorganized its existing staff resources to support significant growth in its program activities. We appreciate this, but staff time and focus appears spread across many programs, activities, and approaches, which could present a feasibility risk.
According to the tables provided, the programs proposed will require a significant amount of volunteer hours to succeed. WMFR acknowledges that volunteer engagement is a challenge in its work and is taking steps to address this challenge; yet, it has proposed a plan that relies heavily on volunteer participation without establishing proven strategies for deepening engagement.
WMFR is implementing new programs like its regional approach on a very large scale rather than testing these new approaches as pilots; focused pilots may give the organization time to review and understand opportunities and challenges before implementing projects at scale.

Summary of expert opinions and/or community commentary

Summary of the opinions of subject-matter experts on the plan

No expert commentary or opinion was requested for this plan.

Summary of community commentary on the plan

WMFR is engaging thoughtfully in the proposal process, and has put significant effort into creating this proposal and in engaging with FDC staff and the community on the proposal form discussion page. At the same time, community engagement in the FDC process has been relatively limited, perhaps because the FDC proposal form is in English. We recognize that this is a challenge for many organizations participating in the FDC process.

For detailed comments from the community, the FDC and FDC staff, as well as responses from the organizations themselves, please visit the proposal form discussion page. The summary offers a brief overview of the questions or comments from community members, and is not an endorsement of the views expressed on the proposal form discussion page.

A community member expresses support for the plan, including volunteer-led activities, strong relationships with partners, diversified funding streams, and a strong track record. On the other hand, the same community member expresses concerns about investing more in supporting well-represented languages in wealthier countries.
A community member expresses concerns about creating new language Wikipedias.
A community member expresses concerns about WMFR’s staffing plan, since staff will be involved in many activities under the current plan.
A community member expresses concerns that the activities in this proposal may not be sufficient to address cultural challenges to gaining new contributors in the French Wikipedia community.

Scores, comments and feedback

Scores provide only a preliminary assessment for consideration by the FDC. The rubric for scoring is below.

Dimensions	Criteria	Score (1–5)	Comments and feedback
Potential for impact against strategic priorities	(A) The plan addresses the Wikimedia movement strategic priorities of the Wikimedia movement (infrastructure, participation, quality, reach, and innovation)	4.00	The plan directly addresses many movement priorities, including Increasing participation, Increasing reach, Improving quality and Encouraging innovation. While the rationale presented in this proposal is not always clear, the links between this organization's programs and these priorities are strong.
	(B) The initiatives in the plan have the potential to lead to movement strategic priorities, if executed well	3.50	Given the potential for impact in this language area (French language projects, including French Wikipedia) and its reach in areas of the world where little Wikimedia work is happening (Africa), the potential for impact on a global scale is significant. However, it is not yet clear if the activities in this plan will definitively lead to global impact, particularly if the Africa-related work does not build local leadership.
	(C) The initiatives in the plan have the potential to lead to significant impact, given context and amount of funds proposed	3.50	The initiatives could have significant impact at scale, both in the organization's local community and in the broader movement. At the same time, the amount of funding proposed is considerable.
Ability to execute	(D) The organization has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed	3.50	This organization has relevant resources and demonstrated capacity in many areas through its track record in the Wikimedia movement, but concerns exist regarding its ability to effectively engage volunteers on the scale needed for this plan to succeed. This ambitious plan may also be a challenge to execute even for an organization with demonstrated capacity.
	(E) The organization has a record of success in similar initiatives	3.00	WMFR has a track record for success in institutional partnerships as well as in other areas; at the same time, it is attempting to do innovative but resource-intensive work on a large scale rather than implement new projects as pilots (for example, its regional program work). Past successes have also primarily been low-cost, small-scale initiatives.
	(F) The organization's leadership is effective, committed, and relatively stable	4.00	WMFR's board and staff have demonstrated effective and dedicated leadership while managing a difficult transition well, and done so transparently and in the spirit of sharing movement learning.
Efficient use of funds	(G) The funding requested is reasonable given the proposed initiatives	3.50	While the proposed initiatives have potential for impact, the amount proposed and the proposed growth rate in FDC funding is significant.
Efficient use of funds	(H) The organization has a record of using funds efficiently and staying on budget	3.50	While WMFR has faced challenges in this area in past years, it is continuously improving its financial reporting and is beginning to spend on target.
Quality of proposed measures of success	(I) Clear indicators of success are outlined in the plan	3.50	WMFR has improved its systems for understanding success; it is measuring direct outcomes of projects as well as impact on the online projects.
	(J) A plan is in place to track the proposed metrics	4.00	This organization has presented a detailed plan for tracking global indicators across projects and has demonstrated past ability to track direct outcomes of its activities in order to establish links between its activities and these global indicators.
	(K) The organization has a feasible plan in place to track the proposed metrics	4.00	Based on past performance, it seems likely that WMFR will be able to follow through on its plan; at the same time, the scale and varied focus areas of this plan may make tracking results a challenge.
Potential to add knowledge and/or other benefits to the movement	(L) The initiatives in the plan, if successful, could be helpful and/or productively adapted elsewhere in the movement	4.00	If successful, the proposed initiatives could be adapted to other contexts and lead to significant impact on the online projects.
	(M) The organization is ready to share with the movement the lessons learned from the initiatives	5.00	WMFR has demonstrated extraordinary willingness, even in the context of a difficult transition, to share both its failures and successes with the movement, and has strongly encouraged all movement organizations to share such learning more openly with one another.

Scoring rubric

Dimensions	Evaluation criteria	1 = Weak or no alignment with criterion	3 = Moderate alignment with criterion	5 = Strong alignment with criterion
Potential for impact against strategic priorities	(A) The plan addresses the Wikimedia movement strategic priorities	The plan will not directly address any strategic priorities	The plan will partially address strategic priorities or other goals that are closely related to the strategic priorities	The plan will directly address several strategic priorities
	(B) The initiatives have the potential to lead to the strategic priorities, if executed well	The initiatives have no clear connection to strategic priorities	Some or all of the initiatives have an unclear relationship to strategic priorities	The initiatives have been shown (by the organization or the movement) to lead to strategic priorities
	(C) The initiatives have the potential to lead to significant impact, given context and amount of funds proposed	The initiatives will have marginal impact in the local community and the broader Wikimedia community, given context and amount of funds proposed	The initiatives proposed may have a significant impact in the local community, given context and amount of funds proposed	The initiatives could have significant impact at scale, both in the organization's local community and in the broader movement, given context and amount of funds proposed
Ability to execute	(D) The organization has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed	The organization has neither the number nor the amount of resources, skills, and capacity needed for success.	The organization has some relevant staff and volunteer capacity, but may be under-resourced for some initiatives	The organization is well-resourced with the relevant skills and capacity for success
	(E) The organization has a record of success in similar initiatives	The organization has had trouble succeeding in similar initiatives	The organization has not engaged in similar initiatives in the past	The organization has conducted similar initiatives in the past and has been successful in achieving desired results
	(F) The organization's leadership is effective, committed, and relatively stable	Leadership has been unstable (e.g., changing on an annual basis)	Current leadership is committed to the organization; some concerns exist around stability or effectiveness of leadership	Current and past leaders have demonstrated the ability to develop and execute on strategic plans; leadership team has been stable and transitions have been smooth
Efficient use of funds	(G) The funding requested is reasonable given the proposed initiatives	Based on FDC experience and relative to other proposals, the plan contains significant over- or under-budgeting, and/or has not thoroughly accounted for costs	Based on FDC experience and relative to other proposals, the budget is generally consistent with the initiatives; some initiatives are likely to be over- or under-budgeted	Based on FDC experience and relative to other proposals, the budget is reasonable, and rigorous cost projections have been presented
Efficient use of funds	(H) The organization has a record of using funds efficiently and staying on budget	The organization has a poor record of budgetary management: e.g., consistently and significantly below budget (such as 50% underspending), and drawing on organizational reserves due to overspending	The organization has a middling record of budgetary management; e.g., generally on-budget for its biggest initiatives and cost items, properly recording any over- and under-spending, and with plans in place for improvements	The organization has consistently strong record of financial management; e.g., consistently staying on or below budget (within ~5% of budget), and putting surplus funds towards the organization's and/or movement's goals
Quality of proposed measures of success	(I) Clear indicators of success are outlined in the plan	Initiatives have no associated indicators of success, or the proposed indicators are unrealistic and/or unmeasurable	Each initiative has indicators of success, but at least some indicators are unrealistic or are not clearly connected to the initiative	Each initiative has a realistic and carefully framed set of indicators of success associated with them, which are rigorous and time-bound
	(J) A system is in place to track the proposed metrics	The plan does not include methodology or timing for tracking and collecting metrics to measure progress against proposed indicators of success	A proposal to track progress against indicators exists, but has flawed methodology or unrealistic timelines and initiatives	The plan includes timing and tracking of proposed metrics, and additional initiatives (e.g., surveys, research) to track indicators
	(K) The organization has a feasible system to track the proposed metrics	No systems are in place to track metrics, or staff or volunteer capacity to manage metrics tracking	There is some overall capacity (e.g., staff, volunteers) to track and monitor metrics, but the organization does not clearly have the relevant skills to do this	High-quality systems, and the necessary skills and capacity to manage these systems, are in place
Potential to add knowledge and/or other benefits to the movement	(L) The initiatives, if successful, could be helpful and/or productively adapted elsewhere in the movement	The initiatives would not be relevant to other movement groups	Some initiatives may be relevant to other movement groups	Many initiatives proposed are new or have seldom been tried elsewhere in the movement; if successful, other movement entities would benefit from doing similar initiatives
	(M) The entity is ready to share with the movement the lessons learned from the initiatives	The entity is isolated from other movement groups and rarely shares information; the entity's leaders and members seldom participate in movement-wide functions	The entity is somewhat active within the movement, and is developing more relationships with other movement entities with which it could share lessons learned	The entity has regular communication with other movement entities, is a regular participant in Wikimedia events, and has a track record of sharing with others

This form was created by: Winifred Olliff (FDC Support Team) talk 20:17, 8 May 2014 (UTC)