Grants:APG/Proposals/2017-2018 round 1/Wikimedia CH/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$305,312	$311,500	2.04%
Budget	$913,534	$1,119,088	22.5%
Staff (FTE)	3	5	66.66%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

• Lack of past results
• Plan lacks focus
• Evaluation is very basic
• Fundraising progress

WMCH must strengthen the link between funds spent and impact. If they use their strategy to influence the development of future plans in a coherent way, rather than creating an assortment of various activities, they may achieve this result. More sophisticated and relevant metrics will be important to helping them articulate the impact that they are having in these strategic areas. WMCH must also ask itself what its added value both to its community and to the movement might be, and articulate both to justify the amount of funds it is currently using.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

Wikimedia CH has been struggling for years to work with a distributed, difficult to identify, multilingual community, and unfortunately this focus does not seem to have yielded results. Year after year, Wikimedia CH laments on their difficulties due to multilingualism, and we see no evidence that they are effectively addressing this challenge. For example, they approach their communities as separate although this does not seem to have yielded the desired results.

There are other opportunities in WMCH’s context, which we have identified in past years, including fundraising potential. We see that important progress has been made in the current year, with WMCH raising 150 000 CHF from major donors. This is a way that WMCH is uniquely positioned to benefit the movement at large.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

WMCH’s metrics are so rudimentary that it is difficult to interpret their results. Based on the information we have, it seems like their large grant in 2017 has resulted primarily in large numbers of images generated through content donations and photography support work, without any indication of their quality or usefulness. In the current year, WMCH is tracking their number of partners, which does not give us much information about the depth of their partnership work.

Gender gap work in the Swiss communities continues to produce good results in terms of articles created or improved.

Results from GLAM work are diffused, ranging from WMCH’s involvement with the national network of 11 GLAM partners to work with a number of small institutions. There is a lack of continuity between successful initiatives in the current year, such as the Archives Day, and the 2018 plan.

Past work in the area of education does not indicate that WMCH will be able to achieve what they have set out to achieve in 2018.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

WMCH is planning to add 2.0 FTE staff this year, and based on past results and the current year’s plan, we think this is unlikely to lead to more impact. Some staff roles are not well-defined, or have definitions that are too broad.

This organization has not been successful in producing desired results in the past, and seems to lack the ability to translate strategy into an effective plan.

Our interactions with Wikimedia CH throughout the year have shown that they have been organizing some successful activities, such as the Archive Day. Some activities they are supporting, such as the gender gap events, may be leading to good results. The reorganizations due to some turnover in staff seem to have brought about a dedicated and motivated team. While efforts have been made to deliver on commitments, unfortunately this does not show clearly in WMCH's last report or this proposal.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

Last year we noted some of WMCH's strategic plan's strengths and weaknesses. We are disappointed that this strategy is not informing their annual planning, as this plan still reads as a list of projects that are not unified by common objectives or theories of change, and many of which seem greatly dependent on the specific individuals or interests involved.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

We appreciate the context and definition provided for each program. Yet, programs include a large number of goals that do not clearly convey the program’s purpose or give us an idea of how success should be interpreted. Furthermore, these targets are low, and setting low targets has been a recurring problem for WMCH.

Overall, this program plan gives the impression of a diffuse set of activities. It is not clear why some programs are presented as priorities, while others appear to have been deprioritized from the current year. One example of this is the focused support for a network of GLAM institutions, which is not clear in the 2018 proposal. Another example is the Swiss Archive Day of 2017, which appeared to be a successful initiative that is not built upon in the 2018 proposal.

Despite Technology/know-how being an enabler in their strategic plan, WMCH seems unsuited to address tools and technical infrastructure:

• We don’t understand if the GLAM statistics tool has evolved since last year, or how this will benefit the international community beyond WMCH.
• While it seems extremely important that fundraising activities can happen in a secure environment, it is unclear how WMCH is best placed to foster technical innovation in this space.
• The role of WMCH in supporting Kiwix is not clear.

We are glad to see WMCH finally measuring images in use as part of their 2018 plan, and featuring this as a grantee-defined metric. Yet, the targets set in this area are very low. You can compare them with the targets of organizations functioning with a fraction of this grant amount in 2018, and they are still low. The number of people reached may be a relevant metric in WMCH’s context, due to their unique fundraising opportunities in Switzerland, which are potentially dependant on their image and notoriety.

Yet again, it appears that WMCH has set some very conservative targets for new editors and pages added or improved, which are actually below their 2016 achievements. This makes it very difficult to justify a staffing increase or an increase in the overall budget or grant amount.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

In general, we can’t follow the link between the proposed request for funding and impact. At this level of funding, WMCH must present a much more focused proposal. We appreciate the work done on networking which led to their success in fundraising in the current year, and encourage WMCH to continue to build in this area.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Neither	The organization has an interesting strategic plan in place. However, some programs seem more like a list of ideal or potential activities rather than activities designed to maximize impact within their strategic framework. It does not seem clear that WMCH is directing resources toward activities with the most potential for impact.
P2. Potential for impact at scale	Concern	We do not think this plan is likely to result in impact at scale with proportion to the funds requested. Wikimedia CH needs to explore real opportunities for impact at scale.
P3. Objectives and evaluation methods	Concern	We are disappointed to see that WMCH does not appear to be providing relevant metrics. This is concerning at this level of funding, especially since we see organizations with a fraction of this funding including more relevant targets.
P4. Diversity	Neither	WMCH, despite having diversity as one of its strategy enablers and "Diversify communities" as a program, only seems to devote a very small part of its resources on deliberate diversity actions. It also seems to focus on language barriers.
Organizational effectiveness
O1. Past results	Concern	While WMCH is improving in their ability to achieve their targets, targets set are low, and quality of the achievements is variable. For example, we see large content donations or numbers of images produced through photography support, but we have no way to understand if these images are useful to the projects.
O2. Learning	Neither	In the past, we have been appreciative of WMCH's approach to learning, and we continue to appreciate their efforts in this area. Yet, this does not seem to be leading to improved results.
O3. Improving movement practices	Neither	WMCH mentions many initiatives in their plan that could lead to further integration with the movement, but those are not clearly defined or don't seem definite. WMCH routinely works with neighbour chapters and actively supports FKAGEU. Some interesting projects such as the global statistics tool don't seem to get much focus in this plan.
O4. Community engagement	Neither	WMCH has a fractious community that is difficult to engage and difficult even to define due to the chapter's geographical and linguistic scope. The chapter makes efforts to listen and engage, and we are curious to see where this could lead.
O5. Capacity	Concern	Programs seem overly reliant on specific staff or contributors. In the wake of the governance crisis of recent years, we have not seen an improvement in outcomes as this organization stabilizes.
Budget
B1. Past budgeting and spending	Concern	We have concerns that past funding has not led to proportionate impact. Despite this, we have raised WMCH's score in this area because fundraising achievements in the current year (from major donors) are very impressive.
B2. Budget is focused on impact	Concern	It seems unlikely that this grant will lead to impact proportionate with funding, and there may be problems with how staff resources are being directed to program outcomes.

This staff proposal assessment is the work of FDC staff and is submitted by: Morgan Jue (WMF) (talk) 00:18, 9 November 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.