Grants:APG/Proposals/2017-2018 round 1/Wikimedia Nederland/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$403,998	$403,998	0%
Budget	$439,645	$545,027	23.97%
Staff (FTE)	3.8	3.8	0%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

Themes:

• Results are improving
• Relevant metrics
• Fundraising achievement
• Problems with program structure
• Education not leading to results

While WMNL has struggled to achieve commensurate results in the past, we are pleased to see that quantitative results are improving in 2017, especially for thematic areas. Programs for 2018 are still highly relevant, emphasizing community health and diverse content, and global metrics express the organization’s priorities. At the same time, improvements could be made to the 2018 plan, seems to lack coherence and still includes communications as a program.

WMNL achieved significant progress in fundraising, securing €100,000 for the gender gap and nature programs, but they are struggling to achieve results through their education program.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

WMNL succeeds at assessing opportunities in their environment, both on Dutch Wikipedia and other Wikimedia projects, and in the world. For example, the Nederlands in the World program makes good use of opportunities to make relevant content more available.

We have been encouraging WMNL to take more advantage of opportunities to secure funding from other sources. In 2017, they made progress in this area, but have set a modest target in 2018. We hope they continue to pursue these opportunities in order to lessen their reliance on APG funding in the long term.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

Due to similarities in program structure from year to year, it is easy to track improvements.

The transition to a new grant metrics structure has certainly benefited WMNL’s comparative results, since large-scale content donations (of images) can now be included in the total page count, regardless of whether or not they are used or assessed to be of quality. This explains (in part) the dramatic increase in achievements in this area from previous years. For example, in 2017, they achieved a large content donation of images and this significantly enhanced their pages achievement. Despite this qualification, we see results are improving in the real sense. Thematic initiatives in particular seem to be generating large numbers of articles created and improved (more than 4,000), and furthermore these are in specific thematic areas that have already been identified as useful. This is in addition to large numbers of images being generated through this work.

We are very interested in WMNL’s progress with respect to editor retention, which is not something that most chapters are bold enough to measure. At the midpoint, they have achieved the retention of 10 editors out of a goal of 27 for 2017. This goal for the coming year seems very high relative to their achievement, but we hope that some of the improvements made in the 2018 program plan will actually lead to loftier achievements in this area.

Achievements in content on other language projects are also impressive with respect to the goals WMNL set in this area (already three times what they set out to achieve at the midpoint of 2017), and also very positive from any standpoint.

It helps that this proposal clearly explains how past results and risk assessments may contribute to the success or failure of different programs.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

We continue to appreciate WMNL’s approach to thoughtful planning, which we have discussed in many previous assessments.

It is good to see evidence in this plan of how WMNL functions as a learning organization. For example, the identification of new approaches within existing programs is very helpful for understanding what things WMNL has decided to change about their existing programs and why. Perhaps it is a good time to reflect on how much progress WMNL has made in this area since entering the FDC process, rising to be a leader in this area among APG organizations!

We also appreciate WMNL’s thorough approach to risk assessment in the proposal, with explanations about why programs are likely to work, or why they may be risky.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

We appreciate that WMNL is working toward longer term goals. This is evident in their quality strategy and the alignment of their plan to this strategy. They are still working under their 2017-2020 strategy and the current program plan is similar in structure to last year’s, and so most comments in this section from last year’s assessment will apply. Wikimedia Nederland has also actively participated in the development of the new WIkimedia strategic direction.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

Given that this year’s program plan is similar to last year’s, many of our appreciative comments from the previous assessment apply. Yet, so do several of our criticisms.

As we have stated time and again that it is not helpful for WMNL to include communications as a program. WMNL, kindly allow us to be direct, (maybe a little Dutch?) with you for a moment and tell you directly to please stop doing this.

This plan is structured as two program buckets, community and content, but it seems obvious that each of the subsections are distinct programs which don't seem to be of equal importance. For example, the education program does not seem to be resulting in significant outcomes, community support could probably be more distributed under each program, and the two thematic approaches be grouped in one program. We appreciate the consistency from year to year, but it may make sense to rethink this structure to better reflect what WMNL most wants to emphasize.

The grantee-defined metrics are still ambitious and relevant and reflect the values and long term strategic goals of the organization. We have said it before, but let us say it again. We love that WMNL’s grantee-defined metrics express a commitment to impact beyond the Netherlands and to long term community health.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

It is not possible to assess how funds are directed toward impact with the level of information provided, since funds are linked to high level groupings and not to what appear to be actual programs; therefore it’s difficult to see, for example, how funds are invested in education program (less potential) work vs. the thematic priorities (more potential).

WMNL needs to bring in more financial support from other sources to sustain a budget of this size. We appreciate that they have already secured external funding for their gender gap and nature programs in 2018, and hope they can build further on 2017’s stroopwafel!

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Major Strength	We continue to appreciate WMNL's approach to strategy, although we think that their annual plan can be improved if they remove communications as a program (we realize this may be extending from this emphasis on communications in their strategy).
P2. Potential for impact at scale	Strength	We appreciate that WMNL is continuing to test out new approaches in the area of community health that may contribute to movement learning in this area, and that they have had success with their thematic approaches. We also see WMNL's focus on other language projects and content relevant outside of the Netherlands as a way for them to scale their achievements.
P3. Objectives and evaluation methods	Strength	We continue to appreciate WMNL's very relevant global metrics, and their approach to evaluation that includes a clear view of their progress in each report.
P4. Diversity	Major Strength	WMNL is continuing their focus on other language projects, and is featuring this as a global metric. Their work with the UNESCO or with refugee communities, as well as on bridging the gender gap has ambitious targets and we appreciate that this a focus of this year's plan.
Organizational effectiveness
O1. Past results	Neither	We appreciate that WMNL is generating improved achievements each year. While achievements in 2016 seemed low relative to the amount of funding received, we saw great progress in the 2017 progress report. Especially, WMNL is generating impressive numbers of articles in target areas through their thematic work, in addition to media files generated.
O2. Learning	Major Strength	We continue to appreciate WMNL's leadership in this area! It's easier to understand their results and what WMNL has learned by reading their clear and engaging reports. They always think deeply about their approaches, and apply learning to make improvements.
O3. Improving movement practices	Strength	We continue to appreciate WMNL's contributions in this area, and see that they are experimenting with approaches that they will share with other groups.
O4. Community engagement	Neither	This situation has not changed significantly from last year. We will be closely watching WMNL's progress in this area, and like that they continue to tackle challenges in this area head on especially through their work on conflict resolution.
O5. Capacity	Strength	WMNL has presented another clear plan that they have the expertise and person power to implement effectively. We appreciate their ongoing attendance to community engagement as an aspect of increasing their long term capacity.
Budget
B1. Past budgeting and spending	Neither	It is not very clear how WMNL ends up projecting an underspend in regard to their budget in 2017, while they are planning a substantial growth of their budget in 2018. WMNL secured a major donation to support their work in 2017 and earmarked it for diversity and nature related work.
B2. Budget is focused on impact	Neither	While we are impressed with WMNL's plan and past work, we continue to note that WMNL's grant request is large relative to the impact they are generating. Finally, it is difficult to track spending to specific impact areas with the level of detail given.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 01:42, 9 November 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.