Grants:APG/Proposals/2016-2017 round 1/Wikimedia Österreich/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$278,663	$295,382 (2017, 2018)	+6% (2017), +0% (2018)
Budget	$306,529	$344,427 (2017), $353,344 (2018)	+12.36% (2017), +2.59% (2018)
Staff (FTE)	2.75	2.75	0%

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• WMAT is a strategic organization with the ability to respond to volunteer needs.
• WMAT’s staffing structure has proven effective.
• WMAT has an elegant evaluation plan with the capacity to implement.

Concerns

• WMAT is requesting a large grant, relative to the size of its community.
• WMAT does not have female board members.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

• WMAT continues to maintain close ties with German-speaking contributors in Austria and beyond its borders. WMAT’s community meets regularly in person to work on projects and discuss relevant topics, and takes a personal approach to volunteer engagement and development. Volunteers regularly meet with staff and with one another to discuss their ideas and share skills.
• WMAT is well-positioned to collaborate with other German-speaking groups and organizations, and is positioned in Central Europe, allowing them access to networks of Eastern and Western European chapters. WMAT takes full advantage of these opportunities for networking, contributing actively in these geographic networks. WMAT’s focus on photography and their volunteer-centric approach has encouraged cross-border contributions to their projects, especially from contributors in Germany.
• WMAT works with contributors to German-language projects, as well as with a community of photography experts that primarily contributes media to Wikimedia Commons. Work on German Wikipedia is challenging for many contributors in Austria, since German Wikipedia’s norms do not embrace the use of Austrian German on the projects.
• Despite a difficult environment for fundraising in their country, which they had detailed in their 2015 proposal, WMAT has managed to secure long-term funding from external sources, especially around their Open Data Portal.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

• WMAT exceeded their target of 10,000 articles in 2015, achieving 12,695 by the end of 2015. However, WMAT’s perfomance in articles created or improved has declined relative to their 2015 achievements, with 12,695 articles achieved in 2015 and 3,217 achieved at the midpoint of 2016.
• WMAT’s best results are in the area of quality images. Due to their program focus in photography, WMAT is improving their results in media used year over year. In 2015 they achieved 7,158 of a target of 5,000 media in use, and at the 2016 midpoint they had already achieved 3,919 media in use. At the 2016 midpoint, WMAT achieved an impressive 3,233 awarded images, which means they are on track to exceed their 2015 achievement of 4,601 awarded images.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

• WMAT consistently produces high quality results with attention to detail. They have effective organizational systems in place to ensure the quality of their work, and meet local standards of excellence for nonprofits in Austria.
• WMAT has a stable and committed board with good expertise. In many ways, WMAT has been a leader in modeling good governance in the Wikimedia movement, through their implementation of the good governance codex. However, WMAT has no female board members.
• WMAT’s staffing model works well for the organization and their community, and leads to good results. Besides a committed ED, their program staff has strong community ties and relevant expertise. Responsibilities are spread among staff in a way that limits the possibility of a single point of failure. WMAT’s staff has stabilized, and they are planning to keep the same size staff in 2017 and 2018. Despite past concerns about the size of their staff with respect to the size of their community, results have proven that this staffing model works. Staff has a close working relationship with the board and key community members, enabling seamless and effective coordination among staff and volunteers, and enhancing WMAT”s capacity.
• As noted in previous assessments, WMAT is an organization with a strong learning culture.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

• WMAT has a dynamic strategic plan up to 2020, which they plan to revise in late 2017 in order to integrate the new movement strategy. WMAT’s strategic plan includes few long term targets (other than fundraising), which may make it challenging to track long term progress.
• WMAT has integrated its fundraising goals into its strategic approach, and has emphasized regional collaboration with other German-speaking groups.
• Their focus areas of volunteer support, free content, and free knowledge are mirrored in their plans for 2017 and 2018.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

• WMAT does not have a program that specifically addresses diversity, although it is a cross-cutting approach and is included as one of their grantee-defined metrics. It will be interesting to watch how WMAT monitors their diversity results, as some of this data may be difficult to collect due to privacy reasons for example. WMAT points out that they will focus on getting their results from projects addressing the specific populations they are trying to support.
• WMAT’s community support work is traditional in some ways (e.g. regular meetups, providing travel reimbursements), but thoughtful metrics are applied to measure motivation, leadership, and retention of new editors. These will form useful baselines for measuring the efficacy of this work over time.
• In order to give focus to its activities in the next funding period, WMAT has been developing a concept around Austria’s heritage: WikiDaham. This focus is aimed at boosting regional expansion as well as allow more involvement from communities outside Vienna.
• WMAT’s Free Content program also has a healthy ratio of innovative work and work that is based on WMAT’s core expertise in supporting photography. Beyond a pool of photography equipment, WMAT provides structure and support to photographers and finds interesting and engaging ways for photographers, editors, and the public to interact and create content. WMAT is focusing on an interesting partnership with Simpleshow to work on storyboards and explainer videos. In this program, WMAT is measuring two dimensions of use: useful content and versatile content which aim at measuring impact beyond the German Wikipedia.
• WMAT is supporting projects that have a potential for impact outside of WMAT’s primary sphere of influence, such as Wikipedia for Peace, in partnership with Service Civil International, a project which has allowed for contributions in more than 10 languages, and their project with the Simpleshow Foundation. WMAT is supporting FKAGEU.
• WMAT has produced a simple yet effective evaluation plan, including relevant metrics. WMAT has clearly been building capacity in this area over several years. In this vein, they have selected two relevant grantee-defined metrics for awarded images and participation from members of target groups to assess their progress on quality and diversity. We continue to appreciate that WMAT provides baseline data when setting targets and reporting on metrics. We appreciate that WMAT has improved its ability to set targets that are achievable yet appropriate to the scale of its past achievements.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• WMAT is increasing their grant request by about $16,500 from 2016 to 2017, and keeping a consistent grant amount in 2018. WMAT plans to be 80% reliant on APG funding by the year 2018 by keeping their grant amount consistent and raising more additional revenue.
• WMAT has made progress in fundraising outside the movement in 2016, and has set ambitious yet achievable targets in this area for 2017.
• While WMAT does not track staffing expenses by program, non-staff expenses are weighted toward community support.
• WMAT has been systematically tracking in-kind donations and has achieved good results in this endeavour ($18,000 in 2015 and expected in 2016). They have shared their strategy in that field in a learning pattern.
• WMAT holds about $72,000 in reserves, which would cover about 6 months of staffing in the event of an emergency.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Strength	WMAT has a clear strategy that includes relevant organizational goals as well as areas for program work, and plans for 2017 and 2018 closely mirror this strategy. We are encouraged that WMAT is taking a dynamic approach to the design of their strategic plan, with plans to integrate larger movement strategy into their long term vision. We do not, however, see clear targets included in the long term strategy, which may make assessing progress more challenging.
P2. Potential for impact at scale	Strength	While effective, WMAT's staff-intensive approach to community support is unlikely to scale. However, work in other areas may have the potential to benefit the broader movement if others adopt WMAT's good practices in areas like photography support and their collaboration with the Simpleshow Foundation. Support for FKAGEU (a joint effort) has potential to lead to impact at scale. WMAT's work currently extends beyond its borders through the involvement of German-speaking volunteers from other countries, but on a small scale. We see this as an area where WMAT has the potential to grow in the next two years.
P3. Objectives and evaluation methods	Major strength	We are pleased to see that WMAT has set realistic targets for the upcoming years that truly reflect their ability to achieve. Strong practices in evaluation continue and are being refined. In particular, WMAT is setting highly relevant metrics for each of its program that provide good insights into its potential achievements. At the same time, WMAT is selective in the metrics they present, creating the impression of a feasible plan and leading to an effective storyline. We continue to appreciate the ongoing evaluation that WMAT presents in its reports, as well as places where they have established baselines to compare achievements year over year. We'll be interested how WMAT reports on its diversity metric.
P4. Diversity	Neither	We are pleased to see WMAT include a grantee-defined metric that is related to diversity. At the same time, we could not find specific plans or program-specific metrics related to gender. This makes it difficult to understand the link between WMAT's program activities and how they expect to make progress in this area. Overall, we did not find a strong emphasis on diversity in this plan.
Organizational effectiveness
O1. Past results	Strength	WMAT has achieved impressive results in image quality, such as 3,919 images in use and 3,233 awarded images by the midpoint of 2016.
O2. Learning	Major strength	WMAT is an organization with a strong learning culture, which carefully and thoughtfully documents learning and actively shares what it learns. They are applying learning to refine their programs and approaches in program and organizational areas.
O3. Improving movement practices	Major strength	WMAT has a history of hosting or co-hosting movement events of good value. They have been sharing actively relevant learnings about organizational growth and best practices and have been a model in this to other movement entities. WMAT has also been actively participating in several network groups, including FKAGEU, and has served as a mentor for several younger organizations in the movement.
O4. Community engagement	Strength	WMAT's core community is small but close, and is empowered to have influence in the organization's direction. There is a good pipeline of committed volunteers to fill different roles in the organization, and WMAT's approach to community engagement extends across borders.
O5. Capacity	Major strength	WMAT has a competent and stable board. The ED and staff have shown skill and expertise in implementing programs and achieving results. WMAT has good governance practices and benefits from engaged volunteers who support their work. Given that their planned growth is well contained (over two years), their proven staffing model should continue to keep the organization running successfully.
Budget
B1. Past budgeting and spending	Strength	WMAT has a track record of accurate budgeting and spending. While in 2015 they were underspent by about 10% of their anticipated budget, their revenue target and their spending target for 2016 are projected to be on track.
B2. Budget is focused on impact	Strength	WMAT's budget is focused on programmatic impact, and is weighted toward staff expenses. WMAT's staffing model is efficient and focused on program results. Travel costs are still very high as a percentage of non-staff expenses, though WMAT argues that this is an important aspect of their work.

This staff proposal assessment is the work of FDC staff and is submitted by: Winifred Olliff (WMF Program Officer) talk 21:28, 8 November 2016 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.