Grants:APG/Proposals/2017-2018 round 1/Amical Wikimedia/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$58,223	$74,146	27.35%
Budget	$80,206	$89,117	11.11%
Staff (FTE)	1	1	0%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

• Good performance
• Lean structure
• Lack of evaluation
• Lack of strategy
• Involvement in the movement

Amical is a unique organization in many ways. They work within a very close-knit language community and benefit from a very motivated and efficient support from volunteers. As such, they manage to be a very efficient organization, driving and supporting good projects and yielding good results. Amical has been advocating for others in the movement to adopt their lean structure and frugality, without really engaging on a more global scale or working to understand the context of other organizations. They also are resistant to feedback, and lately have simply ignored recommendations from the FDC and the APG staff, especially concerning their metrics, targets and strategy. We question Amical belonging in the FDC-APG process as they don't seem to want to take advantage of it, nor seem to wish to contribute to it constructively.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

Amical's context has not changed since last year. The organization's operations are set in a highly favorable context and benefit from a highly engaged community.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

Amical has been reporting on basic grant metrics, and has been very clear that they don't think those metrics inform their decisions. They decided not to report on their own set of grantee-defined metrics as shown in their progress report, which is somewhat at odds with their declared focus on other forms of evaluation. It is not clear how the metrics they define for each program actually inform their programmatic evolution, since they do not report on them.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

We are glad to see that Amical has decided to use to the services of a professional accounting service. Amical continues to be a frugal and lean organization with a small budget. They have managed to adequately leverage past work with their partners to ensure their autonomy and achieve sustainability and continuity in their programs.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

Amical's strategic plan has been in place since 2014, and seems to be reviewed periodically with the members and the community. Organizing the Hackathon this year seems to be a strategic direction that Amical is looking to use as a focus to inform other activities.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

Amical develops a lot of very interesting activities, which we can't really assess because of the lack of targets and reporting against their own metrics. We can give a special mention to PESCAR MEDITERRANI for its innovation scope, and how the Hackathon is linked to the Autonomous University of Barcelona to find synergies.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

Amical is asking for a budget increase to scale for programs this year. Given their track record, we believe this has potential to be put to good use. The lack of clear targets, however, makes it difficult to assess this exactly.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Concern	Last year we expressed concerns about Amical's approach to strategy, which is not what we would expect for an organization in the FDC process. Given the lack of improvement in this area and the lack of incorporation of feedback of the FDC and APG staff, this is an increasing concern.
P2. Potential for impact at scale	Strength	(last year's comment applies) WMCAT's approach to leveraging networks and developing self-sustaining partnerships while empowering volunteer leaders is scalable, and serves multiple organizational goals simultaneously. Their Bibliowiki project involves more than half of all Catalan public libraries for example. This approach is employed in their GLAM and education work, to empower partners to devise their own projects without ongoing intervention from WMCAT staff. Apparently however, Amical has not been able to scale as effectively since it is asking for a budget increase this year.
P3. Objectives and evaluation methods	Major Concern	Last year we expressed concerns about Amical's approach to evaluation, which is not what we would expect for an organization in the FDC process. Given the lack of improvement in this area and the lack of incorporation of feedback of the FDC and APG staff, this is an increasing concern.
P4. Diversity	Neither	(last year's comment applies) WMCAT has been partnering with Wikimujeres and integrating more gender approaches into its programs (Viquidones, Viquiprojecte 20x100 Dones in Wikiquote). It is, however, less clear in this year's proposal how they intend to address diversity questions. While they mention diversity as one of their core focus (tackling gender, geographical, linguistic and historical diversity), there are no specific targets associated with this topic.
Organizational effectiveness
O1. Past results	Strength	(last year's comment applies) Relative to their size, WMCAT is a consistently high performer in terms of articles improved, and in terms of qualitative achievements around empowering partners and volunteers at the center of its work.
O2. Learning	Neither	While Amical has proven that they can adapt their actions based on experience to maximize impact, they have an offhand approach to feedback from the FDC and APG staff, which is not what we would expect for an organization in the FDC process.
O3. Improving movement practices	Concern	Apart from very few individuals, the trend seems to be that Amical's community does not wish to integrate into the movement and often lacks the willingness to consider other perspectives. This attitude is concerning.
O4. Community engagement	Major Strength	(last's years comment applies) WMCAT engages a high ratio of Catalan contributors in its work. Community is core to this organization's DNA and volunteer leadership is a core component of every single program. Amical exceeds expectations in this area, and models good practices for other organizations.
O5. Capacity	Strength	(last year's comment applies) WMCAT has a unique organizational model that builds capacity in its community at large by designing programs that aim to be self-sustaining and promote volunteer leadership. Amical is constantly making efforts to bring in new contributors, volunteers, and supporters, even with limited resources. Amical's sole program staff takes on a lot of responsibility, as does their volunteer board that manages operations. At the same time, staff and board are highly competent in program implementation. With this model, it may be difficult for Amical to meet FDC-level standards in operational and strategic areas and receive the support they need, but it is clear they can achieve results.
Budget
B1. Past budgeting and spending	Strength	(last year's comment applies) Amical appears to use funds effectively year over year, with program results that are commensurate with grant amounts. Amical is not always accurate at forecasting revenue or expenses, which may be more typical for a smaller and more dynamic organization.
B2. Budget is focused on impact	Major Strength	(last year's comment applies) The budget is focused on areas that are likely to lead to strong program results, especially investment in Amical's sole program staff, with minimal operating expenses.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 01:36, 9 November 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.