Wikimedia Meta-Wiki

Strategy/Wikimedia movement/2018-20/Transition/Discuss/Evaluate, Iterate, and Adapt

< Strategy | Wikimedia movement | 2018-20 | Transition | Discuss

Discussion of priorities: Increase the Sustainability of our Movement • Improve User Experience • Provide for Safety and Inclusion • Ensure Equity in Decision-making • Coordinate Across Stakeholders • Invest in Skills and Leadership Development • Manage Internal Knowledge • Identify Topics for Impact • Innovate in Free Knowledge • Evaluate, Iterate, and Adapt

This is an open discussion space dedicated to defining potential priorities for implementation from the Evaluate, Iterate, and Adapt recommendation.

Monitoring, evaluation and learning at all levels

edit
  • Provide financial and human resources and expertise for collecting data for monitoring, evaluating and communicating progress on each of the recommendations and Movement activities at all levels of the Movement (global, regional, local).
  • Distribute responsibilities around the processes of monitoring, evaluating and learning with all Movement stakeholders so that they have mutual accountability (including coordination and knowledge management) and ensure continual progress.
  • ..

Develop a comprehensive evaluation system

edit

Evaluate the relevance, effectiveness, efficiency, and impact of the development of the activities of the Movement.

  • Continuously evaluate the implementation of the strategy recommendations in real-time, and disseminate to impacted stakeholders.
  • Collaboratively establish common and contextualized indicators and desired impacts for measuring and monitoring progress on the goals of the strategic direction. This would include indicators for equity, advocacy, public policy, diversity in people and content, inclusion and openness towards newcomers, community health, technology, platforms’ usability, and accessibility, among others.
  • Facilitate joint analysis, discussion and learning with the participation of communities and impacted stakeholders as appropriate.
  • Access external, independent research to complement self-evaluation where needed and appropriate (always based on principles of participatory evaluation regarding Programs).
  •   Support this as a priority. We need good ways to evaluate not only the strategic recommendations, but all activities that aim to improve the Wikimedia Movement. Without that, we can spend too much time and resources in activities that bring low or no positive return. Danilo.mac talk 19:03, 17 November 2020 (UTC)[reply]
  •   CommentI totally approve of an evaluation of WMF. But not of the non-existing "movement". There is no "movement" to be evaluated, bute there is the Foundation. How much did the staff of WMF increase over the last ten, five, three years? In which fields of work and departments? And is there any benchmark to evaluate the outcome of these departments and their projects? Pretty much everything I see coming out of 1 Montgomery Street gets abandoned after rabid protests from the communities addressed. WMF has an abysmal cost-result-relation but nothing ever changes, but more jobs get created in community relations, as if WMF's failures are in explaining their brilliant projects to unwilling people. --h-stt !? 14:48, 30 October 2020 (UTC)[reply]
Dear h-stt, thank you for sharing your perspective!
First of all, this recommendation calls forward for clear metrics to track the progress of the work on the movement strategy and ensure that they are implemented in a contextually relevant way, which means that there needs to be flexibility for iterations depending on the context. We also need to be open for adapting to external changes. I believe that overall this is a healthy approach to any change process.
Notion of the movement
I hear you noting that there is no "movement". While I understand that most of the volunteers / editors can just do their edits without any feeling of attachment, in most of the wikis we actually have a core group of editors who build up a certain sense of community. In smaller wikis there are fewer of these people in larger wikis more. This commonality is helpful in managing the policies and the whole framework of using different namespaces. As a result, every wiki has a face and feel of their own, created through collaborative approach. This establishes the community, of which probably most of the editors don't even feel part of, but this does not mean it does not exists.
In some ways these communities function in silos (just doing their own thing), but there are people who work across different projects and function as connectors, transferring templates, providing policy examples and other valuable information from "communities" they are active. This is where the movement starts to build. Of course, these contributors who are interconnected with several communities is a significant minority, but these people exist. And these types of interactions are creating a "movement" in my understanding. I feel that "meta" project where you wrote your comment is a clear part of that global interaction space and in a way with your participation in the discussions on meta you contribute to the existence of the "movement".
It gets even more complex with the organizational side, as we have 39 chapters, 2 thematic organizations and 136 user groups as of now. Coordination and collaboration between these organizations helps us to learn from each other to overcome common challenges and better seize the opportunities. The work varies from collaboration with GLAM partners, educational and government institutions to advocacy work, technical tooling and volunteer support (sic! there are many volunteers who actually feel engagement with the local organization to be highly beneficial). Naturally, regional and thematic ties are created between those organizations that also essentially builds the "movement".
While Wikimedia Foundation has a notable role, the recommendations need to be implemented in a collaboration with online communities and organizations. Even though most of the volunteers do not feel part of the community or movement, we need to work with the "connectors" to ensure that we implement the recommendations in a contextually sound way or defer from implementation in contexts where it is not helpful. Also we rely on this network in iterating when we get some things wrong or see possibility for improvement. This is essential part of the implementation and that is why the notion of "movement" has been used in the recommendations. It is not only about the Foundation, but collaboration across different stakeholder groups is of essence here.
Evaluation of the activities of Wikimedia Foundation
Regarding the Wikimedia Foundation, there is a constant (quarterly) evaluation of the work in relation to the proposed plan and relevant metrics. The system adopted is the OKRs, focusing on the key objectives and related key results, also facilitating connections between objectives for different teams and departments so it is possible to leverage the resources and support collaboration. This happens as quarterly tuning sessions with the slide decks available on Commons.
Most of the aspects of the work of Wikimedia Foundation work well and there is a significant amount of work in the backend that most of the people won't notice if it is functioning well. As the scale of the project and expectations to our interface increase there needs to be more support available, which does result in numbers of growth. Also the community members have a tendency to mind their own business while things are working or they are in agreement of what is being done - there is only an active engagement if something is not going in the direction in which they expect or prefer it to go. That is completely fine and probably a fine use of the time of the volunteers, but I don't feel it is fair to make the reduction that because we see more engagement around things that have a more negative perception, then all the actions of the Foundation are negative. Does this makes sense to you?
Regarding the community relations, I feel that there is a need for better connection. This is not about the PR aspect of the communication, as you seem to project it, but rather about improving the actual work by synchronizing better with the communities, so implementation of ideas is not out of step with the expectations from the community side. I don't see that there can be substantial constructive conversation if the connections are not well established. As a result, the communication becomes unilateral and there is no real touching point, which creates the perception you are describing.
In short, I don't feel that downplaying the potential of collaboration is beneficial for the future of the movement. You are welcome to take part in it or not to take part in it, as you please. Thank you for your kind attention! --KVaidla (WMF) (talk) 10:40, 13 November 2020 (UTC)[reply]
User:KVaidla (WMF): tldr; Other than you I don't get paid reading or writing walls of texts. I'm a volunteer, my time is valuable. :Please don't waste it. Cut this diatribe to one third at max, and I might be inclined to read it. --h-stt !? 13:30, 13 November 2020 (UTC)[reply]

Iterative change processes

edit

Iterate change processes in the areas of technology development, coordination, capacity building, policies, and governance systems, and promote validation after research and testing. This requires us to:

  • Involve affected communities/stakeholders to assure that lessons from the evaluation are shared to inform their activities at local and regional levels.
  • Involve affected communities/stakeholders to implement, determine and communicate changes to the strategy recommendations and related activities/programs.
  • ..

Adaptive Policies

edit

Adapt to meet new and altered situations and challenges by adopting policies and procedures based upon evaluations of the changing Movement and the changing world. This requires us to:

  • Increase the flexibility and adaptability of the structures and approaches we employ to implement each recommendation.
  • Incorporate evaluation results into decision-making at all institutional levels.
  • Use evaluation results to inform planning and budgeting of Movement stakeholders with fiscal and programmatic responsibilities.
  • Evaluate external conditions (socio-political, legal, environmental, etc.) to anticipate them, act on them if we can, and adapt to changes.
  • ..
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Strategy/Wikimedia_movement/2018-20/Transition/Discuss/Evaluate,_Iterate,_and_Adapt&oldid=23616136"