9.00 - 18.30 at Wikimedia Deutschland office – How can we evaluate the success and impact of our volunteer support work?
Introduction: What is success, what is impact?
State of the Metric: Get an overview of what metrics and tools the Wikimedia Foundation and chapters use in order to evaluate community support activities. Let's assess these methods together: which (would) really help us to better understand successes and failures of our work?
Lunch
Future of Impact: Discussion and further development of possible ways to evaluate our work.
19.30 (near the hotel, location details will follow)
09.30 – 14.00 at Wikimedia Deutschland office – What are challenges we face in our work as volunteer supporters and how can we handle those?
Challenges related to our jobs and related to working with Wikimedia volunteers – share your story and (if you want to) get advice from your colleagues.
How do I reach my communities? It’s difficult to support your volunteers if you can’t communicate with them (properly) – perhaps you don’t know each other at all or there are some other obstacles. Let’s share best practices and discuss specific problems we might have with reaching our communities.
For the first time, 12 volunteer supporters from 10 Wikimedia chapters met in order to discuss and work on issues related to volunteer support in the movement. It was a very productive meeting with a great group of people and a positive and friendly atmosphere throughout.
The meeting started with a series of lightning talks where all participants introduced themselves, their context / chapter and shared insights on their work, as well as the challenges they face.
The next day revolved around the question of success and impact of our work as volunteer supporters.
In order to have a common starting point, different approaches and models for planning and understanding the success of volunteer support activities were reviewed (including the stairway to impact and logic models), with the difference between outputs, outcomes and impact being illustrated.
In a first brainstorming session, participants then stated what successful support of their communities looks like to them and what they are trying to achieve with their work. It quickly became clear that - even though all volunteer supporters present came from different backgrounds in terms of community size and community / chapter activities - everybody shared a common understanding of the purpose and the intended effects of their work. The more than 60 points mentioned were clustered into six main topics. More than one third of all points referred to the purpose of volunteer support being to help with „community happiness and motivation“ (22 mentions), which also included aspects like appreciation and empowerment. The other categories were „ensuring good community / chapter relations“ (13), „expanding community“ (8), „increasing skills, productivity & efficiency of volunteers“ (7), „increasing the reach of all things Wikimedia“ (6), and „increasing and/or improving content in the Wikimedia projects“ (5).
With those desired outcomes of our volunteer support in mind, the group took a closer look at what is currently being reported in chapters when it comes to evaluating the effects of their volunteer support. To do this, a variety of metrics were presented and evaluated with regards to their importance for participants. After that, tools and methods for measuring those metrics were presented.
Results revealed that the metrics for evaluating skills, satisfaction, content pages and working atmosphere were seen as most important, while the least interesting seemed to be the metrics for materials / resources, bytes and events. For many metrics however, there was no homogenous group tendency - the assessment showed that some metrics were of varying value to participants. Among the metrics which got the most mixed responses were diversity, time spent and usage of media files.
While the group agreed that the shared metrics are usually fairly easy to evidence and can thus be used to convey a general impression of supported activities and their outputs, it was noted that they are somewhat limited in their informative value when it comes to demonstrating the outcomes or impact of volunteer support activities.
After having looked at where they currently stand and where they want to go in terms of outcomes and impact, the participants formed groups and discussed what was necessary for them in order to be more impact-oriented in their work. The result was a wishlist of tools and skills that are currently missing, factors preventing one from focusing on results beyond the output-level, and particular aspects that participants want to concentrate on more in the future. This wishlist includes technical issues as well as methods and practises for a better exchange of knowledge among Wikimedia volunteer supporters and can now serve the Volunteer Supporters Network as a reference point for future endeavours.
The amount of input from the group throughout the day was tremendous and provides an excellent starting point for further discussions and work on the topics of measuring and evaluating the success and impact of volunteer support.
The second day of the meeting focused on exchanging experiences, sharing stories, identifying common challenges and struggles and discussing possible solutions. Among the aspects covered was the question of how to best increase the core community, reach existing communities and turn online into offline volunteers. More importantly, however, there was also time and a safe space to discuss a number of specific problems and help each other out with advice and moral support. It became clear that this type of possibility for personal exchange is something that is extremely helpful and had previously been missing for many of the attendees. It was discovered that work- or community-related challenges are almost the same for everyone, regardless of community or chapter size, yet most Wikimedia volunteer supporters previously felt like there weren't many people those issues could be discussed with.
In addition to continuing to work on the topics covered the first day, the group also agreed to hold regular meetings for volunteer supporters which offer a possibility for conversation and exchange of experiences and advice. As next steps, it was decided to keep the discussions going online and keep working on the topic of „impact of volunteer support“, most likely during sessions at Wikimedia Conference and Wikimania.
