Learning and Evaluation/Evaluation reports/Wikimania 2019 Evaluation
Wikimania 2019 was held in Stockholm, Sweden and hosted by Wikimedia Sverige. This year's theme was "Stronger Together: Wikimedia, Free Knowledge and the Sustainable Development Goals". Wikimania 2019 also piloted a new structure around "Spaces," aimed toward a more decentralized program design.
Key Findings for Consideration
editAbout the value of Wikimania 2019:
- The majority of participants (73%) applied what they had learned in their work in the months following Wikimania.
- The vast majority made new connections with whom they were still in contact long after the event, connections that facilitated both learning (83%) and productive collaborations (80%).
- Informal, in-person interactions also enhanced existing connections, humanizing and familiarizing previously only online relationships.
About the conference experience:
- The vast majority of participants found the venue and conference environment safe and comfortable.
- Wikimania participants--especially first-time participants--are more likely to be female than is generally seen in online contributor communities.
- Participants found conference content relevant and interesting, but responded less positively to the conference theme and "Space" structure.
- A common complaint was how far venue sites and conference accommodations were from one another, impeding participation and interaction. Many also expressed a desire for clearer signs and directions.
Data Overview
editThis evaluation report summarizes data collected through a survey of Wikimania participants in the two weeks following the event, as well as follow-up survey data collected 3 months and 6 months following the event. It also draws from aggregated data collected at registration and three focus groups conducted with 17 Wikimania 2019 participants. 879 people attended, 156 of whom were Wikimedia Foundation staff. All participants were sent the initial survey, though Foundation staff responses are excluded from most of the analysis. Foundation staff did not receive follow-up surveys. Below it is noted where Foundation staff data are included.
| Volunteers | Wikimedia Foundation Staff | |
|---|---|---|
| Number of Wikimania participants | 723 | 156 |
| Initial survey respondents (% of attendees responding) | 409 (57%) | 94 (60%) |
| 3-month follow-up survey respondents (% responding) | 287 (40%) | n/a |
| 6-month follow-up survey respondents (% responding) | 194 (27%) | n/a |
What did Participants Gain from Wikimania 2019?
editWikimania participants that we surveyed at follow-up reported many benefits from attending the event, including making and strengthening social contacts and learning new information or approaches that proved useful in Wikimedia-related work after Wikimania.
Was what they learned useful?
editParticipants who responded to surveys 3 and 6 months after Wikimania indicated that what they learned had impacted how they think about their Wikimedia work, and that they had applied what they had learned. This was true at both 3 and 6 months following the event, indicating that the value of what was learned had lasting effects long after Wikimania.
I learned more information about user groups, shared it with the regional community and we finally applied for the creation of our user group.
We recently organised [a campaign]. Based on what I learned about partnerships, I identified my resources in the form of friends who really helped in the organisation of the event. This helped very much in creating awareness of the project and also the participation of students.
When providing examples of how they had applied what they learned, respondents were most likely to say that they used the new information to improve their personal content contributions to Wikimedia projects (37% at 3 months, and 28% at 6 months said this). They also often indicated that the information had been shared with their home community (23% and 20%), here we have an example of someone who used what they had learned about user group structures to discuss and start the process of creating a user group for their regional community. Another common theme was respondents using lessons learned to either implement a new campaign or improve an existing one (20%). Respondents also often cited using what they learned at Wikimania to improve one of their own events, to improve structures or processes within their affiliate group, growing or strengthening partnerships, and creating or growing editing parties.
.svg)
During Wikimania I strengthened my friendship with people from [neighboring country] and we discussed creating some mutual projects. And now we're organizing a camp for high school pupils which will take place in [neighboring country]. So we will collaborate and send students from [my country and the neighboring country] to attend that camp and they will write articles on Wikipedia and take photos of nature and buildings and similar stuff. --Wikimania 2019 participant, based in Europe
How did Wikimania impact social connections?
editWikimania had a variety of positive impacts on the social connections of participants-- both increasing the size of their social networks but also improving the quality of their social connections.
Wikimania broadens productive social networks. At three and six months after Wikimania, we asked participants about the new connections they had made at the event-- specifically, those that serve as a medium of sharing information ("learning connections"), and those that serve as a medium to advance Wikimedia work ("collaborative connections").
At three months, 87% of all respondents were still in contact with at least one learning connection that they had first made at Wikimania, with an average of 5.3 new connections per respondent. At six months, 83% of respondents were still in touch with at least one other person in this capacity, with an average of 4.7 new learning connections maintained. Collaborative connections were also made at Wikimania: 74% of respondents were still in contact with at least one new collaborative connection (3.4 on average), and at six months those connections had grown (80% of respondents in contact with at least one other person in this regard, 3.7 collaborative connections on average).
I think personal interaction is more important than online, because when you're online you might not really get what someone is trying to say... as you are reading, you might have this notion in your own head, but once you have the interaction with the person, one on one, [you are] able to tell, "oh, no, actually what they meant is this." --Wikimania 2019 participant, based in Africa
So I really enjoy just meeting ...people just in serendipitous ways and finding out about what they're doing, what they're passionate about, tools or tips or hacks that they might use that you might not be aware of. [...] It broadens my experience in, quite literally, who the community is, and it gives me a better insight into who the editors are: the contributors as people, as opposed to these content-producing machines. --Wikimania 2019 participant, based in Europe
In-person interaction at Wikimania improves the quality of interpersonal connections. A common theme that arose during focus groups with 2019 Wikimania participants was that in-person interaction with people you would otherwise only interact with online has immense value. Focus group respondents indicated that the most valuable time at Wikimania consisted of informal socializing with others between sessions and during more interactive sessions. This informality facilitated a different level of interaction, where participants could better understand the context and intention of others--motivations that are not readily apparent in on-wiki interactions. In essence, these informal interactions serve to humanize other Wikimedians in ways that improved on-wiki interactions after the event.
Focus group participants noted that informal, in-person interaction was also highly productive--connecting people to resources they might not have otherwise been aware of and progressing on-wiki projects.
Demographics: Who Attended Wikimania 2019?
edit
Wikimania participants came from 83 countries. The vast majority of attendees came from Europe (68%), with large contingents from Sweden (n=88) and Germany (n=86), as well as from the United States (n=68), France (n=33), and India (n=27). 12% came from Asia and the Pacific, 11% from North America, and 2-3% each from South/Latin America, Africa, and Arab States (see this page for more information on how countries are classified into regions).
38% of attendees identified as female, 57% as male, and 1.5% identified as genderqueer or nonbinary (due to the way gender was collected at registration, transgender attendees are not counted here and this is likely an incomplete picture of gender minority attendees). While female participants are underrepresented at Wikimania compared to the general population, they are more represented at Wikimania than in estimates of the Wikimedia contributor population (ranging from 6% to 25% female, depending on data source and project). A similar gender makeup was also seen among Wikimania 2018 attendees.
We also asked attendees how many days of work it took for them to pay for registration, lodging, or travel. 49% of all attendees worked more than 5 days to pay for their attendance, and this figure was much higher for those who applied for a scholarship (61%) than those who did not (38%).
While only 62% of attendees reported that English was one of the languages they were most comfortable communicating in at registration, 95% indicated in the survey that they were "mostly" or "completely" comfortable receiving information in English. This indicates that survey respondents might be more comfortable communicating in English than those who did not respond.
Who Was New to the Event?
edit
At registration, 46% of attendees indicated that 2019 marked their first Wikimania. New attendees were less likely to respond to the survey than return attendees, with only 38% of survey respondents said that this was their first event. New attendees were only slightly more likely to come from Europe (73%) than returning ones (68%). Of those who completed the survey, they were also slightly more likely to be female (43%) than return attendees (34%).
Most (69%) new attendees who completed the survey said they intended to return to another Wikimania, whereas 26% were unsure if they would. The 5% of first time attendees who did not plan to return cited reasons like next year's location of Bangkok being too far for them to travel (n=2) or that they had only intended to attend once to learn about the conference or the movement (n=2).
What Was the Experience of Scholarship Applicants?
editThe majority of scholarship applicants found the process easy to understand (85%), the application easy to complete (82%), and that they were informed of the result in a timely manner (81%). Applicants who were not awarded scholarships found the application similarly easy to complete (78%) as those who were (82%), but were far less positive about the ease of understanding the process (56%) and especially the time it took to be notified of their application status (41%). No major themes emerged in open-ended feedback about the application process, but several respondents indicated that they would have liked to have booked their travel earlier.
Conference Venue and Environment
editThis section includes responses from all attendees including WMF staff, which reported similar experiences as all other attendees.
Survey respondents were generally satisfied with the conference venue, particularly its physical (92%) and language (93%) accessibility, feelings of safety (94%), and upholding of the Friendly Space Policy by other attendees (91%). Many attendees thought the location was beautiful and inspiring (n=20). Overall, 88% were satisfied with the conference venue and environment, a slightly lower proportion than the 93% reported by Wikimania in Cape Town attendees in 2018. Reports of physical and language accessibility were also slightly lower than in the previous year.
The university was a very nice space. I liked the indoor and outdoor nature of the venue and the fact it is a university (a place for knowledge).
There [were] way too few signs pointing to the various spread out session rooms. Quite frequently, I was lost and had to ask around. The area for [one] session was unbearably noisy. It would have been nice to get an entire building/hallway for the... track with lots of small breakout rooms to have discussions in a silent and safe environment.
While most indicated that they were able to participate in a way comfortable to them (90%), they were somewhat less positive about being able to fully see, hear, or participate (79%). This often related to room layout (n=25), with reports of some spaces being too small, big, or noisy for the session's size or purpose. Only 76% of respondents were able to stay in a place convenient to the conference site, and only 61% found it easy to get where they needed to go at the venue. The most frequent critical feedback was that conference rooms were confusingly labeled and difficult to find even with the maps provided (n=62), and that the distance between accommodations (n=32) and venue buildings (n=45) made it difficult to make it to sessions on time or at all. At times, this also eroded feelings of community and made food and quiet spaces less accessible (n=8). Several people (n=9) also noted that there were limited dietary options for vegetarians and vegans, or that there were poor food options generally. Another 6 mentioned that the venue was not set up to accommodate people with limited mobility, though these respondents did not indicate having such difficulties themselves.
Experience of Presenters
edit132 non-WMF staff respondents had presented or led a session. 9% of presenters/session leaders reported issues with technology in their session, but 85% were satisfied with the tools and technology available. In qualitative feedback many cited problems having their slides ready, not having the necessary equipment to connect their computers, and a lack of clarity regarding how to access help with these issues (n=27). 83% were satisfied with the communication from organizers, as well as their overall experience. 80% found it easy to find how their content fit within the session or "Space" it was placed in.
Conference Content
edit
The vast majority of community attendees found the content of the conference to be relevant to them (92%), and a majority was pleased with the quality of the plenaries (79%). Attendees also indicated that they gained relevant knowledge (88%) and that they intended to apply it to their Wikimedia work (90%). Attendees were less likely to report gaining explicitly practical skills, however (64%).
Fewer felt positively about the "Spaces" structure (65%) and the conference theme (63%). Many articulated the value of this year's theme as providing an important orienting value to both the conference and the movement (n=56), and others saw value in connecting the Wikimedia movement's activities and goals to the United Nation's Sustainable Development Goals (SDGs) in order to acknowledge the work that Wikimedians already do related to these goals as well as offering more partnership opportunities (n=41). More critical comments found either that SDGs were not relevant to Wikimedia (n=13) and artificially constrained the content of the conference (n=7), or that the theme was not adequately reflected in most of the content (n=16). Only 59% indicated that they were able to attend most of the sessions that interested them, which might reflect in part the difficulties getting around discussed above, as well as a high level of interest in many of the scheduled sessions.
Wikimania "Spaces"*
editSessions were organized into 19 thematic Spaces. Survey respondents were asked which three Spaces they attended the most sessions in, and the most common responses were the Technology, GLAM, Strategy, and Education Spaces. The least common responses were the Libraries and Readership Spaces, though these Spaces offered far fewer sessions.
All respondents (100%) who attended many sessions in the Readership Space were satisfied with the quality, but did not offer qualitative feedback. The GLAM (94%), Research (92%), and Environment Space (92%) attendees also responded positively to those sessions' quality. No themes emerged from the qualitative feedback provided about sessions in each of these spaces.
Respondents were the least satisfied with the quality of content in the Advocacy (71%), Strategy (72%), and Diversity (73%) Spaces. Qualitative feedback about the Advocacy Space indicated that while some sessions were of high quality, others were more problematic (n=2). Critical feedback about Strategy Space sessions highlighted issues with the size and noise level in session rooms (n=9), but also a lack of clarity in the information that was shared (n=3) and ineffective facilitation (n=2). Some respondents (n=2) took issue with the focus of the Diversity Space sessions on mostly gender and not other diversity issues.
*Note: The first 73 respondents to the post-event survey saw a question that presented them with an incomplete list of Wikimania Spaces to indicate their participation, so these responses are excluded from summaries of Space attendance. They are included in measures of participant satisfaction with specific Spaces.
What Are Attendees' Favorite Parts of Wikimania?
editOn the whole, attendees (WMF staff included) valued Wikimania for the chance to interact with other Wikimedians in both structured and unstructured ways (n=137), especially those whom they would have no other opportunity to connect with in-person. Some critiqued the more dispersed physical spaces of the venue for inhibiting some of the usual social interactions (n=8), but the vast majority mentioned that these valuable interactive experiences were their favorite moments at Wikimania. They also noted the value of sharing their own ideas and work, and getting feedback and recognition, through presentations (n=13).
The people's reaction to Maher's statement in closing ceremony: 'There is no place for harassment in Wikimedia'.
There were some very informative, helpful and interesting discussions with Wikimedians from other countries. These were impossible to predict but very rewarding.
Surströmming! The Swedish hosts were just amazing! I am so impressed by their hospitality & organization. They did an amazing job.
The friendly and enthusiastic reception of my poster presentation!
This year in particular, many people noted that the opening and closing ceremonies were especially moving and that they enjoyed the ensuing receptions (n=80). Many also noted tasting Surströmming (n=16) as quite memorable.
.jpg)