User:MCruz (WMF)/Sandbox/Community Health learning campaign/Report

Community Health learning campaign - The Report

Communication

• The campaign was first communicated in person at Wikimania 2015 on July 15 - July 19.
• On July 28, an announcement was published on the Education Newsletter.
• On July 31, emails were sent to the following lists: GLAM, Wiki Loves Monuments, Gender Gap,
• The campaign was massively distributed to 777 community spaces on August 1st.
• We also implemented 1-on-1 communication with 37 Wikimania participants who had contributed a drawing.

Participation

• The questions page of the campaign was viewed 1,914 times from July 16 to August 24. The landing page was viewed 3,544 times in the same time frame (the day after MassMessage registers 594 views alone).
• 61 editors from 24 communities gave input either by submitting a drawing or responding to one of the 6 questions posted on the Campaign's page.
• 26% of editors were from English Wikipedia; 18% of editors came from Spanish Wikipedia; 6.6% from German Wikipedia; 4.9% from Dutch Wikipedia; 6.6% of users had Meta as their home wiki.

Contributions

• 85 drawings representing a Healthy community were submitted in the timeframe of the campaign; 3 of these were created online, 82 created at Wikimania.
• 186 responses were contributed to the 6 different questions posted. While the questions under Welcome and Unwelcome had the highest number of contributions, the questions around Living projects and Help each other learn have the longest responses.

Lessons learned

Language

The campaign was launched marked for translation, but with no language other than English available to read the different pages of the campaign. The assumption behind this was that translation would be an indicator of success: the more translations, the wider the campaign spread and the more motivated users were to take part. However, first participants to visit the campaign pages were disappointed to ask to take part and not have the questions available in their own language. There were two opposing reactions: editors who protested, and editors who translated the page, with no comments made. Even though there were equal parts to both positions, it would have been better to translate the page in advance, following the rule: if we are requesting user feedback, we should have it as many languages as possible. A good criterion to chose what languages to translate to is density: the top 20 Wikipedias are the ones that concentrate the largest number of articles and active users.

Legal language was also a barrier. Before massive distribution, a community member pointed out to us that it seemed no one in the movement could take part in the campaign. This is because the legal wording was not optimal. In other points, the rules and regulations were very specific about the content that was shared on the campaign, and it was interpreted as though it couldn't be negative towards WMF. In general, legal terms are seen in the movement as an americanization, and are not very well regarded. For the purpose of this campaign, we could have avoided using legal terms, and describe in a more clear way how all participants that sign up on the list are participating for a scholarship, under the TPS grant terms and conditions. We should try to avoid to use legal language unless it is absolutely necessary.

Accessibility

Sending the MassMessage in English to all Village Pumps is not good practice. Users expect to see their own language when they read their Village Pump. A good practice for mass message distribution would be to first curate the village pump distribution list, pulling out the top 20 Wikipedias' Village Pumps. Then, draft a message on Meta, have it translated to those languages. Finally, send the mass message in English to the newly curated list, and post one by one on the short list. After surveying Village Pumps spaces, we were able to find out that the message was not well received on Dutch, German, French and Polish Wikipedia communities. While we had Dutch and German editors take part on the campaign pages, the same didn't happen with French and Polish. This could have been prevented with translation in advance.

With the original message being only in English, and the campaign pages as well, the conversation didn't set off for a good start. In spite of this, half of the users reacted well and translated the message and questions, and the other half complained. Further, these complaints were enhanced by a generalized negative perception some users had of WMF (in relation to past attitudes). The negative tone of a few users on the Talk page and in their contributions may have set a negative environment in the campaign in the first few weeks. Although hard to prove, this could have been why some users recurred to alternative solutions to contribute: requesting an anonymous survey, and responding on behalf of a group.

Making information accessible also means providing enough background. Even though the campaign page with «Instructions» frames the conversation around Wikimedia's mission and why it is important to talk about this, editors were still wondering what would happen to the feedback, and how it was going to be processed. Writing an FAQ page would have been a good idea, since it might have reduced the time dedicated to answering comments on the Talk Page by 25%. An FAQ page could give more extensive information on the background, links to relevant research or past planning, providing in this way a platform to have a more holistic approach to the topic of the campaign. This could give editors a reassurance and also helps focus the conversation on the topic of the campaign, at the same time that it holds the coordinating team accountable for this effort.

Find your allies and work together

The campaign was a good case study to show how working with community members side by side can be very productive and beneficial for both parts. Shortly after mass distribution, Ad Huikeshoven reached out to tell us the message was not very well taken in the village pump. He also shared that, as board member of WMNL, he had the task of working on community health on a local level, and was wondering how he could start that conversation. After a few exchanges, he decided to translate the campaign's questions and create a new space on Dutch Wikipedia and have users respond there, instead of the campaign pages. This proved very effective, as 13 people responded on the newly created page, over 4 times more than the number of Dutch Wikipedia editors that contributed on Meta.

A good strategy could have been to work with local communities on 5 of the 20 in the short list, and develop a local liaison, to create spaces on language wikipedias and have people answer there as well. Ideally, these liaisons would be members of the chapter, or engaged in an active way, so to ensure an understanding of the both worlds: the staff and the volunteers. Choosing only 5 comes from the number of village pumps that registered negative responses, and also, a good proportion of the total amount.

Respond to comments

Engaging in conversation is key to both show appreciation for the editor's time invested in that dialogue, and also to rein the conversation back to the focus when needed. Sometimes the comments on the Talk Page had a mixture of issues with the ongoing campaign and negative feelings (resentment) for past actions of WMF. It is important to focus on the present campaign, and not aim to answer all the points. Other points can be addressed in a general way, but giving way to foreign topics will not get users to contribute their views to the campaign. It is important to focus on the positive: this means that even when an editor is not saying things in a positive way, the moderator can thank them for the feedback. If you could rephrase a negative sentence into a positive sentence, without changing the meaning or core message, would it be useful? This is what focusing on the positive means.

Sometimes, technical issues arise and cause something that is later on the root of a comment. Acknowledging that there are technical difficulties or we don't know how to fix something is good practice, because it shows the user there was no intention in generating harm. Once this is cleared out, most editors are actually willing to help out and fix it themselves.

There is a practice in different communities of the last word. If an editor posts a comment and they expect an answer, there should be an answer. If there isn't one, they will feel like their feedback doesn't matter, or that we don't care. When there is an answer, there will probably be a follow up answer. It is up to the moderator to continue answering or not, and in this point it is really up to each situation and what is being discussed, but serve well to know that sometimes simply thanking the editor for their comment is good practice. In a way, it is like saying «thank you for your feedback».