User:MCruz (WMF)/Sandbox/Consultation Tool

The Consultation Tool is a guide that aims to help community members run their own consultations within their community. Inquiring and surveying editors can help a Wikimedia organization better work towards shared goals, and strengthen engagement with local editors. After developing successful consultations at Wikimedia Foundation, teams in the Community Engagement department have captured lessons learned and put together this guide, which is divided in three stages: before, during and after the consultation takes place, with key topics to look out for. This resource, as any other, is a work in progress. Anyone can edit and contribute their own experience, simply by clicking the Edit button in each section.
We have found a few key topics that come up in all stages of a consultation. These are:

• Accessibility. It is important to have a shared understanding of the topic that is being consulted upon, as well as have clear questions, and availability in more than one language (if applicable).
• Public and private channels. While consultations are usually open for anyone to read, having a private channel may encourage sensitive feedback that wouldn't otherwise be offered in a public space.
• Actionable data. All the work done in a consultation, from question design, to finding your audience, to the final report, should focus around actionable data and clear next steps.

Before

Content ▼ Plan enough time to create the content of the consultation, and understand how much time it will take for you to coordinate with different experts and stakeholders. Ask someone with experience about how much time each stage of the consultation will take, and work with the different people involved to make sure everyone agrees to your timeline. Define a clear scope of themes that you need information about. Be clear about what parts of the scope you are able to change, and what has already been decided in advance. You want to get information you can act on. Framing your questions clearly and specifically is one way to get information you can act on. Simple, short, and direct questions like "What are your concerns about this idea?" and "What do you like about this idea?" got some of the best feedback when addressed to the right audience. Bring in survey expertise, and keep survey experts involved in developing the content of the consultation. This will help the team design effective questions that will get the exact information you need, and balance the advice of the survey expert with other priorities. Your survey experts can also help your team make a plan to analyze the consultation content in advance, to make sure you will get the exact information you need through the consultation. Make a process to make sure everyone on your team agrees about the content of the consultation. Think of the criteria to pick / veto / modify questions, and focus discussions around that. Make your consultation accessible to people who speak different languages. Consider translating important questions and inviting participants to answer in any language. You may need time to coordinate requests for community translations and reach out to the translators community both on email lists and on Meta, through the function Notify Translators. Find out what languages are relevant for your audience. If you have responses in more than one language, you will need to translate them to one language for analysis. If you can't get something translated, apologize that you may be not using the reader's own/favourite language. Edit section Preview ▼ Screen your consultation with some relevant audiences to test the content and dynamics. You can use this as an opportunity to engage certain audiences strategically. If you include a diverse group in the preview, you may get better insights into how your larger audience will react to the consultation. Allow for agile changes while doing the screening, to make changes to the consultation content quickly. Edit section

During

Format and Distribution ▼ Use different channels (both public and private) for your consultation. While some consultations are public and open, it is highly recommended to also have a private channel where users can respond more freely about sensitive issues. Private feedback can be collected through surveys, forms, conversations, or Emails. The right combination of channels is specific to your context. Those doing consultations on tech products, for example, may find IRC (Internet Relay Chat) an effective way to collect feedback. A consultation around a sensitive topic may need a private survey. It is important for somebody on the team to respond quickly to relevant feedback. Timing is critical to engaging in productive conversations, and being present throughout the whole process is keeps participants motivated and engaged in the consultation. Questions may help clarify important aspects of the feedback you receive. This is why it is important to have someone dedicated to monitoring and responding to different conversations, and why channels that enable conversations and responses may be helpful for getting some types of feedback. You might consider implementing changes as you receive them, to show participants how your team is responding to conversations and suggestions in real time. Consider how you will distribute information about your consultation carefully, to get the results you want. Find your audience where they are, and reinforce communications by reaching out to users on the projects where they are active, regardless of the consultation topic. Some ideas for distributing information include: MassMessage village pumps. Consider translating the message to the language of the project you are posting to. Message users with 10+ edits in a relevant namespace or on a relevant project Send Email announcements to relevant mailing lists (for example, wikimedia-l and announcements-l) Central Notice banner. If your audience is specific, this may not be the best option to promote your consultation. Find your allies. People with influence will be very helpful in spreading the word, including active contributors, board members, committee members, and other leaders. Engage your allies them early on in the process, by making them a part of the planning or preview phases of the consultation. Edit section Audience ▼ Reach out to people who have real experience with the subject you want information about. If you request broad input from everyone, you may not get information you can use to define next steps. Who is most likely to be affected by any changes you make as a result of the consultation? These are your stakeholders, and part of your consultation's goal should be to engage them in the consultation. For example, contribution count might be an effective criterion for selecting people to reach out to, or to send a survey to for private feedback. For example, users with over 10 edits on Wikiproject:Women in Architecture may be stakeholders in your consultation about women biographies. Edit section

After

Discussion ▼ Follow your plan to analyze the qualitative feedback. You may be using qualitative coding if you have a large number of responses, and your coding should focus on the information that you need to get from the consultation. Consider how you will handle repetitive responses or multiple responses from the same participant. Be sure all analysts agree on any codes used in advance. Follow your plan to analyze any quantitative feedback relevant to your consultation. If your consultation includes this, you will have planned how to do this during your consultation with survey experts during the planning phase. Include other relevant information and resources to reinforce your analysis. For example, past surveys on relevant topics. Focus on actionable suggestions. Having clear suggestions to act upon after the consultation starts when designing the content of the consultation: make sure you work towards getting information you can act on from the beginning of the project. Edit section Report ▼ Publish the report right after the consultation ends. Timely reporting of outcomes helps maintain the momentum of the conversations and keep the report relevant. Respondents who invested time participating in your consultation will feel more motivated and involved, especially if changes take place short after the consultation ended. Thank participants who took part in the consultation. This may be time-consuming, but acknowledging the time it takes each participant to engage in productive conversations will pave the way for future consultations and strengthen important relationships. Have a process in place to streamline thank you notes to participants. Edit section