Organizational effectiveness/Tool/User guide

Translate this page

Wikimedia organizations: Please read this Introduction before taking the Organizational Effectiveness Questionnaire.

This document will explain what the organizational effectiveness tool is, and how your organization can prepare for the questionnaire and interpret your results report.

Welcome to the Wikimedia Organizational Effectiveness Tool User Guide.

This guide will help you prepare to take the Wikimedia Organizational Effectiveness Questionnaire, and will help you interpret your organization's results.

Introduction

Context and purpose

The Wikimedia Foundation has initiated a project with TCC Group – a consulting firm focused on social impact – and the larger Wikimedia community, to help Wikimedia organizations of all shapes and sizes, to improve their effectiveness and their ability to have impact for the movement. The project was initiated in response to growing interest on behalf of volunteers and organizations, to better understand how organizations in particular have impact in the Wikimedia movement, which is unique in that it is online, growing extraordinarily fast, and created and supported almost entirely by volunteers.

The project seeks to help you understand 1) how impact is defined from an organizational perspective, 2) what strategies Wikimedia organizations use to achieve that impact, and 3) what resources and skills Wikimedia organizations may need to be more effective in the strategies they choose to pursue.

In the first “impact” stage of the project, TCC interviewed several organizations, administered an impact survey to all organizations, and attended Wikimania 2014 in London where the project was discussed with AffCom, an informal OE working group, the FDC and other grantmaking committees, as well as other individuals in the movement. TCC also conducted in-depth research on three organizations, resulting in case studies illustrating different organizational models leading to impact. Many thanks to those of you who took your valuable time to participate in one or more of these exchanges; Wikimedians around the globe created the foundation for this work.

The second and third stages of the project involve development of an online Organizational Effectiveness Questionnaire, this Guide, and a Learning Center, which may help organizations think about the different strategies they are using and how they could build specific capacities to be more successful with those strategies. TCC worked closely with a handful of Wikimedia volunteers from different organizations to design the Questionnaire you are about to take.

Wikimedia Foundation has provided funding for this project and has consulted closely with TCC over the course of the engagement. Please note that the Organizational Effectiveness Questionnaire is not intended to be a test; the Foundation will not be provided with individual organization results. As part of this consultation, TCC will aggregate findings from the movement-wide Organizational Effectiveness Questionnaire and provide them to the Foundation’s grantmaking team, along with a “capacity building roadmap” to help the team think about how organizations can build capacity for specific strategies. The Wikimedia Foundation contact for questions regarding the Organizational Effectiveness project is Winifred Olliff, <wolliff at wikimedia dot org>.

How the questionnaire, guide and learning center may help your Wikimedia Organization

The Questionnaire, Guide and Learning Center are lengthy. Using them is a significant investment of time, which is in short supply for Wikimedia volunteers. What do you get out of it?

We hope they will help you:

Start an ongoing conversation about organizational effectiveness topics with your colleagues and everyone who is interested in making your organization better; you may use this as an opportunity to develop a stronger learning culture at your organization
Truly understand what you do well in order to leverage your organization’s strengths and share them with others in the movement
Understand how your challenges and more clearly see any “gaps” in capacity that can be filled by motivated volunteers or other resources
Deepen your understanding of the specific strategies you use, and to get a sense of what kind of information you might capture to track your effectiveness in these areas
Put together a capacity-building plan for your organization that will clearly show how your organization could benefit from building specific resources, tools and skills
Understand and prioritize what your organization needs, so you may consult a list of community resources to help you put the plan into action
Make an evidence-based case with funders for requesting support in areas where your organization needs to build capacity

Part 1: Take the questionnaire

Preparing to take the organizational effectiveness questionnaire

Now you are ready to take the online Organizational Effectiveness Questionnaire. Throughout the Questionnaire and Guide we use several terms which may be unfamiliar. Below is a list of key terms or phrases, with their definitions.

Wikimedia Organization: A Wikimedia chapter, user group, thematic organization or other organized group
Capacity: The set of resources, tools and skills that an individual or organization uses or has at its disposal
Organizational effectiveness: The way a Wikimedia organization uses its capacity (resources, tools and skills) to achieve impact
Outcomes: The range of changes or benefits that may result from a particular strategy
Impact: What you hope your outcomes collectively lead to; the fundamental change you hope to see in the world as a result of your work; the reason you exist!

The Questionnaire was sent to the same individuals who received the impact survey from TCC Group, in the form of a SurveyMonkey link in an email. It should take between 30-45 minutes to complete. Before you open the link, we recommend that each organization nominate one respondent to be the “point-person” for TCC and the Wikimedia Foundation, someone with overall responsibility for the tool. We suggest that between 3 and 7 individuals from each organization take the Questionnaire on their own. If two individuals take the Questionnaire they may go through it together and submit one response. Respondents should be a mix of volunteers who are not on the board, staff, or volunteers who are on the board, who understand how the organization operates (including how it makes decisions, what activities or strategies it engages in, and how the budget is allocated). For most organizations with staff, an ideal mix of respondents would be include more volunteers than staff and would include at least an equal or greater number of volunteers who are not board members as volunteers who are board members. In some contexts, a different mix of respondents may work better, depending on who in each organization has the best knowledge.

The Organizational Effectiveness Questionnaire has some introductory language followed by 3 question sections. The first section is basic background information. The second section involves “Core Competencies,” things we believe are relevant for all organizations no matter their type (chapter, thematic organization or user group). The third section has a binary structure, in other words it asks you whether your organization engages in a particular strategy and based on your answer, may provide you with a detailed list of statements. The Questionnaire will ask whether you disagree or agree with the statements on a scale of 1 to 5. For TCC to receive your responses, you must click “Done” at the end of the Questionnaire.

If you have any technical issues with the Organizational Effectiveness Questionnaire, you can contact Rika Gorn at TCC Group <rgorn at tccgrp dot com>.

How to get your results

Once all participants have submitted complete response(s), please have the point-person send an email to Rika Gorn at TCC Group <rgorn at tccgrp dot com>. TCC will then send a PDF-version of your results report. For groups where multiple people participated, numerical results will be averaged. Complete open text responses will be provided as well.

Now what?

Once you have your organization’s results in hand we recommend you share results with everyone who took the Questionnaire, and maybe even a larger group. You are now ready to move on to Part 2.

Part 2: Interpret your report

After taking the questionnaire and receiving your results

Where to start

The Questionnaire is intentionally detailed to increase the chances that it will be applicable to a broad range of Wikimedia organizations. The User Guide and Learning Center are structured to help your organization walk through your results in a clear, but hopefully not too burdensome way.

Below, we have outlined a few ways to start the interpretation process:

Put together an internal organizational effectiveness group to work through these results. This group may contain board, volunteers and staff. It is okay to have people in this group who did not take the Questionnaire but most people in the group should have strong knowledge of your organization. A few newcomers may offer fresh perspective. It may be helpful to elect a “leader” for this organizational effort who reviews this Guide in advance and also makes sure he/she understands the results in the OE Questionnaire report (this could be your point-person, or someone else). This way other members of the group will know whom to contact if they have questions.

Together, decide what your goals are for this process and how you will use what you learn. In Part 3 of this Guide we have suggested a master capacity-building plan and a process for prioritizing. You may choose this approach or something else entirely, but it will be helpful for everyone involved to know what you are trying to accomplish and why it is important. Is it to build capacity in a new area you want to pursue? Is it to make a clearer case for a future funding request? Try to find some concrete outcomes and peg them to a timeline.

Schedule a few times to meet to discuss. Trying to interpret all of your results at once might be too time consuming and encourage participants to rush through the results. Instead set up 2-3 meetings of 1-2 hours each where everyone can get together to discuss your results.

Be clear about your perspective moving forward and keep it positive. The intention behind the Questionnaire, Guide and Learning Center is to create a useful way to think about your organization’s capacity. It is not a “report card” that assigns personal blame for any capacity gaps. In TCC’s long experience with organizations taking capacity assessments, it has never encountered an organization with all capacity strengths and no capacity challenges. Try thinking about your scores as an opportunity to learn instead of a chance to grade yourself.

How to read the report

Your accompanying Organizational Effectiveness Questionnaire report has a quite a few graphs. In this section, we’ll walk you through how to understand these graphs.

Throughout the report, a high score (e.g. a 4 or 5 out of 5) indicates agreement whereas a low score (e.g. a 1 or 2 out of 5) indicates disagreement.

Numerical Score	Agreement Level with Statement
5	Strongly Agree
4	Agree
3	Neither agree nor disagree
2	Disagree
1	Strongly Disagree
Blank	(This item was skipped by all respondents)

Each graph also has a corresponding data table. An example table is below.

Data tables have the same information as graphs, presented in the same order. However, instead of providing the average of all scores for a certain item, they present all individual responses. If three people took the Questionnaire, the table would show how each person answered each question (while still keeping answers anonymous). Data tables provide a more nuanced picture. For example, an item that has two scores – one score of two and one score of four – will be averaged into three. But, an item that has two scores of three will also be averaged into three. The data table can show you if have varied responses are to a particular item, and if there is consensus or a big difference of opinion on a particular point.

How to make meaning from your report

Capacity strengths

Now it’s time to analyze your results. A good starting place for this is to look at your strengths. What is your organization doing well? Take a look at all the graphs in your Questionnaire report and identify the items where you scored the highest. Make a list of these items.

Once you have this list of capacity strengths, there are a few things to think about:

What’s common or similar across these strengths? Are there any themes?
Do these strengths resonate with your understanding of what your organization does well?
How difficult or easy is it to maintain these strengths? Are there any areas where you will have to actively work to remain strong, or where you will need resources to remain strong?
Do you have strengths in common with other Wikimedia organizations that could suggest collaborations or opportunities for knowledge sharing?
How can your organization more effectively articulate its strengths to others?
How do these strengths change your thinking about what your organization should be focusing on? Are there any strategies or activities that your organization might want to consider leveraging more?

Capacity challenges

Now look at capacity challenges. Take a look at all the graphs in your Questionnaire report and identify the items where you scored the lowest. Make a list of all these items.

Once you have this list of capacity challenges, there are a few things to think about:

Do these challenges resonate with your understanding of where your organization struggles?
Of these gaps, are there any that are significant barriers to your organization achieving impact right now?
Are there gaps in capacity that your organization can easily address, for example with little investment in volunteer time or money, or with minimal financial support?
What are the gaps in capacity that will take intense effort and significant resources (volunteer time, money) to address?
What’s common or similar across these challenge areas? Are there any themes, or can any capacity gaps be grouped together? Could addressing one gap lead to increased capacity across a range of strategies or activities?
Of these gaps, are there any that you need to address to achieve your organization’s longer term goals or strategy?
How do these capacity gaps change your thinking about what your organization should be focusing on? Are there any activities or strategies that your organization might want to reconsider?

Next steps and making a capacity building plan

Consult the Learning Center on Meta-Wiki for recommendations about the areas where your organization excels, and where your organization may be challenged. Consider using the above questions (in the section called “How to Make Meaning from Your Report “) – along with any others your group is asking – to build a prioritization table with your top strengths and challenges. This may help you focus your capacity-building work.

Think about what you do now that is truly worth building, such as areas where an investment in capacity will create concrete results and increase your organization’s impact.
If you scored low in several areas, don’t assume you need to build capacity in all of them. *What will add the most value? These will be areas that you should prioritize in a capacity-building plan.
Keep in mind that capacity means not just resources but the ability to tap into or access resources that may already exist within your organization, your volunteer base, or the movement as a whole.
Once you’ve gone through this prioritization process, collect your “top priority” strategies for capacity building in the near future, the matching suggestions in the Learning Center, and consult the community resources list for the areas you identified, to create a capacity building plan for your organization. This plan is distinct from your strategic plan or your annual plan because your organization’s priorities overall may be different from your priorities in capacity building – although there may be areas of overlap.

Table 1. Master Capacity Building (CB) Plan Template
Example Priority #1: Using Online Contests to Draw in New Contributors	Example Priority #2: Training Volunteers to Train Others through Workshops, etc.	Example Priority #3: Getting GLAMs in our Region/Thematic Focus to Liberate Content
Rationale for selection (Are we filling a gap in capacity in an area we have not pursued to date, or do we want to scale a set of strategies that we currently engage in and have resources for?)	Rationale for selection (Are we filling a gap in capacity in an area we have not pursued to date, or do we want to scale a set of strategies that we currently engage in and have resources for?)	Rationale for selection (Are we filling a gap in capacity in an area we have not pursued to date, or do we want to scale a set of strategies that we currently engage in and have resources for?)
Guide Recommendations for where/how we should build capacity to be more successful: X Y Z	Guide Recommendations for where/how we should build capacity to be more successful: X Y Z	Guide Recommendations for where/how we should build capacity to be more successful: X Y Z
Community Resources List: Organizations within the Wikimedia movement WMF contacts Other resources	Community Resources List: Organizations within the Wikimedia movement WMF contacts Other resources	Community Resources List: Organizations within the Wikimedia movement WMF contacts Other resources

Decide if further prioritization is necessary: do you have the volunteers and/or staff to devote to all areas for capacity building in the coming year? Can you work with other organizations to leverage their expertise and resources? Does the timing make sense? How do your top priorities for capacity building fit within your existing annual plan, strategic plan and budget?

Think about what parts of your plan you want to share, how you will communicate it, and who you want to be part of the conversation. This includes how you might use the master capacity building plan to make a case for more resources (including technical support or training in addition to financial resources) with funders and other partners. Look at the community resources list and decide whom you should speak with before you ask for assistance. For example, if using online contests is a strategy you would like to scale and build resources and skills for, who can you tap from within the community who would share their knowledge of what works for online contests?

Based on our experience working with organizations in similar situations, we strongly suggest breaking your plan into smaller pieces so you don’t get overwhelmed. For example, you might think about which four strategies you are most interested in building capacity for, and then plan to develop these areas at different times throughout the year rather than all at once so that you can work more intensively on building resources and skills in two to three areas per year, rather than tackling every area simultaneously.

Decide who within your organization needs to vote or approve the plan before it can start. Once the plan has gone through the approval process, decide who will take ownership of it (this could be an individual or a group or subcommittee) and the information it contains. Ownership means accountability for taking action based on your capacity building plan, scheduling check-ins to gauge progress and resolve issues, and generally keeping the community up-to-date on any developments as you go along. Consider asking about volunteer interest in creating an Organizational Effectiveness Blog or other online forum or listserv, to communicate and collect suggestions from the community related to the capacity building plan. For more ideas, you can get in touch with other organizations that are already thinking about organizational effectiveness in their own contexts.

What is next?

We are now transitioning “ownership” of this process and the Questionnaire and Guide from TCC and the Wikimedia Foundation to the organizations in the movement. We see these materials not as a conclusion, ending, or final word on whether or not your organization has the capacity it needs. Instead we see it as a jumping off point for organizations that want to strengthen organizational effectiveness to achieve more impact. Some may choose to adapt the Questionnaire to make it more applicable to their particular strategies and local context, to use it as a learning tool for themselves and their peers as part of a larger capacity building initiative, or to use it as a basis for a discussion about the roles and structures of Wikimedia organizations. Where it goes next, is up to you.