Which were your team's most important research questions?
The primary focus at the department level was overall contributor engagement, and we came up with questions engineered to measure that on multiple dimensions: (a) overall engagement with contributing; (b) the ability to collaborate on projects with others; (c) the ability of users to solve problems individually and collectively; (d) the perception users have of their own biases and those of others and how they impact behavior; (e) the extent to which users feel empowered to work; (f) the extent to which users feel recognized for their contributions; and (g) the high-level, overall assessment of Foundation stewardship. The clusters of questions were engineered to cross-check each other for accuracy. (a)-(f) essentially are all components of a strong and healthy online engagement experience, but (a) provides us with the clearest metric of contributor motivation to continue on the sites.
To what extent were these questions answered?
Some of the constructions didn’t support each other as expected, so data from (e) and (f) was not really usable. (d) was closer in consistency, but didn’t hang together as well as we’d want to really draw strong conclusions from that, either, except that we could tell that people have no great confidence in the ability of others to recognize their own biases. As some of these questions were adapted from other uses, it is possible that we modified them to the point that they no longer accurately assess what we intended. Where they are important for the upcoming strategic direction, we’ll re-evaluate. I'm pleased that (a)-(c) and (g) held together.
What additional analyses does your team need?
Jaime Anstee will explore comparative analyses looking at potential gender or language community differences along the community health constructs. Specifically, looking within enwiki editors across home languages, differences across language groups more generally, and potential gender differences. Courtesy of our access to Jaime, we do not need assistance with this, just the data.
Which questions surprised your team?
While data on harassment is about where I expected, this information surprised me: "Of all survey participants, 49% indicated they avoided Wikimedia projects at least one day in the past month because they felt unsafe or uncomfortable". This is a higher aversion rate than I expected.
How does your team you plan to use the data?
Looking at (a)-(c) and (g), the data suggests that individuals are generally likely to feel good about their own contributions and interested in continuing to contribute, but see areas for improvement in collaboration and problem-solving. We also see growth areas in Foundation stewardship. As we approach annual planning, I have begun talking with team leads and with the leads of other departments about some of the results and how to do further analysis to determine the interplay of the data with the emerging strategic direction. The data should offer us a baseline to track the effectiveness of measures we take in these areas. The effectiveness of the data for this use will increase as Jaime further analyzes it. (The community health data beginning at 115.01 is likely of greatest use to Support & Safety and should be prioritized by them; this answer corresponds to the department "team" level.)
What survey questions would your team like to ask next time?
We'd love to continue to measure the constructs set out in this survey but recognize that the survey overall suffered from bloat and that we contributed to this. Accordingly, we will review our constructs against strategic priorities defined in the current strategy consultation to ensure that we are focusing attention in the areas directed by that consultation. Flawed constructs will be re-evaluated and re-tooled as necessary.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- My contributions to Wikimedia are challenged or reverted inappropriately.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- Other Wikimedians properly recognize good contributors.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- Wikimedians are free to experiment and try new ideas.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement, including all Wikimedia projects, organizations or offline activities: -- People are sincerely interested in supporting each other.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement, including all Wikimedia projects, organizations or offline activities: -- People are committed to building long lasting cooperative relationships.
Forty-three percent of all participants indicated they agree or stongly agree people are interested in building long lasting cooperative relationships.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement, including all Wikimedia projects, organizations or offline activities: -- People can be hyper competitive and see others as adversaries, seeing conflict as a battle to win at any cost.
Forty-eight percent of all participants agree or strongly agree that people can be hyper competetive and see others as adversaries, seeing conflict as a battle to win at any cost.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement, including all Wikimedia projects, organizations or offline activities: -- Wikimedians with collaborative projects work together well.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement, including all Wikimedia projects, organizations or offline activities: -- Wikimedia contributors are good at reaching out when they need to consult with others.
Fifty-six percent of all participants agree or strongly agree that Wikimedia contributors are good at reaching out when they need to consult with others.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement, including all Wikimedia projects, organizations or offline activities: -- Wikimedia contributors have sufficient freedom to work effectively online.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- People are dedicated to working together to solve problems.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- People are committed to seeking fair solutions that are responsive to the interests of all parties.
Fifty-seven percent of all participants agree or strongly agree that people are committed to seeking fair solutions that are responsive to the interests of all parties.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- People are sincerely willing to talk and negotiate and are able to separate “people issues” from the problems.
Forty-one percent of all participants agree or strongly agree that people are sincerely willing to talk and negotiate and are able to seperate "people issues" from the problems.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- People lack self-awareness and awareness of what motivates others.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- People are aware of their biases and patterns of behavior.
To what agreement do you agree or disagree with each of the following related to the Wikipedia movement: -- People are not aware of their own defensiveness to the situation at hand.
To what extent do you agree or disagree with each of the following related to the Wikimedia movement: -- People have a difficult time understanding and empathizing with others.
To what extent do you agree or disagree with each of the following: -- The Wikimedia Foundation makes it clear that volunteers and partners are key to the success of the movement.
Seventy-eight percent of participants agree or strongly agree that the Wikimedia Foundation makes it clear that volunteers and partners are key to the success of the movement.
To what extent do you agree or disagree with each of the following: -- The Wikimedia Foundation communicates well about their projects and initiatives.
19% of all survey participants reported that they have avoided twitter for at least one day in the last month because they felt uncomfortable or unsafe in the space.
During the past month, on how many days have you avoided the following spaces online because you feel uncomfortable or unsafe in the space?--Wikimedia Projects
Of all survey participants, 19% indicated they have avoided another space not listed at least one day in the past month because they felt unsafe or uncomfortable
In the past 12 months, how often have you been sexually harassed on Wikimedia projects or at Wikimedia events, such as sexual remarks made toward you or someone touching your body inappropriately? -- Wikimedia projects (online)
In the past 12 months, how often have you been sexually harassed on Wikimedia projects or at Wikimedia events, such as sexual remarks made toward you or someone touching your body inappropriately? -- Wikimedia events (offline)
In the past six months, how often did you report to the following groups for being harassed or attacked: -- Functionaries (bureaucrats, stewards, arbitrators, users who perform specialized roles and have specialized technical access, etc.)
In the past six months, how often did you report to the following groups for being harassed or attacked: -- Other volunteers (anyone who contributes to Wikimedia)
Overall, how useful was the response you received the last time you reported it to: -- Functionaries (bureaucrats, stewards, arbitrators, users who perform specialized roles and have specialized technical access, etc.)
Of the people who reported harassment or attacks to Wikimedia Foundation staff in the past six months, 77% indicated the response was not at all useful.
Of the people who reported harassment or attacks to chapter or affiliate staff in the past six months, 75% indicated the response was not at all useful.
To what extent was the problem resolved the last time you reported it to: -- Functionaries (bureaucrats, stewards, arbitrators, users who perform specialized roles and have specialized technical access, etc.)
From all survey participants, 2006 was the year the commonly reported (11%) in which survey participants began contributing to the Wikimedia movement or its projects.
1 – Your race or ethnicity or because people think you are of a certain race or ethnicity (8%)
2 – Your level of education or what people presume your level of education to be (12%)
3 – Your citizenship status (7%)
4 – Other reason (please specify): (35%)
5 – None of the above (19%)
6 – I do not feel unsafe contributing to Wikimedia. (18%)
7 – Your sexual orientation (for example, being gay, lesbian or bisexual) or how you present your sexual orientation (4%)
8 – Your gender (5%)
9 – How you express your gender (how traditionally “masculine” or “feminine” you are in your appearance or in how you act) (3%)
35% of participants selected "Other reason" and 18% selected "I do not feel unsafe" as the reasons they feel unsafe when contributing to the Wikimedia projects. From the specific reasons, the most often selected was education level or presumed education level (12%)."
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of...--Your race or ethnicity or because people think you are of a certain race or ethnicity
Forty-one percent of survey participants indicated in the past 12 months they have sometimes,often or almost always been attacked because their race or ethnicity.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your sexual orientation (for example, being gay, lesbian or bisexual) or how you present your sexual orientation
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your gender
Fifty-one percent of survey participants indicated in the past 12 months they have sometimes,often or almost always been attacked because of their gender.
Be cautious with interpreting the results. Read more >>
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- How you express your gender (how traditionally “masculine” or “feminine” you are in your appearance or in how you act)
Fifty-seven percent of survey participants indicated in the past 12 months they have sometimes,often or almost always been attacked because of how they express their gender.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your disability or because people think you have a disability
Most participants (87%), indicated in the past 12 months they have never or rarely been attacked because their disability or people thinking they have a disability.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- You age or because people think you are of a certain age
Forty-one percent of survey participants indicated in the past 12 months they have sometimes,'often or almost always been attacked because their age or because people think they are a certain age.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your religion or because people think you are of a certain religion
Fifty-two percent of survey participants indicated in the past 12 months they have sometimes,'often or almost always been attacked because their religion or because people think they are a certain religion.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your body type (size, weight, height, etc)
Four of the six (67%) participants who answered this question indicated they have been attacked at least once in the past 12 months because of their body type.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your income or economic status
Eighty-four percent (21) of participants who answered this question indicated they have been attacked at least once in the past 12 months because of their economic status.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your level of education or what people presume your level of education to be
Fifty-three percent of survey participants indicated in the past 12 months they have sometimes,often or almost always been attacked because of their level of education or percieved level of education.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of... -- Your citizenship status
Forty-one percent of survey participants indicated in the past 12 months they have sometimes,often or almost always been attacked because of their citizenship status.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of...--Other reason (please specify)
Forty-four percent of survey participants indicated in the past 12 months they have sometimes,often or almost always been attacked because of another reason not previously listed.
This question has errors in question design. Specifically that "none of the above" should not be a scale. Please interpret the results with caution.
In the past 12 months, how often have you been attacked (name-calling, threats, etc. directed at you) when contributing to a Wikimedia project because of...-- None of the above
Forty-seven percent of participants indicated that in the past 12 months they have sometimes,often or almost always been bullied or harassed on Wikipedia.
Eighty-eight percent of participants indicated that in the past 12 months they have never been bullied or harassed on another platform other than the ones previously listed.
1 – Advocating to change laws that influence the Wikimedia movement (20%)
2 – Wikimedia movement fundraising efforts (9%)
3 – Partnerships with governments, universities, companies or other organizations (29%)
4 – Technical contributions, including everything from giving feedback, testing, and writing code (25%)
5 – None of the above (49%)
From survey participants, 49% were not strongly interested in any of these areas, while 29% were interested in partnerships, 25% in technical contributions, 20% in advocating to change laws, and 9% in movement fundraising.