Community Insights Collaboration, Diversity, & Inclusion Supplement (2021)/Methodology

2021 Community Insights Community Climates Supplement

• Executive summary
• Methodology
• Statistical endnotes
• Data appendices

Appendix: Methodology

To develop measures to assess Collaborative Engagement, beginning in 2016 we consulted some of the engagement constructs assessed in our own Foundation staff assessment as well as WestEd’s California Schools (CalSCHLS) surveys School Climate and Gender & Sex-Based Harassment supplements and the Gay, Lesbian & Straight Education Network (GLSEN) US Schools Climate Survey. Items were developed after consultation of a few existing measurement frameworks and their validated questions available for adapting to our ecosystem:

• Radical Collaboration’s TLT Collaborative Skills Climate Survey framework (2006), specifically aiming to develop questions relevant to their concepts of Collaborative Intention, Awareness of Self and Others, and Problem Solving & Negotiation factors.

• Culture Amp’s engagement model, specifically aiming to develop questions relevant to their concepts of Engagement, Belonging, Enablement, and Leadership factors.

• GLSEN’s School Climate Survey, specifically aiming to develop questions relevant to the Wikimedia ecosystem develop questions relevant to their concepts of Harassment and Bullying, Safety, and Reporting Harassment and Assault.

As with the Collaborative Engagement constructs developed for the first insights survey, we sought to enhance our assessment by developing measures for key risk and protective factors for diversity and inclusion within the Wikimedia ecosystem. In 2017 we revisited our consultation of existing measures to include our own Foundation staff assessment’s Diversity & Inclusion Survey supplement, as well as the Society for Human Resource Management’s (SHRM) Diversity Survey and the Corporate Leadership Council’s Diversity Survey. Items were developed after consultation with these existing frameworks to adapt similar constructs to our ecosystem from:

• Culture Amp’s Diversity & Inclusion Survey framework drilling into diversity and inclusion aspects of Belonging.
• SHRM’s diversity model specifically aiming to develop questions relevant to their concepts of Attitudes Toward Diversity, Discrimination, Culture of Inclusion, Inclusive Interactions.
• Corporate Leadership Council’s Diversity Survey specifically aiming to develop questions relevant to their concepts of Individual Commitment to Diversity.

After developing contextually relevant items to our movement ecosystem, we consulted internally to synthesize key concepts from across the consulted frameworks into proposed question sets to capture similar social climate constructs about Wikimedia environments. Following response collection, items were explored to confirm the reliability of the scale items as a construct and confirm the factor loading of our developed question sets.

These social climate constructs are intended to help gauge and track progress in reaching these shared goals. For this reason, it is not only important to understand the audience differences in the measured experiences, but also to understand the attitude target of survey participants’ ratings. In 2019, we examined just that.

For this supplement, we also assigned a single ordinal category code to each survey participant according to their highest leadership level of engagement in the movement with the following ordered classification system: Editor (1), Developer (2), On-wiki Admin (3), Movement Organizer (4; includes any grantees, affiliates, and/or volunteer organizers), and Organizing Admins (5; On-wiki Admins who also organize through grants, events, or volunteer programs and events). For the most part, all audiences were considering online projects, primarily Wikipedia (64-68%), other Wikimedia projects (34-45%), or, for developers and affiliates, their own developer (23%) or affiliate group (35%) respectively. This was especially the case for questions focused on Strategy Leadership, Engagement, Collaboration, Awareness of Self & Others, and Problem-Solving and Negotiation, where 39-41% of developers and affiliates would report reflecting on those community spaces compared to the normal 10% and 20% distribution for the affiliate and developer community spaces, respectively. Overall, Editors were significantly less likely to reflect on non-Wikipedia projects than either Developers or Organizers, and Developers were three times more likely than editors to reflect on an Affiliate group.

For the construct data as a whole, scales had good reliability for the most part. However, skewness and kurtosis were significantly non-normal with a regular tendency of moderate to strong negative skew and occasional strong positive kurtosis for certain constructs, demonstrating that the distribution tails extend further than normal in comparison to the distribution mean.

Although results showed the data as non-normal and in violation of the assumptions of homogeneity of variance, the suggested statistical transformations (i.e. reflect and square root OR reflect and log transform), while successful at reducing the skewness for the most part and kurtosis to a great extent, were not successful at helping to normalize the distribution to any significant effect. Thus, the original data were maintained for analysis and nonparametric tests have been used in place of more powerful tests which rely on meeting the statistical assumptions of normal distributions. For this reason, both means and medians are reported in the footers of data tables as well as in more detail in the appendix (See Appendix: Methodological and statistical endnotes).

Limitations and next steps

• There was a lack of sufficient statistical power inhibiting our ability to enter all predictive factors into a single model. It is important to continue to balance randomization of presented scales within the survey with continued efforts to increase the response rates, as well to ensure adequate data capture for more elegant statistical modeling.
• The “Awareness of Self & Others” and “Non-Discrimination” construct items continue to have low reliability according to their Cronbach’s alpha and continued work to refine the indicators is recommended.

Appendix. Methodological and statistical endnotes

Community Insights survey participants were weighted based on their sampling group and monthly edit count used for sampling, and compared across groups to understand differences in line with the main report and analysis (See also Survey Methodology and the tab labelled “Appendix: Methodological and Statistical Endnotes”). The analysis in this supplement diverges slightly from the main Community Insights report and analysis; here, all participants have been coded to one of five contributor groups, associated with their highest level of leadership in the movement (i.e. Editor, Developer, Admin, Movement Organizer, Organizing Admins) and those Movement Organizers who were sampled outside of the Editors sample are included in the between-groups analysis using a weight of “1”. As many as 2193 and as few as 568 individuals responded to the various question sets to capture our social climate factors. Participant counts for the different question sets vary as some were randomized to shorten survey length while others were presented to every survey participant (See Appendix: 2020 Descriptive statistics). All responses were collected on a 5-point likert scale of agreement with an option to respond “unsure.” All responses within each question set were scored such that a higher score is positive; this means that some items were reverse scored where appropriate (as noted) and averaged to produce each factor score presented.

Along with the following statistical tests, means and medians are also reported in the data tables for clarity even though assumptions of normality and outliers are violated. In addition, comparisons of percentage favorable have been included in the narrative to help translate the data into more meaningful terms; these data, along with all key aggregates of mean data, will be available in the interactive report after publication and can be accessed here. Sometimes t-tests data are also reported alongside Mann-Whitney U results to ease interpretation.

1. Due to the non-normal distribution of the data which could not be corrected via statistical transformation, a Kruskal-Wallis test was conducted to determine if there were differences in demographic representativeness between contributor groups. There were differences for gender representativeness. Specifically, ratios of participants who identified as Men were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in the proportion of participants identifying as men were statistically significant between contributor groups, χ2(4) = 77.831, p = <.001, N = 1694. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant differences between Movement organizing Admins (mean rank = 906.13) and Movement Organizers (mean rank = 700.03) (p = .002) as well as between Movement Organizers and Editors (mean rank = 872.70) (p = .000), Movement Organizers and Developers (mean rank = 905.38) (p = .010), Movement Organizers and non-organizing Admins (mean rank = 917.25) (p = .000); but not between any other group combination.
2. Due to the non-normal distribution of the data which could not be corrected via statistical transformation, again a Kruskal-Wallis test was conducted to determine if there were differences in the distribution of contributor groups based on geography: distributions of contributor types were not similar for all regions, as assessed by visual inspection of a boxplot. Distributions were statistically significantly different between the different contributor groups, χ2(5) = 103.392, p = .000, N =1716. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant differences between Europe (mean rank = 818.72) and Africa (mean rank = 1359.64) (p = .000), and Europe and Asia (mean rank = 911.30) (p = .005); as well as between Africa and Oceania (mean rank = 740.45) (p = .000), Northern America (mean rank = 863.80) (p = .000), Latin America & the Caribbean (mean rank = 880.63) (p = .000) and Asia (p = .000), but not between any other group combination. Spaces with higher mean ranks were represented more by Movement Organizers and Admins than less engaged active Editors.
3. Due to the non-normal distribution of the data which could not be corrected via statistical transformation, a Kruskal-Wallis test was conducted to determine if there were differences in Collaborative Engagement factor scores between contributor groups: Editors, On-wiki Admins, Developers, Movement Organizers, and Movement Organizing Admins. (Note: n-value varies by item see details in Appendix: 2020 Descriptive statistics for n-values, means, and medians)
   1. Distributions of Awareness of Self & Others scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Awareness of Self & Others scores were statistically significant between the different contributor types, χ2(4) = 13.099, p = <.011, N = 1200. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant higher Collaborative Intention scores among Movement Organizers (mean rank = 673.71) compared to non-organizing On-wiki Admins (mean rank = 548.77) (p = .040), and Editors (mean rank = 590.12) (p = .018) but not between any other group combination.
   2. Distributions of Collaborative Intention scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Collaborative Intention scores were statistically significant between the different contributor types, χ2(4) = 27.396, p = <.001, N = 1210. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant higher Collaborative Intention scores among Movement Organizers (mean rank = 716.51) compared to non-organizing On-wiki Admins (mean rank = 562.10) (p = .004), and Editors (mean rank = 584.43)(p = .000) but not between any other group combination.
   3. Distributions of Engagement scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Engagement scores were statistically significant between the different contributor groups, χ2(4) = 62.196, p = < .001, N = 2119. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant lower Engagement scores among Editors (mean rank = 1015.62) (p = .000) and Admins (mean rank = 978.83) (p = .000) compared to Movement Organizers (mean rank = 1257.31); as well as for non-organizing Admins (p = .000), Developers (p = .033) and Editors (p = .000) compared to movement-organizing Admins (mean rank = 1411.32); but not between any other group combination.
   4. Distributions of Feelings of Belonging scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Feelings of Belonging scores were statistically significant between the different contributor groups, χ2(4) = 90.705, p = .000, N = 1777. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant lower Feelings of Belonging scores among Editors (mean rank = 826.74) compared to Movement Organizers (mean rank = 1086.36)(p = .000) and movement-organizing Admins (mean rank = 1264.06) (p = .000); as well as between non-organizing Admins (mean rank = 898.44) and Movement Organizers (p = .003) and movement-organizing Admins (p = .000); but not between any other group combination.
   5. Distributions of Fairness scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Fairness scores were statistically significant between the different contributor groups, χ2(4) = 29.135, p = <.001, N = 1767. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant higher Fairness ratings among non-organizing Admins (mean rank = 1080.93) compared to Editors (mean rank = 860.57) (p = .000), Developers (mean rank = 793.33) (p = .020), and Movement Organizers (mean rank = 888.48) (p = .001), but not between any other groups.
   6. Distributions of Movement Leadership scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Movement Leadership scores were statistically significant between the different contributor groups, χ2(4) = 31.575, p = <.001, N = 2206. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant lower scores among Admins who did not organize (mean rank = 827.57) compared to Movement Organizers (mean rank = 1154.28) (p = .000), and higher scores among Editors who were not admins (mean rank = 1117.32) compared to Admins as well (p = .000), but not between any other groups.
   7. Distributions of Movement Strategy scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Movement Strategy scores were statistically significant between the different contributor groups, χ2(4) = 44.974, p = <.001, N = 558. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant lower Movement Strategy scores among non-organizing Admins (mean rank = 187.86) compared to Editors (mean rank = 259.29) (p = .017), Movement Organizers (mean rank = 327.58) (p = .000), and movement-organizing Admins (mean rank = 291.13) (p = .014), as well as higher scores among Movement Organizers compared to Editors (p = .000), but not between any other group combination.
4. There are some differences in the year-over-year analysis completed this year compared to previous years which excluded those who did not progress to at least 50% completion of the survey. That cut point was required to create stability with the previous 2018 comparison year. This year’s report does not exclude by progress cut-point and thus also changes the 2019 comparison statistics some from what was reported previously. As reported in the methodology of the 2019 data report, this cut-off point tended to more frequently exclude from the sample those sharing less favorable ratings (who come from less engaged roles) than those with more favorable ratings (who are more deeply engaged in movement roles). Due to the non-normal distribution of the data which could not be corrected via statistical transformation, a Mann-Whitney U-test was used to determine if there were differences in factors scores between 2019 and 2020 data. (Note: n-values vary by indicator and year and are specified in the parenthetical notes). When compared to 2019, an independent sample U-test found contributors more likely to report experiencing higher levels of Engagement (2020 mean = 4.14, N =3683, t = 6.918; U = 1876791.500, 2019 mean rank = 1701.51, 2020 mean rank = 1945.70, p = <.001), Feelings of Belonging (2020 mean = 3.70, N =3025, t = 4.637; U = 1216015.00, 2019 mean rank = 1427.13, 2020 mean rank = 1573.31, p = <.001), Movement Leadership (2020 mean = 3.33, N = 4277, t = 3.232; U = 2413761.5, 2019 mean rank = 2076.49, 2020 mean rank = 2197.68, p = .001) and Fairness (2020 mean = 2.82, N = 2982, t = 4.506; U =1173114.50, 2019 mean rank = 1409.47, 2020 mean rank = 1547.90, p = <.001). While these may indicate a true difference, we recommend caution as we also have undergone changes to the sample recruitment methodology moving from opt-in email invitation means from repeated talkpage posting of invitation reminders. It is unknown what effect this may also have had on the comparison. (See also Appendix: Changes from 2019 to 2020)
5. Due to the non-normal distribution of the data which could not be corrected via statistical transformation, again a Kruskal-Wallis test was conducted to determine if there were differences in Diversity & Inclusion factor scores between contributor groups as follows:
   1. Distributions of Non-Discrimination scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Non-Discrimination scores were statistically significant between the contributor groups, χ2(4) = 36.513, p = <.001, N = 1817. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant discrimnation being reported more often by Admins and Movement Organizers than Active Editors without these responsibilities. Non-Discrimination scores among Editors (mean rank = 948.59) were higher compared to Admins (mean rank = 820.05) (p = .033), organizing Admins (mean rank = 685.73) (p = .004), and Movement Organizers (mean rank = 808.89; p = .000), but not between any other group combination. When looking more closely at the two items underlying this construct with a binary lens for those who had experienced harassment in the last year vs those who had not, these associations become more pronounced, χ2(4) = 63.018, p = .000, N = 1598. This is also the case for those who had been made to feel unsafe or uncomfortable vs those who had not, χ2(4) = 17.656, p = .001, N = 1764. Post hoc analysis revealed experiences of harassment and discrimination being experienced more often by Movement Organizers (mean rank = 909.63), and especially Organizing Admins (mean rank =1069.19 ), than active Editors (mean rank = 760.19, p = .000) without these responsibilities as well as for Organizing Admins compared to On-wiki Admins who did not organize (mean rank = 841.58, p = .006). Having felt unsafe or uncomfortable contributing to Wikimedia projects in the last 12 months was also more prevalent among On-wiki Admins (mean rank = 974.94; p = .032), and Movement Organizers (mean rank =933.03 ; p = .059) compared to Editors (mean rank = 857.97) but this was less pronounced.
   2. Distributions of Inclusive Culture scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Inclusive Interactions scores were statistically significant between the different contributor groups, χ2(4) = 18.762, p = <.001, N = 1651. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant lower scores among Editors (mean rank = 806.75) (p = .002) and Admins (mean rank =751.74) (p = .007) compared to Movement Organizers (mean rank = 919.38), but not between any other groups.
   3. Distributions of Individual Commitment to Diversity scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Individual Commitment to Diversity scores were statistically significant between the different contributor groups, χ2(4) = 20.852, p = <.001, N = 1289. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant lower scores among Editors (mean rank = 625.33) (p = .001) and Admins (mean rank = 769.01) (p = .020), compared to Movement Organizers (mean rank = 733.32), but not between any other groups.
   4. Distributions of Leadership Commitment to Diversity scores were not similar for all groups, as assessed by visual inspection of a boxplot. Differences in Leadership Commitment to Diversity scores were statistically significant between the different contributor groups, χ2(4) = 29.981, p = <.001, N = 1204. Subsequently, pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons. This post hoc analysis revealed statistically significant higher scores among Movement Organizers (mean rank = 702.78) compared to Editors (mean rank = 576.66) (p = .000) and Admins (mean rank = 555.95) (p = .006), but not between any other groups.
6. Due to the non-normal distribution of the data which could not be corrected via statistical transformation, a Mann-Whitney U-test was used to determine if there were differences in factors scores between 2019 and 2020 data. (Note: n-values vary by indicator and year and are specified in the parenthetical notes). Once again, year-over-year analysis included only the Editors group for which we are able to apply partial propensity score matching to weight for better representation based on the higher tendency for our more active editors to both start and complete the survey. When compared to 2019, an independent samples U-test found overall that contributors reported higher levels of Non-Discrimination (mean = 4.31, N = 3074, t = 2.101; 2020 mean rank of 1563.04 compared to 1500.58 in 2019, U = 1188391, p = .036). While this may indicate a true difference, again, we recommend caution as we also have undergone changes to the sample recruitment methodology and, in addition, one of the underlying items changed presentation protocols: where previously only those who first reported witnessing harassment were asked how often in 2019, in 2020 all participants were asked how often, with an option for “never,” without a screening question. While we have worked to ensure alignment as best as possible, it is unknown what effect this may also have had on the comparison. (See also Appendix: Changes from 2019 to 2020)

Appendix: Changes from 2019 to 2020

Collaboration & Engagement Factors

Construct		Mann-Whitney U
	Mean 2020	Mean Rank 2019	Mean Rank 2020	Change 2019-2020	Sig.
Awareness of Self & Others	2.93	804.66	803.78	▽	0.973
Collaborative Intention	3.46	809.86	806.71	▽	0.906
Engagement (Key to Inclusion)	4.14	1701.51	1945.7	▲	<.001
Fairness	3.72	1409.47	1547.9	▲	<.001
Feelings of Belonging	3.70	1409.47	1547.9	▲	<.001
Movement Leadership	3.70	2076.49	2197.68	▲	0.001
Movement Strategy	3.24	562.57	533.98	▽	0.132
Problem Solving & Negotiating	3.27	n/a	n/a	⌧	0.376

Diversity & Inclusion Factors

Construct		Mann-Whitney U
	Mean 2020	Mean Rank 2019	Mean Rank 2020	Change 2019-2020	Sig.
Non-Discrimination	4.31	1500.58	1563.04	▲	0.036
Inclusive Interactions	3.65	818.62	845.94	△	0.311
Inclusive Culture	3.47	1356.54	1413.72	△	0.061
Individual Commitment to Diversity	4.05	908.39	905.73	▽	0.921
Leadership Commitment to Diversity	3.46	771.09	807.42	△	0.171

Appendix: 2020 Descriptive statistics

	Editors			On-wiki Admins			Developers			Movement Organizers			Movement Organizer Admins			N	Overall Mean	Cronbach's Alpha
	n	Mean	SE	n	Mean	SE	n	Mean	SE	n	Mean	SE	n	Mean	SE
Awareness of Self & Others	893	2.92	0.02	84	2.87	0.07	23	2.81	0.11	195	3.09	0.05	22	2.96	.15	1148	2.93	0.61
Collaborative Intention	901	3.44	0.02	84	3.42	0.07	23	3.46	0.13	195	3.69	0.05	23	3.51	0.17	1156	3.46	0.79
Problem Solving & Negotiating	867	3.25	0.03	83	3.23	0.09	23	3.34	0.18	189	3.44	0.06	22	3.40	0.21	1116	3.27	0.77
Engagement ***	1633	4.11	0.02	143	4.13	0.05	38	4.04	0.15	315	4.38	0.03	46	4.47	0.10	2059	4.14	0.77
Fairness***	1315	3.66	0.03	125	4.19	0.08	31	3.59	0.18	287	3.74	0.06	39	4.00	0.17	1698	3.72	--
Feelings of Belonging***	1324	3.62	0.02	124	3.83	0.07	31	3.85	0.13	292	4.06	0.04	40	4.30	0.09	1708	3.70	0.68
Movement Leadership***	1705	3.37	0.02	140	3.00	0.07	38	3.35	0.15	326	3.41	0.05	45	3.35	0.13	2124	3.33	0.69
Movement Strategy	220	3.28	0.06	59	2.84	0.12	18	3.16	0.22	203	3.66	0.06	38	3.39	0.18	431	3.24	0.85
Leadership Commitment to Diversity	895	3.43	0.03	84	3.43	0.07	25	3.54	0.20	200	3.74	0.06	25	3.90	0.16	1160	3.46	0.87
Non-Discrimination*	1372	4.38	0.03	122	4.13	0.10	31	4.23	0.21	288	4.10	0.06	41	3.91	0.18	1753	4.31	0.61
Inclusive interactions	943	3.64	0.03	84	3.75	0.07	25	3.60	0.19	204	3.71	0.06	26	3.84	0.16	1213	3.65	0.65
Inclusive Culture	1194	3.46	0.02	121	3.45	0.07	31	3.50	0.15	287	3.63	0.05	39	3.65	0.14	1571	3.47	0.77
Individual Commitment to Diversity	976	4.03	0.02	91	3.99	0.06	14	4.08	0.18	199	4.21	0.04	37	4.27	0.08	1246	4.05	0.66

Explore the interactive appendix in Google Data Studio!

Demographics of the analyzed sample set

	Overall	Editors	Developers	On-wiki Admins	Movement Organizers	Movement Organizer Admins
Wikimedia Movement Participation
Online Contributor	100%	100%	97%	99%	88%	100%
Software development	4%	0%	100%	16%	4%	58%
On-wiki Administration	12%	0%	0%	100%	22%	100%
Movement Organizing	10%	0%	0%	0%	79%	58%
Affiliate	3%	1%	1%	0%	27%	58%
Native Language
English	25%	24%	19%	33%	24%	11%
Another language	75%	75%	81%	67%	76%	89%
Another language not translated	12%	9%	20%	24%	24%	31%
English Fluency
English Fluency	82%	81%	98%	93%	87%	77%
No English Fluency	18%	20%	2%	7%	13%	23%
Gender
Men	84%	85%	88%	74%	63%	90%
Non-Men	17%	15%	12%	9%	37%	10%
Women	15%	14%	*	*	34%	*
Non-Binary	2%	*	*	*	*	*
Other Gender	2%	*	*	*	*	*

* N too small to report at this level of disaggregation

Appendix: Metrics to movement goals mapping

#1 Promote sustainability and resilience Support dynamic volunteer base, staff, and local groups in new acquiring funds and resources.	#6 Foster and develop distributed leadership Train, support and retain socially- and technically-skilled individuals from different backgrounds that reflect the diversity of the global communities.
#2 Create cultural change for inclusive communities Foster an inclusive, welcoming, safe, and collaborative environment for future growth.	#7 Invest in skills development Foster technical and people-centered skills - e.g. communication, conflict resolution, intercultural dialogue.
#3 Improve user experience Research, development and testing new features to enable contributors from all backgrounds to enjoy a fluid, effective, and positive experience.	#9 Coordinate across stakeholders Cooperate and collaborate with different stakeholders to advance towards more equitable decision-making, with relevant training and enabling systems.
#4 Provide for safety and security Ensure contributors have the proper conditions and resources enabling them to work without compromising their security.	#13 Plan infrastructure scalability Create a fluid infrastructure to serve our needs as we grow; establish protocols, communication, and roles and responsibilities to invest sufficient resources for scalability
#5 Ensure equity in decision-making Ensure a diversity and richness of perspectives, by focusing on the knowledge and communities that have been left out due to power and privilege.

Factor/Metric	Reporting Favorable Conditions	YoY Trend	Strategy Recommendation
			#1	#2	#3	#4	#5	#6	#7	#9	#10
Awareness of Self & Others	19%	▽			O	O					O
Collaborative Intention	59%	▽	O	O	O		O	O
Engagement***	87%	▲	O	O	O	O	O	O	O	O
Fairness***	66%	▲				O	O				O
Feelings of Belonging***	73%	▲	O	O		O	O	O		O
Inclusive Culture	56%	△	O	O	O	O	O		O	O
Inclusive interactions	69%	△	O	O	O	O	O		O	O
Individual Commitment to Diversity	85%	▽		O				O
Leadership Commitment to Diversity	58%	△	O	O	O		O	O		O	O
Movement Leadership***	54%	▲	O				O	O	O	O	O
Movement Strategy	53%	▽	O				O	O	O	O	O
Non-Discrimination*	83%	▲	O	O	O	O
Problem Solving & Negotiating	55%	⌧	O		O	O		O	O		O

Appendix: 2020 Collaborative Engagement item averages by audience

2019 and 2020 data for this report, including the base questions on which the Collaborative Engagement and Diversity & Inclusion factors were constructed, can now be viewed on the Collaboration, Diversity & Inclusion: Interactive Data Report.