Global Data and Insights/Movement Data/Equity Landscape/Pilot & Consultation/Directed Review Questions

Pilot

_{Project Home}

• Consultation
• Calls to Action
• Input Prompts
• Early Outcomes & Adjustments
• Ask a Question or Sign-up

General Reflection Questions

First, we present five general questions related to the words and data we use which we hope most folks might have perspective on.

Question 1. Considering the output metrics developed to understand Wikimedia presence & growth; are there major domains we have failed to identify?

Click here to review the metrics at hand The metrics of the pilot and initial public dashboard will be limited to our first round of data exploration and mapping and include: The domain-level metrics Review this deck to learn more about the project The project focuses on triangulating our best available signal metrics across key engagement and enablement domains. All metrics are scaled 0-100, averaged across triangulation points, and then scaled 0-100 again so that the global percentile rank is 50 and those scores nearer 100 represent geographies signaling the most engagement in any named domain and those nearer 0 signaling the least. All scores are comparative indices in which no further modeling has been implemented beyond averaging and scaling triangulated signal metrics. The initial domains we have been mapping are: Readership Editorship Program Leadership Grants Leadership Affiliate Leadership Access Freedom WP Editor Population Penetration Population Presence & Growth The facets In addition to the main domain-level metrics, for those domains in which we have robust enough data (Readers, Editors, Grants, and Affiliates) we will also share sub-metrics along two facets: Presence Growth The high-level metrics In addition to each of the domain metrics we will share three high-level roll-up metrics: Overall Engagement Overall Enablement Rank in under-representation Additional metric domains under consideration which we hope to progress toward In progress Gender Equity Estimated Contributor Gender Gap (Estimated Contributor Gender Ratios (% women), World Bank Gender Ratios (% of total population women)) Gender Inequality (Gender Inequality Index, Gender Development Index) General Inclusion & Equity Human Development (Human Development Index (HDI), Labour share of GDP, comprising wages and social protection transfers, Income-adjusted HDI (IHDI)) Palma Ratio (Income held by the richest 10%, Income held by the poorest 40%) Income Inequality (Income Inequality GINI Index, Income Index (GNI per capita) Human Inequality (Overall loss in HDI due to inequality, Coefficient of Human Inequality) Inclusion (Looking at Peoples Under Threat & Inclusiveness Index) In exploration On-wiki Leadership (Looking at rights and use of administrative rights on the projects) Content (Total wiki content and net new content added) New Editors (New Editors, New Active Editors, and New Retained Active Editors) Content Interactions (Integrating content previews along with pageviews) Disinformation and cyber security threats (Oxford data) Partnerships (Count and Type across grantees and affiliates) Grants Leadership (Improved to integrate outcome metrics as well as grants counts and dollars) Answer on the talk page

Question 2. Are there input measures that could be improved from the landscape of existing data or measures which should not be part of the measurement framework for Wikimedia presence & growth?

Click here to review the input data to the metrics Click here to review the underlying metrics at hand Domain Measure Readers Average monthly unique devices Average monthly pageviews Editors Average monthly editors Average monthly active editors Programs Count Education Events Count GLAM Events Non-affiliate project grants Count organizing hubs engaged Grants Annual grants FY Historic grants Growth in Grants Affiliates Highest Governance Type Affiliate grants Affiliate count, size, and tenure Count new recognitions Population World Population Population Growth Access Population Accessing Internet GSMA Mobile Connectivity Score Access to Basic Knowledge Access to Information & Communications Freedom Freedom Index World Press Freedom Index Control of Corruption Score Answer on the talk page

Question 3. Are you aware of any key data sources we have missed for understanding Wikimedia presence & growth?

Answer on the talk page

Question 4. What are possible use-cases of these metrics for you?

Answer on the talk page

Question 5. Do you have any other thoughts you would like to share with us about the Wikimedia presence & growth measurement framework or the planned dashboard elements?

Answer on the talk page

Specific Design Considerations

Here we go a little deeper with design considerations on naming and analysis that are intended to be responded to as they are relevant to you as a potential data partner or user.

Consideration 1. The Nomenclature (or what do we call this project and its data)

^{Click in this space to reveal the question prompt and background information}

Do any of the proposed project, metric domain, or facet labels present potential challenges to you or your group’s planned projects and partnerships?

We have active voting on the title of the equity data landscape in All our ideas You can review the metrics schema table to review metrics names and comment with any challenges Answer on the talk page

Consideration 2. Metadata frame

^{Click in this space to reveal the question prompt and background information}

To what extent do the metadata classification systems enable or inhibit potential cross-use with datasets you or your team use to understand Wikimedia communities?

We plan to catalog the following metadata for cross-referencing across datasets: UN Country labels and codes UN Continent & Subcontinent classifications IBAN 2- and 3-Digit Country Codes Maxmind country label Wikimedia organizing hubs alignments (i.e., CEE, ESEAP, Iberocoop, WikiIndaba, North America (& US Coalition), Northern Europe, WikiArabia, WikiFranca, South Asia, Chairperson's Group, Chapter EDs Official language listings and ISO 639-1, 639-2, and 639-3 Language codes as available Answer on the talk page

Consideration 3. Referring to (In)Equity

^{Click in this space to reveal the question prompt and background information}

Does referring to equity or inequity resonate more with recent patterns of use you have seen in research and evaluation of inequity in your operating space?

Referring to equity vs inequity. Some comments have been shared that we should focus on improving equity rather than reducing inequity and labeling our aims and all coefficients positively in this way. Answer on the talk page

Consideration 4. Measuring Diversity & Inequity

^{Click in this space to reveal the question prompt and background information}

Kolm categorized absolute and relative measurements of inequality as “rightist,” “centrist,” and “leftist,” depending on their treatment of inequality as an absolute or relative concept. Leftist measures are sensitive to absolute changes; they do not change when all incomes go up by the same absolute amount. An example of this type of measure would be the absolute gini, which is a standard gini coefficient multiplied by the mean of the distribution. Kolm defines centrist measures as measures which show increased inequality when average incomes rise and the relative distribution stays the same, and decreased inequality when all incomes rise by the same absolute amount. Rightist measures are purely relative; when all incomes go up in the same proportion, they are unchanged. Using percentile ranking relies on a relative view of each group as opposed to an absolute view of capacity. When it comes to measuring change over time, we will rely on measuring the distributions of the input measures. For this we have several options: For (In)Equity: Gini coefficient (Relative measure) Absolute Gini (Absolute measure) Intermediate Gini (Combination of Relative and Absolute measure) Atkinson Inequality Index (Absolute measure) Hoover index (Absolute measure) Theil & Generalized Entropy indexes (Relative measure) 20/20 ratio (Centrist - Relative measure) Palma ratio (Relative measure) For Diversity & Dissimilarity Simpson Index Shannon's H Richness R Pielou’s Evenness E If you are experienced with any of the above, please share your thoughts on these options via direct comment. Answer on the talk page

Consideration 5. Scaling and Weighting

^{Click in this space to reveal the question prompt and background information}

For scaling and weighting we have identified some key options and are leaning to the options noted below. Please share what, if any, concerns or alternative suggestions you may have. Scaling: For relative comparisons and roll-up ranking we need to triangulate across more than one input data point to estimate a more generalized global ranking. This means scaling for relative comparison and combining for relative presence and growth. We considered raw ordered ranks across the distribution, percentile ranks across the distribution, Z-scores, and ratios. With the exception of some background calculations to determine limits to room for growth which apply ratios, we are currently applying percentile ranks 0 to 100 across all inputs for triangulation and calculation of output metrics.

Weights: Grants dollars must be weighted by the local economy and all input measures must be weighted by some population factor in calculating coefficients of inequity. For grants, we considered: median income, per capita GDP, median equivalised income, and Per capita GDP (PPP). We currently plan to use per capita GDP, PPP For GDP (PPP) weighting, there are two available weights for annual per capita GDP, PPP, and current international $ and constant international $. We currently plan to apply constant international $ for the weighting of historical grants and current international $ for the corresponding calendar year of grants. For all other input measures, we are considering weighting relevant resourcing data either by general population, readers, or editors. We currently propose to apply as follows: Once we can derive unique editor counts by geography, resources could be weighted by editor and/or active editors population for consideration of underserved editor populations in the world or by general population to consider underrepresented populations in our ecosystem Editor data could also be weighted by general population to understand a concept such as “activation rate” for growth potential or by readers to understand a concept like “under-activated” consumers. Similarly, Readers stats could be weighted by editor, and/or active editors population, and/or general population in consideration of slightly different reach and inclusion questions and arrive at different answers regarding participation gaps. Lastly, regional roll-ups could be weighted by population. Considering what engagement and resourcing gaps are most important to the strategy work you or your group engage in, please share what excitements, concerns, or curiosities you have about the above weighting options. Answer on the talk page