Research:Reducing the gender gap in AfD discussions: a semi-supervised learning approach
This page documents a planned research project.
Information may be incomplete and change before the project starts.
We propose to develop machine
learning methods for matching discussions in Article for Deletion based on the type and amount of external
evidence available to discussants. This evidence
scoring approach would form the basis for the
development of a AfD matching tool that
discussants could use to review the outcomes of
previous discussions, and check whether they
are consistent with the current one. In doing so,
we want to study the information foraging
practice of AfD participants and understand
how the language-specific affordances of the
AfD process affect notability assessments. For
example, in both the English and Bengali
Wikipedia, discussants are provided with direct
links to external sources (e.g. Google, NYT,
JSTOR), unlike in the Italian. We will thus
conduct a content analysis of a sample of AfDs
to determine the most frequent external sources
used in each community.
Of course the above questions require an
operational definition of the gender of the
subjects of AfD discussions. Here, we choose to
leverage Wikidata as the ground truth about
gender in AfD discussions. Our preliminary
analysis on AfDs in English analyzed by Tripodi
[2] shows that this approach achieves >60%
coverage on average. Furthermore, even though
our focus is on the gap in coverage between men
and women, it allows us to extend our analysis
beyond traditional genders (e.g. transgender
men, women, etc.) due to the availability of rich
gender information.
This project could help researchers and Wikipedia contributors gain a better understanding of AfD debates on biographies of women and other genders not typically considered when dealing with the gender gap in content representation. Our AfD discussion matching service could enable researchers and contributors to compare the potential outcome of ongoing discussions with that of similar discussions based on the availability and type of external evidence. We envision such a tool could promote consistency in outcomes across debates regardless of gender. For example, WikiProjects devoted to closing the gender gap, like “Women in Red”, may benefit from the ability to identify gaps in outcomes between discussions of biographies of women and men. As a proof of concept, we will build a dashboard keeping track of AfDs of biographies with relevant stats broken down by gender. We will take a number of steps to maximize the chances of adoption of our tools. We will deploy our tools on the Wikimedia cloud services and make the source available on Github. We will advertise our research on relevant WikiProjects related to gender and inclusion, like “Women in Red”, and “LGBT Studies” and invite their members to attend participatory design sessions.
Methods edit
We will perform a content analysis of biographicals AfDs to understand how discussants evaluate external sources. Using an open coding approach, we will train human annotators to identify notability assessments made by discussants within the AfD. These assessments will typically include citations to external sources, which will allow us to identify what sources AfD discussants use in practice, and thus to define metrics that operationalize the concept of external evidence.
Our next step will be an analysis of AfD debates of biographical articles, in which we will compare debates about biographies by gender. As the vast majority of Wikidata labels covers men and women, we will focus first on a comparison of these two genders. However, we will also test how our approach performs when using gender labels beyond these traditional genders. To match AfDs based on the strength and type of external evidence, we will experiment with propensity score matching, though we may consider alternative matching methods, like Coarsened Exact Matching. Thanks to the scoring method, we will investigate the effect of the gender of the AfD subject on group composition and stance of the debates, and on how editors assess external sources to gauge the notability of the subject of a biographical article. We will pre-register our study on OSF or a similar repository.
Finally, to achieve our goal of a cross-cultural study of AfD discussions, we will develop a suite of multi-lingual tools for AfD debates. Our initial goal will be to support three languages of interest (English, Italian, and Bengali) including a parser for AfD discussions, and a set of scrapers of core external sources used in each language community.
Timeline edit
Please provide in this section a short timeline with the main milestones and deliverables (if any) for this project.
Policy, Ethics and Human Subjects Research edit
It's very important that researchers do not disrupt Wikipedians' work. Please add to this section any consideration relevant to ethical implications of your project or references to Wikimedia policies, if applicable. If your study has been approved by an ethical committee or an institutional review board (IRB), please quote the corresponding reference and date of approval.
Results edit
Describe the results and their implications here. We encourage you to share preliminary data. Don't forget to make status=complete above when you are done.
Resources edit
Provide links to presentations, blog posts, or other ways in which you disseminate your work.
References edit
Tripodi, F. (2023). Ms. Categorized: Gender, notability, and inequality on Wikipedia. New media & society, 25(7), 1687-1707.