Research:Newsletter/2015/December

Vol: 5 • Issue: 12 • December 2015 [contribute] [archives]

Teaching Wikipedia; Does advertising the gender gap help or hurt Wikipedia?

With contributions by: Piotr Konieczny, Tilman Bayer and Max Klein

Does advertising the gender gap help or hurt Wikipedia?

Reviewed by Tilman Bayer

A working paper in economics provides several novel results shedding light on Wikipedia's much discussed gender gap, focusing on three aspects: The causes of the gender gap in contributors, its impact on Wikipedia's content, and how outreach measures that highlight the gender gap influence participation on Wikipedia.

It uses several sources of data, including the edit histories of all registered English Wikipedia users who have stated their gender in the user preferences, a survey and experiment with 1000 Amazon Mechanical Turk users (from the US only, who were paid $1.50 for a 20 minutes task), and a dataset of biographical articles with the subject's gender obtained from Wikidata (excluding "celebrities like actors, athletes, and pop stars", focusing on "professionals", e.g. politicians and scientists, and cultural figures like writers and composers), together with pageview data.

Regarding causes of the gender gap, the author provides an overview of existing research, for example dismissing the so-called second shift as an explanation ("There are no gender differences in the amount of free time", p. 3) and pointing out that "women contribute no less than men to another example of online public good provision, writing user reviews for products and services".

From the survey, the author concludes that "almost half of the gender gap in Wikipedia writing is explained by gender differences in two characteristics: frequency of Wikipedia use and belief about one’s competence ... The gender difference in the belief about competence could be due to women being less competent or due to women underestimating their competence. The survey data does not allow to distinguish these." (While the paper is otherwise well-informed about pre-existing research, it would have benefited from connecting this result to the work of Shaw and Hargittai; see our review of their paper "Mind the skills gap: the role of Internet know-how and gender in differentiated contributions to Wikipedia").

Moving on to the effect of the editor gender gap on Wikipedia's content, the paper finds "that women are about twice as likely as men to contribute to Wikipedia articles about women", based both on the edit histories dataset and the Mechanical Turk survey. Intriguingly, "the number of readers per editor is higher for articles about women, and the share of articles that no one reads is larger in the case of articles about men". In other words, readers prefer articles about women, editors prefer articles about men. The author indicates that the readership discrepancy mostly comes from the tail end of low-traffic biography articles:

"On a typical (median) day in September 2014, no one read 26 percent of the biographies of men versus only 16 percent of the biographies of women."

The third part consisted of an experiment designed to "test whether providing information about gender inequality in Wikipedia changes editing behavior". Mechanical Turk respondents were divided into two groups that were provided with different introductory information about Wikipedia:

"Wikipedia has been criticized by some academics and journalists for having only 9% to 13% female contributors and for having fewer and less extensive articles about women or topics important to women." (a quote from the article Gender Bias on Wikipedia)

vs.

"Wikipedia started in 2001. English-language Wikipedia has over 4.5 million articles."

They were then "asked to imagine a hypothetical situation in which they edit a person’s Wikipedia page. Respondents were asked to look at Wikipedia articles and find some relevant information from the web that is missing from a Wikipedia article. ... In the end, they were also asked how likely they are to edit Wikipedia in the future."

The first version, highlighting the criticism of Wikipedia's gender gap, is "associated with a 35 percent decrease in the likelihood of editing Wikipedia in the future", i.e. discouraged rather than encouraged respondents from contributing, which the author calls "somewhat unexpected". This negative effect is concentrated among men: "The information that the majority of Wikipedia editors are men, leads men to reduce their editing effort, but it does not change the behavior of women." As summarized by the author:

"The result provides an example where encouraging gender equality can partially backfire. Wikipedia has set a goal to increase the share of female editors. One way to achieve this is by discouraging male editors. However, this might not be desirable ... The implication for Wikipedia and other forms of media is that it is important to balance the efforts of attracting new contributors and keeping the current ones."

She also points out that "there are other examples in the literature where informational treatment has backfired".

The paper is highly innovative and adds several novel results (with direct relevance for Wikipedians' work to combat this kind of systemic bias), some of which are not mentioned in this summary. The author seems justified in calling it "the first comprehensive study of gender inequality in a new media environment such as Wikipedia". A weakness of the part of the paper that studies the effect of editors' gender on their contributions might be its partial reliance on the gender as stated in their accounts' user preferences. The author stresses that her methodology is robust against potential under-reporting by one gender (for example, female editors being less willing to publish their gender in this way because of concerns about harassment). However, she adds that the validity of the results rests on the assumption "that editors don’t systematically report wrong gender. Since the default option is not specifying one’s gender, I would not expect that they are massively reporting wrong gender." In contrast, a 2011 paper by other authors ("WP:CLUBHOUSE", see Signpost summary) that used the same methodology (and concluded that e.g. women vandalize Wikipedia more often than men) explicitly pointed to the possibility that their results might be affected by deliberately wrong reporting (although this might mostly concern vandals with few edits overall, i.e. less relevance to the questions studied here). The paper also falls victim to a survivor bias fallacy when interpreting an otherwise interesting result as "female editors [having] increased from 3.7 percent in 2002 to a peak of 11.5 percent in 2011. In 2013, 10.4 percent of the active editors were female." The option to state a gender in one's user preferences was only introduced in 2009, so it is possible that, for example, there was a much higher percentage of women editing Wikipedia in 2002 who however left before they had the opportunity to state their gender seven years later.

Teaching Wikipedia: The Pedagogy and Politics of an Open Access Writing Community

Reviewed by Piotr Konieczny

This dissertation looks at the opportunities for writing pedagogy offered by the Wikipedia:Education program. It provides an interesting, though not comprehensive, overview of the literature in the field, and then proceeds to describe and analyze a number of educational assignments that the author has carried out on Wikipedia through their 2011 course. The author concludes that the "teaching with Wikipedia" approach is generally beneficial to students in a number of ways, from improving their writing and research skills, to an increase in student's rhetorical skills, and understanding of topics relating to knowledge creation. The main limitations of the study, acknowledged by the author, is that it is based on a small sample of students (the course seems to have only about seventeen participants). Nonetheless, it is a useful addition to our still limited understanding of the practice and benefits of the use of Wikipedia in an educational setting.

"Wikipedia, sociology, and the promise and pitfalls of Big Data"

Reviewed by Piotr Konieczny

This paper, or perhaps an essay or an Onion piece (2,500 words, with little original research), entitled "Wikipedia, sociology, and the promise and pitfalls of Big Data", is a strange beast. Published in the journal Big Data & Society, it doesn't really address the topic of big data; instead presenting a sociologically-informed and critical discussion of a number of aspects of Wikipedia that, while interesting, seems out of place in an academic journal, and reads more like an academic blog entry. The authors display a reasonable familiarity with Wikipedia, though they make a few factual mistakes (such as suggesting that Wikipedia:WikiProject Sociology was formed with the assistance of the American Sociological Association in 2004; in fact ASA has not been aware of WP:SOCIO until late 2000s and its support for it has been limited to linking to the WikiProject from their Wikipedia Initiative Page).

Based on their literature review, the authors don't hesitate to make some strong claims about Wikipedia, primarily in the vein of Wikipedia becoming less friendly to new editors, though most of those claims are more or less supported by the sources cited. The authors' research question is how the discipline of sociology is framed on Wikipedia, with special attention to the concepts of notability of academics (WP:PROF) and the gender imbalance of the Wikipedia biographies of sociologists. Unfortunately, as this is not a proper research piece, the authors' findings are rather sparse, and primarily concern the fact that topics covered by the WikiProject Sociology and its related portal are poorly structured, that Wikipedia's biographies of sociologists are mostly about male subjects (the article omits, however, the question of gender bias in academia - aren't most sociologists male anyway...? ), and that WP:PROF guideline may not be enforced too strictly for sociological biographies. It was an enjoyable reading, but overall, as seen in the article's sections which are entitled Abstract, Declaration of conflicting interests, Funding and Notes, there is something important missing - the article proper. As the authors make a point of stressing (twice) the chaotic and unorganized nature of Wikipedia's coverage of sociological topics, I can't help but feel that the article, which also fails to drive home any particular and well organized point, could well fit that description too.

See also our earlier coverage of the authors' research project: "Gender imbalance in Wikipedia coverage of academics to be studied with 2-year NSF grant"

Briefly

Wikipedia and the Stock Market

Reviewed by Max Klein

Wikipedia may effect the stock market in a "governing" way, says Crowd Governance: The Monitoring Role of Wikipedia in the Financial Market. It looks at how the stock market and insider trading reacts to the creation of a Wikipedia article about a traded firm. Using a sample of 413 articles on S&P500 firms, it was found that stock prices significantly drop on the days their Wikipedia article is created. Furthermore prices drop further for companies that have more insider traders, or which are more institutionally owned. This goes to show, the authors say, that Wikipedia governs the stock market by "reducing information asymmetry". Firm information on Wikipedia would seem to benefit the public more than information in newspapers, that is bad news for Wall Street.

Other recent publications

A list of other recent publications that could not be covered in time for this issue – contributions are always welcome for reviewing or summarizing newly published research.

• "Understanding the ‘Quality Motion’ of Wikipedia Articles Through Semantic Convergence Analysis"' From the abstract: "This study aims to check if Wikipedia’s [quality] ratings really reflect its stated criteria. According to Wikipedia criteria, having abundant and stable content is the key to article’s quality promotion; we therefore examine the content change in terms of quantity change and content stability by showing the semantic convergence. We found out that the quantity of content change is significant in the promoted articles, which complies with Wikipedia’s stated criteria."
• "Wikipedia's Politics of Exclusion: Gender, Epistemology, and Feminist Rhetorical (In)action" From the abstract: "In this article, I explore how Wikipedia functions as a rhetorical discourse community whose conventions exclude and silence feminist ways of knowing and writing. Drawing on textual analysis of Wikipedia's editorial policies, as well as interviews with female users, I argue that Wikipedia's insistence on separating embodied subjectivity from the production of knowledge limits the site's ability to facilitate any substantial, subversive feminist rhetorical action."
• "Knowledge Quality of Collaborative Editing in Wikipedia: an Integrative Perspective of Social Capital and Team Conflict" From the abstract: "Despite the abundant researches on Wikipedia, to the best of our knowledge, no one has considered the integration of social capital and conflict. Besides, extant literatures on knowledge quality just pay attention to task conflict, while relational conflict is rarely mentioned. Meanwhile, our study proposes the nonlinear relationship between task conflict and knowledge quality instead of linear relationships in prior studies. We also postulate the moderating effect of task complexity."
• "Collective remembering of organizations: Co-construction of organizational pasts in Wikipedia" From the abstract: "The authors analyze 1,459 edits of Wikipedia pages of ten organizations from various industries. Quantitative content analysis detects Wikipedia edits for their reputational relevance and reference to formal sources, such as corporate communication or newspapers. Furthermore, the authors investigate to which degree current corporate communication in form of 177 press releases has an influence on the remembering process in Wikipedia. ... The analysis of press releases shows that current frames provided by corporate communication finds only little resonance in the ongoing remembering processes in Wikipedia."

References

Wikimedia Research Newsletter
Vol: 5 • Issue: 12 • December 2015
About • Subscribe: Email • [archives] • [Signpost edition] • [contribute] • [research index]