Understand conversation and coordination spaces in English Wikipedia, with an aim towards understanding the pain points and prototyping semantic enhancements. This is part of my Ph.D. work in DERI's Social Software Unit; we approach Computer Science with Semantic Web technologies. The goal of my Ph.D. is to provide support for conversation and argumentation in the social web using semantic web technology.
So far this has mainly involved study of article Talk pages and Articles for Deletion. My work aims to support decision-making and identify opportunities for socializing new community members.
We have studied the text of AfD discussions, and last year we conducted a round of interviews with experienced Wikipedians and adminstrators. Our overarching goal is to understand:
What are the outcomes of deletion discussions, both for articles and for the community?
What are the purposes and benefits of the current process? What are the "pain points"? What changes, improvements, or support tools could be suggested?
We have read, analyzed, and characterized discussions in two ways, based on decision factors and argument patterns. This helps us to understand research questions such as
What factors contribute to the decision about whether to delete a given article?
When multiple factors are given, what is the relative importance of those factors?
What argument patterns are used in AfD discussions?
Does experience with AfD change the argument patterns?
This helps us understand the process of arguing. Now we would like to better understand the outcomes of the process for the community.
We are particularly interested in understanding how people discover AfD, what their first experience of the process is like, and what they learn about through participating. We are also interested in learning about the problems experienced AfD participants see in the process, and how they determine whether an argument for keeping or deleting an article is a good one.
Other researchers have already noticed friction for newcomers to AfD: Geiger & Ford noticed that AfD participants are "overwhelmingly regulars to the process", and that it is relatively rare (18%) for "the creators of nominated articles [to participate] in that article's deletion discussion." (Participation in Wikipedia's Article Deletion Processes", Wikisym 2011). What we don't know is whether and to what extent this is a problem for the community.
We hope through interviews to find answers to questions like:
What is the experience of participating in AfD like?
What are the benefits to participating in AfD, for an individual? For the community?
What are the costs and challenges of participating in AfD, for an individual? For the community? What are the problems and "pain points" with the current AfD process?
What are the community norms for arguments? Would scaffolding arguments help new and infrequent participants?
This will help us understand the impact and outcomes of discussions, rather than just the discussion style and process.
Previously, through using community spaces, I have recruited experienced Wikipedians and administrators for interviews about AfD. I am currently planning follow-up interviews with previous interviewees who gave me permission to contact them as my research progressed, to check my findings and to ask follow-up questions.
I now need to interview Wikipedians who are occasional or new users of Articles for Deletion, to evaluate my findings and get their perspectives.
In terms of the sample, I propose to choose subjects from data that User:Staeiou collected for a WikiSym 2011 poster. (“G1: Patent nonsense”: Participation and Outcomes in Wikipedia’s Article Deletion Processes. R. Stuart Geiger and Heather Ford.).
I would like to interview 10 novices, so I would propose directly targeting subjects by email (I'm considering Talk page requests but this compromises anonymity.). Since I expect a low response rate (are there any figures on what's expected?), I would plan to randomly select 40 people to email, and then choose additional people as needed. (I would welcome suggestions for the content of the message as well; I would expect a brief description of the study, a link to this page, and a request to spend 30 min - 1 hour interview by skype, phone, or IRC.)
Jodi Schneider & Alexandre Passant, “Arguments about Deletion: Guiding New Users in Making Good Arguments.” In WebSci2011, ACM Web Science Conference. Koblenz, Germany, June 15-17, 2011. abstract (PDF), poster (PDF), poster (Slideshare)
"Deletion Discussions in Wikipedia: Decision Factors and Outcomes"[1] found among other things that "69.5% of discussions and 91% of comments are well-represented by just four factors: Notability, Sources, Maintenance and Bias. The best way to avoid deletion is for readers to understand these criteria." One of the authors also co-presented a demo showing mock-ups of possible "alternative interfaces for deletion discussions in Wikipedia"[2], which would highlight the prevalence of each type of argument (e.g. notability, sourcing...) in a deletion discussion more clearly.
Winning and losing argument patterns in deletion debates: A paper subtitled "How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups"[3] (presented at the CSCW'13 conference earlier this year) applied the argumentation theory of Doug Walton to classify comments in a corpus of deletion debates on 72 Wikipedia articles (all AfD that were initialized or relisted on January 29, 2011). The four Ireland-based authors emphasize that compared to previous related research which used simpler methods to classify deletion arguments, e.g. based on keywords or policy areas such as notability, their manual analysis is much more thorough and fine-grained, coding AfD comments into 17 categories based on Walton's classification. Among these, the "Rules" and "Evidence" categories are the most popular, making up 36% of AfD arguments. The papers's two other main results are that "familiarity with community norms correlates with [newbies'] ability to craft persuasive arguments" and that "acceptable arguments use community-appropriate rhetoric that demonstrate knowledge of policies and community values while problematic arguments are based on personal preference and inappropriate analogy to other cases" (drawing a direct comparison between Walton's list of problematic arguments and Wikipedia's list of deprecated AfD arguments, e.g. Walton's "Argument from Analogy" corresponds to WP:OTHER - "Other stuff exists").
Shortcomings in the design of Wikipedia talk pages. A diagram of deletion processes in the English Wikipedia, from Schneider et al., 2011 [4] Researchers from the National University of Ireland, Galway presented some work in progress from a project aimed at understanding Wikipedia coordination spaces and costs. In a paper presented earlier this year at SAC '11 the authors discuss the results of a small series of semi-structured user interviews with Wikipedia administrators and editors.[5] The results point at a number of drawbacks in the design of Wikipedia talk pages, suggesting that editors find it hard to keep up-to-date with temporally sparse discussions that are often scattered across multiple pages. The interviews also suggest that talk pages often become the target of support requests by new editors that go unnoticed. The lack of connection between discussions and the article itself (e.g. links between threads and specific sections or topics of the article) also emerges as one of the main weaknesses of Wikipedia talk pages. In the remainder of the paper the authors introduce a lightweight solution to allow effective categorization of comments posted on article talk pages by semantically enriching them with an RDF markup. This markup can then be exposed to end users with the aid of a JavaScript bookmarklet, manipulated and exported via SPARQL, and potentially used to generate granular notifications. In a poster presented last month at WebSci '11, the same team of researchers gives an overview of work in progress on AfD discussions and illustrates with a diagram the complexity of deletion discussions and procedures in the English Wikipedia. [4]
↑Schneider, J., Passant, A. and Decker, S. (2012) Deletion Discussions in Wikipedia: Decision Factors and Outcomes, WikiSym '12PDF
↑Jodi Schneider, Krystian Samp: Alternative Interfaces for Deletion Discussions in Wikipedia: Some Proposals Using Decision Factors. Demo, WikiSym'12, August 27–29, 2012, Linz, Austria. ACM 978-1-4503-1605-7/12/08. PDF
↑Jodi Schneider, Krystian Samp, Alexandre Passant, Stefan Decker: Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups CSCW’13, February 23–27, 2013, San Antonio, Texas, USA PDF
↑ abSchneider, Jodi, and Alexandre Passant (2011), Arguments about Deletion: Guiding New Users in Making Good Arguments. In Proceedings of the 2011 ACM Web Science Conference (WebSci '11). PDF
↑Schneider, Jodi, Alexandre Passant, and John G. Breslin (2011). Understanding and improving Wikipedia article discussion spaces. In Proceedings of the 2011 ACM Symposium on Applied Computing (SAC '11), 808. New York, NY: ACM Press. DOI • PDF
.jpg)
