Research:Understanding How Experts Use Talk Pages

Created

Contact

Jacqueline Yeung

Carnegie Mellon University

Collaborators

Jeff Rzeszotarski

Carnegie Mellon University

Duration: 2015-06 – ??

Research:Projects

This page documents a completed research project.

Wikipedia editors explore a huge range of data to make sense of a page before they contribute to it. They might read the page content itself, identifying areas for improvement. They could investigate discussions in Talk to understand how past editors view the page. They might read old edits to understand pitfalls in the history of the page. In this project we are developing new interfaces to help Wikipedia editors better make sense of this overwhelming amount of information. Once we've studied them, we hope to roll these interfaces out either as browser extensions or integrated into MediaWiki systems for general use.

We are actively looking for people to help contribute by having a 30-45 minute Skype conversation about some prototype systems we've designed: Please visit this calendar link to sign up!

Project Summary edit

Wikipedia editors explore a huge range of data to make sense of a page before they contribute to it. In this project we are developing new interfaces to help Wikipedia editors better make sense of this overwhelming amount of information. To help find the right direction, we are conducting interviews and user study sessions with Wikipedians to understand how they use the information available to them right now.'

We have already done some pilot sessions talking with editors while they try to make sense of a new article. We've learned that editors use a wide variety of tools and approaches to understand article history and Talk discussion. However, one common theme is the overwhelming tide of information present. Our ultimate goal is to help editors contribute better edits by empowering them with a more complete picture of the page. To do this, we're targeting the Talk page for our prototyping. We've built algorithms that link together talk threads, sections of the article, and past edits into one consistent model. This will let us do things like show relevant discussions for parts of the page, suggest edits to investigate that relate to what an editor is currently doing, and compress talk threads into summaries.

This summer, we are interviewing editors to understand their needs, and eventually will be testing some prototype interfaces with editors to see what works.

Methods edit

We are seeking out 5-10 "expert" editors who have made at least 1000 contributions to Wikipedia within the last 2 years. We believe these participants will have a good familiarity with the sorts of information made available to Wikipedians, and we hope that their expertise will guide us towards interesting information to visualize and explore. This open inquiry will be conducted through Skype/Hangout interview sessions lasting between 45 and 60 minutes. We have several different tasks to help "prime the pump" in the interview, such as looking at a page and trying to summarize some ongoing conflicts on the Talk page using the article, Talk, and page history. Interviews will be recorded with no linked identifying information. We will code interview results using a Grounded Theory approach. Ultimately, the themes and findings we make from the interviews will be representing collectively in a publication.

Moving forward, we hope to solicit another 10-20 "expert" editors to try out several prototype editing tools. These, as a part of a user study, will help us zero in on what works and what does not in our prototype development. These sessions will take approximately 45 minutes, and findings will be generated through measurements such as time to complete tasks as well as open coding participant responses.

Dissemination edit

Since we are conducting interviews with Wikipedians, we want to be careful to value participant's anonymity if possible. We will be aiming to publish the results of our discussions with editors at a major conference venue, but if that proves unsuccessful we will still release the preprint on sites such as Arxiv.

Ultimately, any prototype code we develop to link Talk and article pages together will be freely available on GitHub (or similar free OSS repository). As we continue to develop prototypes, we will add links to this page accordingly. We hope to roll out the prototypes to Wikipedians in the form of browser addons or components for integration into WikiMedia.

Wikimedia Policies, Ethics, and Human Subjects Protection edit

We are conducting interviews with Wikipedians subject to university IRB guidelines. We have an IRB on file locally, and may seek additional approval if we run any prototype studies. Because we are not reporting any interview results individually or with individually-identifying information, we hope that the impact on any participants will be minimal.

Benefits for the Wikimedia community edit

We want to release tools for the Wikimedia community that link together all sorts of page data into meaningful visualizations. We are iterating using Wikipedia data dump sources, so the schema we are using should naturally be compatible with existing Wikimedia projects. One goal for this work is to transition from individual prototypes into a rollout on a broad variety of Wikipedia pages with a larger number of participants. Regardless of our prototype development, the release of our interview findings should help Wikipedia administrators better understand what newbies and experts do when trying to understand a page, and might better inform the design of the information architecture that underlies articles.

Timeline edit

June - Pilot study with several expert Wikipedians
July - Plan and begin study with more expert Wikipedians
August - Develop and test prototype interfaces
September - Seek initial round of writing and publication

Background edit

Contacts edit

hcicmuresearch@gmail.com