Research:Teahouse/Experiments

This page documents some of the approaches we're experimenting with during the course of the Teahouse pilot. We'll use it to record outcomes, lessons learned, etc., in hopes that this information will be useful for other projects as well.

Finding new editors

DB Queries

Methodology

There are lots of ways to find new editors on English Wikipedia. However, a lot of new users who register for an account never edit at all, make one edit and never return, edit only for self-promotion, or edit purely with a goal of vandalizing the encyclopedia. Past projects have shown that outreach to these editors tends to yield little or no results, and we don't want to waste a lot of host effort on vandals, spammers, etc. In this experiment, we wanted to learn which kinds of database queries could yield the greatest percentage of promising new editors to invite to the Teahouse, while filtering out most of the "noise" of editors who are highly unlikely to be retained or who the community does not want to retain (vandals, COI editors who are here purely to self promote, etc).

We ran a series of queries to pull sets of new editors, reviewed their contribution histories, and coded each editor for whether or not they would be a good candidate to invite to the Teahouse. Below are the results of our initial queries. During the course of the pilot, we'll continue to revise and experiment with new queries based on host feedback, invitation results and Teahouse activity.

Results

Query	Records returned	Sample coded	Invitees found	Notes
All new editors on 12/21/2011 who had made more than 10 edits by 12/22/2011 (roughly, within their first day)	90	64	38	this appears to be the most useful query so far, lots of people highly active on their first day who would be good candidates for immediate outreach. we're thinking of launching with this invites to editors found with this type of query, though we would still want to have humans filter the sample before sending invites.
All newly autoconfirmed editors on 12/26/2011 (autoconfirmed = more than 10 edits AND more than 4 days since registration)	352	32	15	sample yielded a lot of editors who registered a while ago, then came back on 12/26 for single purpose, racked up 10 not great edits in one article, and never came back, asked questions, or attempted to interact with the community. we think most of these editors would be unlikely to respond to a Teahouse invitation.

Inviting new editors

(to be filled in after launch, when invites begin)