Research:Onboarding new Wikipedians/OB2

Overview

Screenshot from just before the test period

This 1-week iteration started on 2013-01-25T00:00:00Z and ended on 2013-01-31T22:45:00Z. We randomly split new registered users on the English Wikipedia into a control group (OB2a) and a test group (OB2b). Users in the control group saw Wikipedia's default welcome screen. Users in the test group were served tasks via GettingStarted. We logged the editing activity of users entering one of the two following subfunnels from the GettingStarted landing page for the test group:

returnto

users clicking on the "Return To" button and attempting to edit the page they came from when they registered an account

gettingstarted

users clicking on one or more links in the task section of the GettingStarted page

Cohort analysis

We defined the following cohorts in order to measure their respective productivity:

OB2 cohorts

ID	Unique users	Description
e3_ob2a	14,595	Users in the control group
e3_ob2b	14,488	Users in the test group being served tasks via GettingStarted

Live account rate

Proportion of new registered users in the OB2 test clicking the edit button at least once in the main namespace within 24 hours of registration.

We measured the proportion of "live accounts" or users in each cohort clicking at least once on the edit button on an article in the main namespace within 24 hours of registration (measurement taken at 2013-02-01T00:00:00Z).

OB2 ns0 24h live account rate

ID	Unique users	Editing users	%
e3_ob2a	14,595	3,938	27.0%
e3_ob2b	14,488	4,533	31.3%

We compared the proportion of live accounts in the e3_ob2a and e3_ob2b cohorts and found a significant 4.30% increase in the test group compared to control (a chi-square test rejects the null hypothesis that the true proportion of live accounts is the same in both groups [X² = 65.1, N = 29,083, p < .001]).

1 edit in 24 hours

Proportion of 1+ 24h main-namespace editors among new registered users in the OB2 test.

We measured the proportion of users in each cohort completing their first main-namespace edit within 24 hours of registration (measurement taken at 2013-02-01T00:00:00Z).

OB2 1+ ns0-edit 24h threshold

ID	Unique users	Editing users	%
e3_ob2a	14,595	2,921	20.0%
e3_ob2b	14,488	3,168	21.9%

We compared the proportion of threshold-hitting users in the e3_ob2a and e3_ob2b cohorts and found a significant 1.85% difference between the test and the control group (a chi-square test rejects the null hypothesis that the true proportion of threshold-hitting accounts is the same in both groups [X² = 14.96, N = 29,083, p < .001]).

5+ edits in 24 hours / 1 week

We also measured the proportion of users in each cohort completing their first 5 main-namespace edits within 24 hours of registration and found no significant difference between the groups (2.9% for control and 3.0% for test). The difference remains non significant when we compare the proportion of users editing 5+ times the main namespace within 1 week of registration (4.0% for control and 4.1% for test).

In the first 24 hours, roughly 14% of those who completed one NS0 edit from the control completed five. In the test group, this proportion was roughly the same, at 13%, though we have not performed a statistical test to compare these two.

OB2 5+ edits in NS0 threshold

	control	test
in 24 hours	424	438
in 1 week	582	599
total cohorts	14,595	14,488