Research:Asking anonymous editors to register/Study 2

Experimental CTAs

In this study, we will two versions of the pre-edit workflow to a control.

1. A control, in which an anonymous (IP) editor see no differences -- represents the current user experience where no calls to action appear before a user registers.
2. A pre-edit v1, Call to action which is triggered by an anonymous editor clicking any of the edit buttons on an article (same as "pre-edit" from study 1)
3. A pre-edit v2, Call to action which is triggered by an anonymous editor clicking any of the edit buttons on an article that emphasizes the optional nature of registration and renders "continue editing" as a button.

• 
  English Wikipedia, pre-edit v1
• 
  English Wikipedia, pre-edit v2

Hypotheses

Rationale: We suspect that the primary reason for reduced productivity in the pre-edit condition from study 1 is that some anonymous editors will refuse to register an account and the CTA did not make it clear that registering an account was optional. We specifically designed pre-edit v2 to make this optional nature clear. We suspect that by making this clear, we can recover those productive edits that were lost due to the confusion.

Hypothesis 1: Anonymous editors in the pre-edit v2 condition will be more productive than anonymous editors in the pre-edit v1 condition.

Rationale: If anonymous editors edit with some regularity, then the pool of anonymous editors may be drained through registration CTAs. If this is true, we should see lower conversion rates for a follow-up experiment.

Hypothesis 2: A smaller proportion of anonymous editors will register in pre-edit v1 than in pre-edit from study 1.

Experimental design

A change was deployed to English Wikipedia that enabled bucketing anonymous users into the experimental buckets at approximately 2014-07-10 00:00:00 UTC. Experimental CTAs were shown to users for approximately 7 days -- until 2014-07-17 00:00:00 UTC.

Timeline. The timeline of experimentation and observation is presented.

Results

Hyp 1 recovered productivity

Article edit rate. The geometric mean number of article revisions saved is plotted by experimental condition.

Productive edit rate. The geometric mean number of productive edits is plotted by experimental condition.

Productive editor proportion. The proportion of productive editors is plotted by experimental condition.

Hyp 2 reduced registration rate

Registration rates (per eligible anon). The proportion of anonymous users who both clicked edit and registered an account are plotted.

English Wikipedia
baseline %	1.367
baseline n (estimate)	4754.273
v1 %	1.871 (+0.504)
v2 %	1.59 (+0.223)
v1 factor %	136.9
v1 n estimate	6508.355
v2 factor %	116.4
v2 n estimate	5531.674

External links

• Analysis code repository: https://github.com/halfak/Anonymous-phenomena