Research talk:Anonymous editor acquisition/Signup CTA experiment/Work log/2014-06-16

Monday, June 16th

Latest comment: 10 years ago3 comments1 person in discussion

Picking up where I left off on Friday, I should have a dataset with a complete set of reverted & archived flags.

--Halfak (WMF) (talk) 14:40, 16 June 2014 (UTC)Reply

Article revisions. The geometric mean number of article revisions saved is plotted by experimental condition.

Productive revisions. The geometric mean number of productive edits is plotted by experimental condition.

Article revisions. The geometric mean number of productive editors is plotted by experimental condition

It looks like -- no matter how I measure it -- overall productivity went down in the pre-edit condition. --Halfak (WMF) (talk) 14:48, 16 June 2014 (UTC)Reply

Now it's time to get some basic stats on burden. Since I just measured the revert rate, I can figure out how much burden relevant anons inflicted on Wikipedians.

Reverted article revisions. The geometric mean reverted article revisions per user/token is plotted by experimental condition. Note that this method for measuring revert activity does not control for decreased article activity.

Reverted revisions prop. The proportion of overall reverted article revisions is plotted by experimental condition. Note that this method for measuring revert proportion over-weights highly active individuals.

Reverted sampled revision prop. The proportion of reverted sampled article revisions is plotted by experimental condition. Note that this method for measuring revert activity controls for both prolific users and decreased article activity.

OK, so I measured this two naive ways and a robust way. #Reverted article revisions plots the expected number of reverted article revisions per user/token. This measurement is affected by the decreased rate of article editing. #Reverted revisions prop plots the proportion of reverted article revisions for the whole group. This measurement is affected most strongly by prolific editors and is therefor, not robust to outliers. For #Reverted sampled revision prop, I randomly sampled 1 article revision from each user/token who made at least one and included that in the proportion measurement. This method controls for both concerns.

So, it looks like editors in the pre-edit condition are consistently less burden than the control.

Here's the data:

wiki	bucket	n	main_revisions.sum	main_revisions.geo_mean	main_revisions.geo_se.upper	main_revisions.geo_se.lower	productive_edits.geo_mean	productive_edits.geo_se.upper	productive_edits.geo_se.lower	reverted_main_revisions.sum	reverted_main_revisions.geo_mean	reverted_main_revisions.geo_se.upper	reverted_main_revisions.geo_se.lower	productive.k	main_editor.k	sample_reverted.k	productive.prop	productive.se	reverted_main_revision.prop	reverted_main_revision.se	sample_reverted.prop	sample_reverted.se
dewiki	control	42441	12314	0.08973791	0.09150446	0.08797422	0.06175928	0.06329375	0.06022702	2123	0.02960919	0.03042239	0.02879663	2383	3636	1329	0.05614854	0.00111745	0.1724054	0.003403962	0.3655116	0.007986389
dewiki	post-edit	38593	8075	0.08627629	0.08798654	0.08456873	0.05689025	0.05833854	0.05544394	1942	0.0303633	0.03121313	0.02951417	2127	3356	1309	0.05511362	0.001161622	0.2404954	0.00475606	0.3900477	0.008419684
dewiki	pre-edit	39331	10166	0.07274083	0.07439115	0.07109304	0.05510174	0.05657932	0.05362622	1319	0.0192839	0.01997635	0.01859193	2014	2722	776	0.05120643	0.001111426	0.1297462	0.003332693	0.2850845	0.008653068
enwiki	pre-edit	198936	61737	0.107385	0.1083025	0.1064682	0.08511951	0.08593776	0.08430188	10209	0.02552958	0.02591357	0.02514574	14982	18763	4380	0.07531065	0.0005916562	0.1653627	0.001495185	0.2334381	0.003088221
enwiki	control	208205	77814	0.1321544	0.1331415	0.1311681	0.09897115	0.09983017	0.09811281	14828	0.03657449	0.03701676	0.03613241	18566	24619	6873	0.08917173	0.0006245774	0.190557	0.001407916	0.2791746	0.002859024
enwiki	post-edit	189530	65349	0.1311697	0.1321865	0.1301538	0.09805692	0.09894065	0.09717389	13201	0.0361366	0.03659432	0.03567909	16891	22510	6338	0.08912046	0.0006544556	0.2020077	0.001570594	0.2815637	0.002997745
frwiki	pre-edit	30115	9673	0.1023034	0.1046157	0.0999959	0.08316826	0.08527892	0.08106169	1191	0.02137573	0.02223767	0.02051451	2228	2764	605	0.07398306	0.001508288	0.1231262	0.003340898	0.2188857	0.007864968
frwiki	post-edit	28649	9874	0.1238606	0.126376	0.1213509	0.09654348	0.09880157	0.09429004	1435	0.02865029	0.02963839	0.02766313	2568	3324	833	0.08963664	0.001687701	0.1453312	0.00354676	0.2506017	0.007516538
frwiki	control	31714	10589	0.1230304	0.1254209	0.120645	0.09200022	0.09410328	0.0899012	1894	0.03245121	0.03347216	0.03143127	2691	3631	1034	0.08485212	0.001564773	0.1788649	0.00372428	0.28477	0.00748957
itwiki	pre-edit	20310	8328	0.117881	0.1210721	0.1146989	0.1004393	0.1034149	0.09747166	800	0.02071121	0.02176015	0.01966334	1669	1984	368	0.08217627	0.001927072	0.09606148	0.003229042	0.1854839	0.008726339
itwiki	control	20297	9856	0.1431397	0.146633	0.139657	0.1156532	0.1188154	0.1124999	1341	0.03125235	0.03259601	0.02991044	1982	2462	549	0.0976499	0.002083567	0.1360593	0.00345347	0.2229894	0.008389027
itwiki	post-edit	18359	9604	0.1442438	0.1479507	0.1405488	0.118409	0.1218074	0.1150208	1047	0.02996144	0.03128555	0.02863902	1808	2237	490	0.09848031	0.002199064	0.1090171	0.003180211	0.2190434	0.008744712

--Halfak (WMF) (talk) 15:43, 16 June 2014 (UTC)Reply

Add topic