Research talk:Autoconfirmed article creation trial/Work log/2017-08-02

Wednesday, August 2, 2017

Today I'll continue working on the instrumentation and operationalization of our hypotheses.

H7: The average number of edits in the first 30 days since registering is reduced.

Measuring this is similar to H2, a combination of the revision and archive tables.

H8: The number of requests for "confirmed" status is increased.

The archive of requests for permissions starts at Wikipedia:Requests for permissions/Archive, where there are links to sub-pages for denied and approved requests per month. Each monthly archive appears to have a section header for each day and an unordered lists for each request. The request is a combination of a template ({{Usercheck-short|Username}}, a link to the page where the request originated (e.g. Wikipedia:Requests for permissions/Confirmed, and a web link to the revision where request was responded to. Not all archives appear to have this format, it seems that it was introduced in August 2009.

H9: The workload of New Page Patrollers is reduced.

The workload of NPPers is mainly affected by the influx of new pages into the page patrol queue. New page creations are now logged through the EventBus system (ref T150369, and we can therefore use that to measure the influx.

H10: The size of the backlog of articles in the New Page Patrol queue will decrease faster than expected.

Skipping this for now, will return to it.

H11: The survival rate of newly created articles by autoconfirmed users will remain stable.

We adapt the definition of article "survival" from Schneider et al, meaning that an article has survived for a "reasonable amount of time" if it is not deleted during the first 30 days of its life. We can combine this definition with a dataset of users reaching autoconfirmed status in order to determine the historic survival rate.

H12: The rate of article growth will be reduced.

The number of articles in the English Wikipedia has been used as a measure of success for many years, meaning it has also received a lot of attention. Some of this has come in form of research, e.g. Suh et al, but it has also been tracked by WMF in their statistics summary for the English Wikipedia. Definitions of what counts as an article can differ, but commonly it refers to all pages in the main namespace that are not redirects, and that include at least one link to another page. Secondly, we must restrict this to measure articles that have survived at least some number of days, and propose to again use 30 days as a "reasonable amount of time."