Research:Characterizing Wikipedia Reader Behaviour/S1-English

Survey 1, English Wikipedia

Design

When

The survey ran from 2015-10-12 at 19:00 UTC to 2015-10-16.

Project

English Wikipedia

Platform

Desktop (though we learned later that due to a bug, the survey was shown to both Mobile and Desktop users.)

Sample

a survey widget was displayed to 5 out of 100K page requests to English Wikipedia (Desktop). The survey is prompted to the user on their first pageview and will be shown to the user until the user discards the survey or responds to it. If the user agrees to participate in the survey, he/she is taken to an external form where a single question is asked. We expected to need to collect a total of 5000 responses.

Question

The question the user was asked was "Why are you reading this article today?". User responses were collected in the form of short free form text entries, the referring article was not included in the response.

Date collection and privacy policy

Data collection occured via Google Forms. The survey widget linked to a privacy policy designed for this survey.

Phabricator task

Analysis

The handcoding of the responses to the first survey was done in 3 stages. In the first stage, all researchers worked on 20 free form entries together to build alignment and a common understanding about how tagging should be done. We agreed that we freely assign tags to each free form text response in the next round, and review our tags to build common tags or trends after that. In the second stage, each researcher randomly selected 100 free form responses to tag. This stage resulted in 500 tagged free form responses. At the end of this stage we reviewed all 500 tagged responses and identified 4 main trends in the responses along with some tags corresponding to each trend. The trends were motivator, information need, current/immediate real time context, and source. each researcher allocated tags to the free form response. In the third stage, each researcher randomly chose 100 responses and assessed whether the response contains information about each of the top 4 main trends identified in stage 2. At the end of this stage we collected 500 tagged responses and reviewed and discussed them in detail. We agreed that the 4 categories identified are the best categories we could identify given the information provided by the users in the survey, but we needed to test how good these tags/categories were in practice. Therefore, we designed an additional survey that we describe below.