Research talk:Wikipedia Mobile Readers Survey 2011

Please provide feedback.

Terminate language

I can understand why in face to face or telephone surveys you reject some people with words such as “Thank you for taking the time to participate in our survey today. We truly appreciate that you were interested in contributing. Unfortunately, the parameters of this study don’t match your background. We hope that you will participate in future studies that are a closer match to your background.” Employing people to ask questions is expensive and if you already have enough "white male childless graduates in their forties" then it makes sense to stop interviews as soon as you've identified that you are dealing with someone you already have your quota of. But an online survey doesn't need to make such an economy as we don't save money by rejecting people. The data would be just as valid if you said we have three times as many "white male childless graduates in their forties" as we need, so we will randomly drop two thirds of them from the full dataset when we create the dataset we use to calculate statistics. Better still use all the data and weight by cell, so if we have surplus "white male childless graduates in their forties" then each such answer would only count as one third of an answer, but if we were short of "white female childless graduates in their sixties" we could weight them at more than 1. WereSpielChequers 13:06, 24 July 2011 (UTC)

Since this is the mobile survey, we are looking primarily at the experiences of those who use Wikipedia on their mobile. People who don't read Wikipedia on their phone can't answer 90 percent of the questions from this survey. I don't see much benefit in including them in the study, and another issue is that at the back-end if we open the survey to everyone we would have to do incredible amount of data cleaning and dissecting. Do you have any suggestions on termination language? Would truly appreciate them. Manipande 18:56, 25 July 2011 (UTC)

Matrix

I'm curious how you decide how to weight the survey without already having a census of the whole population you are surveying. I can appreciate that if 6% of the mobile accesses of Wikipedia were from Atlantis you would want 6% of your survey sample to be from that country. But is there anything else that we know about our readers and can use to weight such a survey? WereSpielChequers 13:06, 24 July 2011 (UTC)

We don't have any useful data that would help us with weighting of the survey. We are thinking of this survey as an exploratory survey to understand some key geographies. I hope this helps. Manipande 17:35, 26 July 2011 (UTC)

Adolescents

I suspect I know why you don't want responses from under 14s. But is there actually a reason not to collect data from them, providing of course we don't collect contact details or other identifying info, and we skip irrelevant questions such as degrees and marital status. WereSpielChequers 13:06, 24 July 2011 (UTC)

For legal reasons, we can't survey people who are younger than 14. Manipande 17:33, 26 July 2011 (UTC)

Are you sure about that? We are allowed to let them edit. I could understand if we were only allowed to collect anonymous info on them. WereSpielChequers 11:24, 4 August 2011 (UTC)

Objectives section

I'm not sure if you consider displaying the goals before the survey starts. If so, I would seriously think about shrinking this section to just 2-3 lines per goal. Otherwise, mobile screens will look cluttered (even in smartphones, perhaps not so much in tablets) and it could be really discouraging for participants to enter the questionnaire. --GlimmerPhoenix 13:17, 26 July 2011 (UTC)

We are not planning for displaying goals before the survey. It would be too much of information for the respondents and would turn many of them away. Manipande 16:40, 26 July 2011 (UTC)

Tablets

This is a "Mobile Readers Survey", but reading the questions it's clear that you focus on phones, entirely. I wonder if you should also consider tablets as part of the mobile devices target group for this survey, specially given their growing popularity. --GlimmerPhoenix 13:17, 26 July 2011 (UTC)

Thanks, we introduced some questions about their tablet use. But you are right, we are primarily focused on phones for this study. Manipande 04:05, 28 July 2011 (UTC)

Comments about questionnaire

• A1.c: Perhaps I'm missing something, but I'm not sure how any user could be answering this survey, given its length and format, from a "basic terminal". In addition, I would use shorter descriptions for each option, and I would try to better clarify the difference between what you understand as smartphone and feature phone. In the current version, the difference (if any) is quite subtle for an average user. --GlimmerPhoenix 13:17, 26 July 2011 (UTC)

Thanks, great point Manipande 17:08, 26 July 2011 (UTC)

• A2: I suspect you should be able to automatically retrieve information about the OS and browser that it's being used to navigate the survey. --GlimmerPhoenix 13:17, 26 July 2011 (UTC)

We possibly can, but to protect the privacy of our users as a best practice we don't store any information like their IP, OS or browser info while collecting data. Manipande 17:08, 26 July 2011 (UTC)

• A9 b) and d): Duplicated? --GlimmerPhoenix 13:17, 26 July 2011 (UTC)

Great point, thanks Manipande 17:08, 26 July 2011 (UTC)

• B1B and S3 questions are duplicated. --GlimmerPhoenix 13:17, 26 July 2011 (UTC)

Again, good catch Manipande 17:08, 26 July 2011 (UTC)

One question

Just a quick question: do we need to translate titles? (I guess yes but just to confirm.) And do we need to translate things like 'Section I' itself?--Bencmq 04:13, 29 July 2011 (UTC)

Yes, I would suggest you translate all of the titles and the "Section #" headings. The only title that you probably don't need to translate is "Terminate language", which I don't think will actually be shown to the respondent—"terminate language" sounds a little mean because it's so blunt. ;-) If you're having trouble translating that one, it basically means "the respondent doesn't meet the parameters for the survey, so stop the survey (terminate it)". Cbrown1023 talk 18:41, 30 July 2011 (UTC)

Mobile devices – e-book readers?

It seems like e-book readers like the Amazon Kindle or even special Wikipedia devices like the WikiReader are excluded from the survey – they are not even mentioned by questions like B1C. Is this an intentional choice? Regards, --Church of emacs talk · contrib 13:44, 31 July 2011 (UTC)

UPDATE: We have added e-readers to the following questions. Please make updates accordingly across all languages currently in translation (and in English):

• S5. Please add the following stub:

An e-reader such as an Amazon Kindle

• B1C. Please add the following stub:

An e-reader such as an Amazon Kindle

• B2B. Please add the following stub:

An e-reader such as an Amazon Kindle

This has been added to all translations. Cbrown1023 talk 23:51, 11 August 2011 (UTC)

Section IV: Mobile Access to Wikipedia

C1. Where do you access Wikipedia on your mobile phone?

• In my home
• In my office
• On my mobile phone

The last option seams out of place here... MarianoC 21:33, 31 July 2011 (UTC)

Thanks, you are right. We deleted it. Manipande 21:55, 2 August 2011 (UTC)

I'm still seen it in the English version. --MarianoC 07:30, 4 August 2011 (UTC)

"On my mobile phone" has been deleted from C1.Resolve 22:50, 8 August 2011 (UTC)

• C8. On a scale from 1-10, how important are the following changes to you for improving navigation within Wikipedia on your mobile phone?
  • Not Important At All
  • Very Important
    • A glossary of different sections that you can click to go to the section in which you are interested
    • The ability to expand and collapse (show & hide) sections of articles
    • The ability to set any page in Wikipedia as the home page for Wikipedia on your mobile phone’s browser
    • An easy-to-find search box on each page

The first two options don't make any sense.

It would be important to revise and freeze the original texts before the translations start; otherwise we end up having twice the job translating something, hunting for the fixes and translate also those. --MarianoC 13:12, 1 August 2011 (UTC)

Thanks for the suggestion, we updated the stubs in the questionnaire. I hope it is clearer now.

• A glossary/list of sections of an article at the top of the page that you can click to go directly to the section in which you are interested
• The ability to expand or collapse (show & hide) sections in an article

The issues I report remain unsolved. I would be very important to fix them as soon as possible: a bad original leads to bad translations. Fixing the translations later implies a lot more work for everybody. Thanks, --MarianoC 08:40, 2 August 2011 (UTC)

Sorry for going back and forth about it. We have revised the first stub to be "A glossary/list of sections of an article at the top of the page that you can click to go directly to the section in which you are interested" and would appreciate some additional feedback on second stub since it seems pretty straightforward. Would additional information help? Manipande 16:50, 2 August 2011 (UTC)

Re "The first two options don't make any sense." — they're the two extremes of the 1-10 scale. 1 is "Not Important At All" 10 is "Very Important". The question will be coded kinda like a table, where the top row will be 1/Not Important At All, 2, 3, 4, 5, 6, 7, 8, 9. The other options will be what you are ranking on a 1-10 scale. Does that make more sense? Cbrown1023 talk 18:45, 8 August 2011 (UTC)

That is correct, the question will be a scale question and the 4 answer stubs will appear along the right side of the table (1-10 across the top of the page). The respondent will then select how important each one is in terms of navigating Wikipedia on their mobile phone.Resolve 17:09, 9 August 2011 (UTC)

Section II, A3: providers

There is no provider for French-speaking countries, is it normal? I know only for France, there are three major providers: Orange, Bouygues, SFR; and some small operators using the networks of these providers (M6 Mobile, Virgin Mobile, Breizh Mobile, a lot of others), but I think only the three majors should be mentionned. ~ Seb35 [^_^] 21:14, 4 August 2011 (UTC)

We have the top 3 mobile phone service providers for France, China and Taiwan to A3. Resolve 22:46, 8 August 2011 (UTC)Resolve

I changed the answers for France, you wrote twice SFR, check if you need to update the survey (@Cbrown1023: I updated all translations)

It would be great to add for the 3 new states a choice "Other".

~ Seb35 [^_^] 13:37, 10 August 2011 (UTC)

An "Other" option has been added to each of the new countries at A3.38.98.13.47 22:59, 10 August 2011 (UTC)

II, A2: Windows Mobile

I've been informed, that "Windows Mobile" has been renamed to "Windows Phone Classic" and most recent OS "Windows Phone 7" is missing. I have thus added it to the English template and will update all other languages. Regards, --Church of emacs talk · contrib 09:23, 10 August 2011 (UTC)

Test questionnaire

It would be useful for translators to be able to test the questionnaires before the official launch. I just checked if it’s possible in LimeSuvey: before activating the questionnaire, an address http://limesurvey.example.org/index.php?sid=87385 can be given to translators and no answer is saved. It’s important not to activate the questionnaire because it cannot be modified when it is activated. ~ Seb35 [^_^] 14:40, 10 August 2011 (UTC)

We will send out the survey to the translators before official launch for everyone to review and provide feedback.38.98.13.47 18:20, 10 August 2011 (UTC)

Proportional data plans

I am wondering if it might make sense to add another choice for "proportional" data plans in A5. Japanese mobile phone provides often provide non-flat data plans (for example, [1]), that are different from either of the two choices given in the survey. At least, you could give another option saying "none of above". If only the two choices of limited or unlimited are given, a considerable number of Japanese participants might be confused. --whym 00:41, 11 August 2011 (UTC)

A "None of the Above" option has been added to A5. Thank you for the suggestion!Resolve 17:35, 11 August 2011 (UTC)

This has been added to all translations. Cbrown1023 talk 23:51, 11 August 2011 (UTC)

A3#SHOW IF PUNCH «29» AT S4 (RUSSIA)

I found the list with Russian providers is very strange. MTS, Beeline, Megafon, Tele2 Russia - it's okay; you haven't Skylink - it's strange; but then you list Uralsvyazinform that nonexists from April. Okay, if you want to list some affiliated companies you may do it; but you don't list mother company Rostelekom at the same time (and it has at least 9 telecommunication operators that affiliated, so why is Uralsvyazinform?). ru:Категория:Операторы сотовой связи России contains more companies then are in this list too. So if you want to list the main companies + option 'others' - it is MTS, Beeline, Megafon, Tele2 Russia and Skylink, if you want to try list them all - here is the links. Now the list seems very, very strange, as I said. Lvova 17:39, 11 August 2011 (UTC)

Thank you for this information. The questionnaire has been updated to show the following Russian providers: MTS, Beeline, MegaFon, Tele2 Russia, Sky Link and Other.Resolve 23:02, 11 August 2011 (UTC)

This has been added to all translations. Cbrown1023 talk 23:51, 11 August 2011 (UTC)

6 Section V: Attitudes & Preferences for Wikipedia User Experience

Redundant options in D7:

• D7. Below is list of user experiences that you might have had while reading Wikipedia on your mobile phone. On a scale form 1-10, please review each one and indicate how much you agree or disagree with the statements.
  • ...
  • I am able to read the article without zooming
  • ...
  • I do not have to zoom to read the article
  • ...
  • I have to zoom to be able to read the article

Plus, I might be mistaken, but the following option:

• The text formatting in the article do not fit on my phone screen

Should be

• The text formatting in the article does not fit on my phone screen

--MarianoC 12:01, 12 August 2011 (UTC)

We have removed the first two responses ("I am able to read the article without zooming" and "I do not have to zoom to read the article"). The last stub will remain in the questionnaire ("I have to zoom to be able to read the article"). Resolve 17:09, 12 August 2011 (UTC)

We have changed "do not" to "does not" Resolve 17:09, 12 August 2011 (UTC)

Question about A6

Hi! Question A6 mentions some direct options and adds two general ones: “Other” and “None of the above”. Don't they exclude each other? Or do you mean with the second one something like “I have no data plan on your mobile phone”? Kind regards, —DerHexer (Talk) 13:30, 12 August 2011 (UTC)

We have removed the "None of the Above" option.Resolve 17:17, 12 August 2011 (UTC)

Turkish Translation

Hi all, The Turkish Translation has been completed and proofread. If someone from the team would be able to provide any feedback, it would be greatly appreciated! Thank you - Resolve 17:42, 29 August 2011 (UTC)

In which country do you live?

• Turkish option, "Turkish Republic of Northern Cyprus" or "Cyprus" should be. Emperyan-message/ileti 11:49, 14 September 2011 (UTC)

German / english mixed in question

I found out that one question is mixed english/german for the german version:

"Angenommen, in Wikipedias mobile Seite würden neue Funktionen integriert, wie wahrscheinlich würdest du dann die folgenden Funktionen zum Bearbeitem oder Beteiligen in Wikipedia nutzen?

• A feature that allows you to upload pictures from a mobile device to Wikipedia
• A feature that allows you to rate Wikipedia articles on a scale
• ..."

Please fix Groetjes Neozoon 23:30, 10 October 2011 (UTC)