Research talk:Tech support satisfaction poll/Questions

Areas

I won't comment on the points chosen, but the closed list of areas seems an excessive restraint. Is there research to show that those points capture the near totality, of even the large majority, of the time spent working on our wikis? At a minimum, you should have a point for "Other". --Nemo 19:44, 20 August 2015 (UTC)

The list items were originally more general, but Ricordisamoa complained that they were too vague (on Phabricator), so I tried to come up with a more specific list. I could add 'other', but that would be extremely vague. Are there significant areas that I'm missing and should add? Kaldari (talk) 05:53, 24 August 2015 (UTC)

You're missing such a minor thing as "expanding/creating articles", for instance. I don't have time to spend on such a difficult job as classifying wiki work, and you probably don't either (it's a sizable research project).

For such arbitrary option lists, it's extremely important to have an "Other" answer so that people can avoid assessing the others if necessary: in that way, you can then know how good your question was, which is a useful result in itself (I mean, depending on whether 1 %, 10 % or 80 % preferred "Other" to the rest). --Nemo 07:12, 24 August 2015 (UTC)

@Nemo bis: The list mostly excludes areas covered by other teams, i.e., editing, searching, site performance, i18n, etc. I'll talk with Edward about the possibility of an Other option. Kaldari (talk) 23:37, 24 August 2015 (UTC)

We've decided against using an 'other' option for 4 reasons: it's too vague to actually give us any actionable information, it doesn't necessarily apply to the scope of our team (for example our team isn't responsible for the editing interface or interface localization), 'other' is already somewhat covered by the "How satisfied are you overall with the technical support provided by the WMF to Wikipedia?" question, and we've already started the translations and are hoping to do the survey this week. Kaldari (talk) 19:02, 25 August 2015 (UTC)

Ok. I don't think team divisions are a given users should be bothered with. Do you expect differences between "How satisfied are you overall" question and the breakdown question to be able to surface dissatisfaction with the provided options? --Nemo 07:53, 26 August 2015 (UTC)

We're not actually asking the users to consider team divisions, we just happen to be asking about areas within the scope of our team. I don't really expect the overall satisfaction question to reveal dissatisfaction with the provided options, but I'm OK with that. This survey is only going to be used by the Community Tech team (which is why it's so limited), and I don't want to ask questions that aren't going to actually be used for anything (which is why I'm OK with removing the gender question). Do you think this will be confusing or frustrating for survey participants? Do you think it would be worth asking about other areas such as editing and language support even if we (Community Tech) aren't likely to do anything with that information? Kaldari (talk) 00:33, 27 August 2015 (UTC)

@Nemo bis: FYI, I'm going to be without any internet access for the next week and a half, so my apologies if I don't respond here for a while. I'm putting the survey on hold in the meantime. Kaldari (talk) 00:47, 28 August 2015 (UTC)

@Nemo bis: We need to move ahead with this survey (as we're 2 weeks behind schedule now). Do you have any thoughts on my questions above? Kaldari (talk) 00:21, 9 September 2015 (UTC)

Sorry, but I don't want to enter politics of jurisdiction between areas of WMF. Most editors are uninterested too, so your question is quite disappointing. I don't think a question tailored to a WMF organisational structure makes any sense and I maintain that "Other" would be useful. If "Other" is impossible, I understand it's not your fault. --Nemo 14:47, 9 September 2015 (UTC)

@Nemo bis: Talked with Edward about this and we added a new question to the survey which should hopefully cover some of your concerns. Kaldari (talk) 22:19, 14 September 2015 (UTC)

Gender is relevant to tech support satisfaction?

I'm surprised to see gender in this survey, and not age or other identity traits. Why is gender being collected? How will that data be used when evaluating the survey results? John Vandenberg (talk) 03:40, 22 August 2015 (UTC)

ugh, the answer on Research:Tech support satisfaction poll leaves a lot to be desired. If you want to collect useful demographic information, please either do a general survey, or clearly indicate that this question will not be used as part of the poll evaluation, otherwise you skew the data. John Vandenberg (talk) 03:44, 22 August 2015 (UTC)

Indeed. We already know that such gender data will be useless to assess the gender gap, per https://mako.cc/copyrighteous/the-wikipedia-gender-gap-revisited . If it has no use to assess tech support satisfaction, it should be removed. --Nemo 07:24, 22 August 2015 (UTC)

@John Vandenberg and Nemo bis: Thanks for the feedback. Which would you prefer: Creating a clearly designated "demographic" section that asks for gender, age, and possibly other traits; or removing the question entirely? Regarding Mako's paper, I draw a different conclusion from it. The paper "describes a method online communities can adopt to estimate contributor demographics using opt-in surveys." It doesn't declare that such surveys are useless. Kaldari (talk) 05:48, 24 August 2015 (UTC)

Removal is best, also because translations are ongoing already. --Nemo 07:07, 24 August 2015 (UTC)

OK, we're going to take it out for now, although we may want to collect more demographic information in future surveys. Kaldari (talk) 18:54, 25 August 2015 (UTC)

Thanks. --Nemo 07:53, 26 August 2015 (UTC)