Grants talk:Project/Rapid/Mcbrarian/Wikipedia and Consumer Health

Comments from I JethroBT (WMF)

Hello Mcbrarian, and thanks for your research proposal focused on how people use health content on English Wikipedia. The only request we have is to detail the specific expenses in your budget:

• How much the is the NVivo software to support coding?
• How much funding is being set aside for supporting your study participants?

With thanks, I JethroBT (WMF) (talk) 16:07, 4 June 2021 (UTC)

@I JethroBT (WMF): Thanks for these questions. Here is a breakdown of the projected funding distribution:

• NVivo Software:
  • $650.00 USD + 13% sales tax = $734.50 USD
• Gift cards for study participants:
  • Budget for 30 cards at $25.00 CAD each (study is taking place in Canada) = $750.00 CAD ($609.34 USD at time of proposal submission)

Comments from Colin

Mcbrarian, I'm worried that 25 or 30 participants won't be enough to provide information about the "population" of users of Wikipedia health information. It would seem very sensitive to their selection, for example, age, background, urban/rural, internet access, mobile access and smartphone usage, smart device usage, etc. The choice of gift card might select for those who live or work in a city and who regularly buy takeaway drinks or food. The online nature of the interview will also be selective (i.e. filter out those whom the digital divide already disadvantages)

If people are accessing health material via a search engine, whether they end up consuming Wikipedia content may vary depending on their location and choice of engine. For example, searching for the drug "levetiracetam" in the UK and in Canada (via a VPN) produces a Google side bar with content from Wikipedia (though the user may think this content is from Google). In the UK, the NHS is the top result and the Wikipedia page is sixth. In Canada, Epilepsy.com is the top result and Wikipedia is not in the first page of results. Searching for "Epilepsy" in the UK produces an elaborate NHS-based side-bar with tabs, and a "featured snippet" that is also based on NHS material. There are two adverts for charities and three news stories, and Wikipedia is the second last result on the page. In Canada, I get a featured snippet from Mayo Clinic and Wikipedia is third last on the page.

Results from other search engines and AI smart devices will vary again. This suggests to me that whether people end up consuming Wikipedia content and whether they are aware that they are doing so, is mostly influenced by factors unrelated to either Wikipedia or the user, but by choices made by big corporations in California. So "having access to Wikipedia’s health content" may be less a socio-economic or digital divide issue, but simply a factor of how they searched.

I'm not sure what the benefit of the study is to WMF or the editing community. For example, smart-device access to Wikipedia content makes it hugely more accessible, but isn't something WMF/community did (aside from the original idea that the content would be free to use). As an aside, the NVivo software would presumably have a benefit beyond the study, but I don't know how WMF weight that when calculating things. -- Colin (talk) 20:11, 18 June 2021 (UTC)

Hi @Colin:, Thanks for your thoughtful and detailed feedback on this study. The actual protocol does go into more detail than what is provided in this grant application and I think that would likely have answered some of your questions. I will be posting the protocol on a project page soon, I'm just waiting for final ethics approval from my institution. Meantime, here are some brief responses for you:

25-30 participants is a manageable population for this type of study (interview based) that will be complemented by a survey to determine whether there is any correlation between findings from the interview and demographic/socio-economic factors. I absolutely agree that the participants in this study will not provide a thorough picture of everyone who uses Wikipedia's health information, but I don't think that is possible to do in a single qualitative study. The online nature of the interviews and survey are an unfortunate reality of proposing research during a global pandemic, but they have the added benefit of removing the geographical barriers. I am physically located in Canada where it is expected that will continue to work remotely until September at the earliest, possibly longer. I do intend to acknowledge the limitations of this study with respect to the nature of self-selection and one's ability to connect online. It will also be limited to populations that frequent the environments that I am able to post recruitment posters in.

I appreciate your observation about how the user searches online can impact the discoverability of Wikipedia and I am going to consider making space in the interview for that when I ask how participants access Wikipedia, if they recall how they came to its content, etc. The main purpose of this study is built on library and information science theories of health information behaviour, which aims to understand what factors motivate an individual to search, retrieve, and use health information and what that looks like. At this time there is no research like this and so this study is essentially a first "dip of the toe" in what I suspect is a very rich field of inquiry. Since it's the first of its kind, that I'm aware of, it is my hope that this initial study will lead not necessarily to answers, but to the development of specific, targeted questions that can explored more deeply.

The overall goal is not to show that people are accessing Wikipedia's health content, that has already been established. The goal is to start a conversation about what might be happening once the access has occurred. Currently we know nothing about this and the narrative in the WP community re: Health content is limited to: "people are looking at Wikipedia's health and medical articles".

Since I am a researcher without access to funds at my institution (I'm a librarian with a research requirement in my position description), I have submitted this funding request from WMF to cover the cost of NVivo software, a specialized tool that makes coding qualitative (in this case, interview) data streamlined and efficient.

Thanks again or your comments. I hope this response provides some clarity. Once I receive final ethics approval (coming in the next 7 days or so, I hope), I will start a project page and tag you here when it's live so you can read more about the study in detail, if you'd like. Mcbrarian (talk) 11:32, 20 June 2021 (UTC)