Grants:IEG/The use of Wikipedia by doctors for their information needs

status: not selected

Individual Engagement Grants
Individual Engagement Grants
Review grant submissions
review
grant submissions
Visit IdeaLab submissions
visit
IdeaLab submissions
eligibility and selection criteria

project:

The use of Wikipedia by doctors for their information needs


project contact:

richard.pullicino(_AT_)gmail.com

participants:



advisors: Henry Potts

summary:

To investigate the knowledge, perspectives and attitudes of doctors on their interactions with Wikipedia to satisfy their information needs.

engagement target:

Wikipedia

strategic priority:

Improving Quality, Increasing Participation

total amount requested:

16,000 USD


2014 round 1

Project idea edit

Medical knowledge is ever expanding and it is getting increasingly difficult for doctors to master and assimilate the right knowledge to use in the right clinical circumstances. In his book Weisz [1] stated that the reason why specialities were formed in the medical community was because of intense professional competitions, a growing public faith in scientific explanations and an increase in public standards and authorities in the field of healthcare. With the increase in expectations came an increase in information needs. Information needs are defined as expressions of missing information that is required to accomplish a particular task – which in doctors may be managing patient care [2]. Research has shown that a doctor needs to know around 10,000 different diseases, signs and syndromes [3] and two million pieces of information to manage patients [4].


Therefore doctors have unmet information needs the reasons of which may be:

  • a problem in converting the information needs into a proper query [5].
  • a deficiency in skills and lack of time to complete searches efficiently [6]
  • a lack of integration of the search process into the clinician’s workflow [7]


Wikipedia provides information that is readily accessible and easy to navigate through and may be the reason why it is used by doctors to satisfy their information needs. Also as search engine algorithms improve, the easier it will get for natural language requests to be converted into proper queries thus making it easier to use. Another reason why Wikipedia is being highly used by clinicians is due to its prominence in search engine results such as Google[8]


There are concerns about the validity of the content present in Wikipedia and that it may be subject to online vandalism. However a study had compared Wikipedia with Encyclopaedia Britannica and had found that both encyclopaedias had similar number of errors [9]. Another study had compared drug information in Wikipedia with an authoritative and free drug reference database and found that while information in Wikipedia had a narrower scope and was less complete, no factual errors were actually found [10] Overall both medical students and doctors are familiar with Web 2.0 technologies such as Wikipedia [11]and Wikipedia has a role in information seeking for medical education despite the occasional shortcomings with regards to the depth of information provided [12]. A systemic review had show that Wikipedia could be an important learning tool when used together with other learning materials [13]. Not only that but the quality of the medical articles in Wikipedia is slowly increasing [14]. Despite knowing that Wikipedia may lack depth in certain areas, clinicians still use Wikipedia as a reference source for clinical decision making [13]. Wikipedia has also been used in the realm of Public Health both to obtain information about outbreaks [15] and to provide public health messages to the general population [16]


Research Question edit

How and when do doctors use the information obtained from Wikipedia in their medical education and clinical work? To what extent do doctors interact with Wikipedia to satisfy their information needs ?


Project goals edit

  1. To assess how and under what circumstances do doctors interact with Wikipedia, both for educational purposes and clinical decision-making.
  2. To assess why Wikipedia is occasionally preferred over more authoritative sources of medical information.
  3. To identify the characteristics of doctors that are most likely to contribute to medical Wikipedia articles and what motivates them to do so.


Part 2: The Project Plan edit

Project plan edit

Scope edit

Activities edit

The study will be using an exploratory sequential design. This is a mixed method research approach which involves the collection and analysis of both qualitative (open-ended) and quantitative (closed-ended) data. This approach draws on the strengths of both qualitative and quantitative research while minimising the limitations of both approaches. An exploratory sequential design is used so as to develop an instrument in the initial qualitative phase of the study.


The first phase of the study will be a qualitative exploration of how and under what circumstances do doctors interact with Wikipedia in which interviews with be collected from a focus group composed of doctors working in different grades and specialities at Mater Dei Hospital, Malta. An attempt will be made to do the same with doctors working at University College London hospital (UCL). After obtaining the necessary permissions from the hospital data controllers, a list containing the doctors' particulars (Name, Surname, Grade, Speciality and Email address) will be obtained from the hospital administration. An email will be sent to all the doctors working in the hospital. Financial reimbursement will be offered. There will be one focus group per hospital composed of 15 to 20 members. Verbal and written consent will be obtained from all participants to digitally record the discussions which will then be transcribed. Field notes will also be collected during the interviews. From the initial exploration and analysis, the qualitative findings will be used to develop assessment measures. The development of the instrument will be the second phase of the study.


In the quantitative phase (third phase), data from the developed questionnaire will be collected from doctors working at Mater Dei Hospital. An attempt will also be made to do the same with doctor working at UCL hospital. A statistical analysis package will be developed according to the different question types and in line with the main research questions. The questionnaire will be on online, via a website running the open source survey project Limesurvey. The site will be hosted on a webserver with SSL/TLS encrypted network communication and will have the capability of generating email invitations. These invitations will contain information about the study and two webpage links. One link is an opt-out link which if clicked will automatically omit the doctor from the study. The other link will direct them to the online questionnaire.


An attempt will be made to advertise a version of the online questionnaire on Facebook and LinkedIn by purchasing ads targeted at doctors. This will depend on the financial resources available.


A pilot study will be performed and will involve testing the instrument on a small convenience sample of 15-20 participants. This should highlight any limitations or misunderstandings that may be present in the questionnaire.


The invitations will not be sent to those doctors who participated in the focus groups. In order to encourage participation, those who complete the questionnaire will be eligible to participate in a lottery with the chance of winning a voucher from an online store.


To encourage participation, a five-minute presentation will be given in different departmental weekly continuous medical education talks. The presentation will highlight what the project entails. Permission to hold these meetings will be obtained from the respective heads of the different hospital departments.


Inclusion Criteria edit

  • Doctors working in Mater Dei Hospital or UCL (for both the qualitative and quantitative phases)
  • Doctors answering the advertisement on Facebook or LinkedIn (for the online questionnaire)
  • Doctors who have used Wikipedia in clinical situations or for medical education purposes


Exclusion Criteria edit

  • Junior doctors in their first six months of training
  • Non-english speaking doctors


Data Management edit

Data will be collected according the Maltese and British data protection acts. Since the study will require particulars of hospital doctors, permission will be obtained from the hospital's Data protection officer. In the qualitative phase the study, the recorded interviews and transcripts will be stored in a USB drive which uses military-grade encryption (IronKey® USB Drive). In the transcription, the participants' anonymity will be maintained and only known to the researcher of the study. The audio files will be erased once the study is finished. With regards to the quantitative phase, backups from the web server and related SPSS® generated data will be stored on a different military-grade USB drive. Anonymity of response will be maintained thanks to a feature in Limesurvey that keeps the participants details and their responses in two different tables unlinked from each other. Thus one wold be able to know whether a participant has filled in the questionnaire (so that reminders can be sent) but would not know their responses. For further security, only https connections will be allowed to the website. Data will be backed up from the server on a weekly basis.


Bias edit

The following form of biases were identified:


Sampling Bias - Membership bias
It may be difficult to obtain a representative sample of the different doctors working at various grades and in different specialities.


Recall Bias
This can occur when filling in the questionnaire, when subjects selectively remember details from the past


Halo Effect
Since the researcher in this study is also a doctor, participants may try to answer in the way they believe the researcher would want then to answer rather than according to their own beliefs. In order to counteract this effect, the researcher will be non-judgemental and will try to be as objective as possible during the focus group interviews.


Central tendency bias
Likert scales will be used in the questionnaire. Participants may avoid using extreme responses.


Ethical Issues edit

Prior to conducting the study, permission will be obtained from the University of Malta Research Ethics Committee (Mater Dei Hospital in Malta is the only university teaching hospital in Malta) and from University College London. Informed consent will be obtained from all participants of the study. Participants will be informed of the purpose of the study and how data will be collected and processed at every phase and how it can be used in the future.

During data collection at both qualitative and quantitative phases, it will be made sure that data that can perceived as potentially harmful to the participant is not collected. Also leading questions will be avoided and sensitive information will not be disclosed.

During data analysis both negative and positive results will be disclosed and multiple perspective will be reported. The privacy and anonymity of participants will be respected.

Data Analysis edit

The qualitative and quantitative data will be analysed separately. Content analysis will be carried out on the qualitative data and the findings from the initial exploratory data will be used to build quantitative measures. A statistician will be approached prior to the start of the study. Data will be analysed using SPSS®. Where possible non-parametric tests will be used. A complete statical analysis package will be compiled after the instrument has been designed and will be modified after the pilot study.

The qualitative and quantitative data cannot be compared because they are drawn from two different samples.


Budget edit

Total amount requested edit

$16000


Budget breakdown edit

Item Budgeted Amount
Financial Reimbursement to doctors participating in focus groups (Target: Total of 40 doctors, $200 each for reimbursement $8000
Lottery Prize for doctors participating in the online questionnaire $2000
Advertisements on social network sites targeting doctors $5000
Hosting, maintenance and processing of online questionnaire Free
Travel Expenses to travel to the UK to carry out the second focus group (1 person) $1000

Intended impact edit

Target audience edit

Knowing how doctors use Wikipedia could be a stepping stone to try to improve editorship by doctors for medically related Wikipedia articles. This will benefit both patients and doctors.

Community engagement edit

  • In the first phase of the study a focus group amongst doctors will be held at Mater Dei Hospital in Malta and an attempt will be made for another focus group to be held at University College London hospital.
  • In the quantitative phase (third phase), data from the developed questionnaire will be collected from doctors working at Mater Dei Hospital. An attempt will also be made to do the same with doctors working at UCL hospital. If funding permits another version of the questionnaire will be advertised to doctors on social network sites.

Fit with strategy edit

Improving the quality of medically related wikipedia articles. By understanding the dynamics on how doctors make use of Wikipedia would help to create programs that welcome and encourage more doctors to edit medically related pages, increasing editorship.

Sustainability edit

This depends on research findings. For example if the interviews and questionnaires show that doctors access Wikipedia more than they access research articles and journals, a future idea would be to try to encourage doctors to edit medically related articles because of the higher impact that they will have on other health practitioners and patients. If this type of awareness is created in the medical community, doctors could be encouraged to edit articles as a way to boost their CVs.

Measures of success edit

  • Response rates
  • Using the research findings to create programs to encourage doctors to edit medically related Wikipedia articles.

Need target-setting tips?

Participant(s) edit

Dr. Richard Pullicino. Trainee radiologist and reading a Masters in Health Informatics.

Dr. Henry Potts. Senior Lecturer at the UCL Centre for Health Informatics & MultIprofessional Eduction. An avid wikipedia user, he is interested in the contributions made to Wikipedia health-related articles and what motivates users to edit these articles.

Discussion edit

Community Notification edit

Please paste a link below to where the relevant communities have been notified of this proposal, and to any other relevant community discussions. Need notification tips?

Endorsements edit

Do you think this project should be selected for an Individual Engagement Grant? Please add your name and rationale for endorsing this project in the list below. Other feedback, questions or concerns from community members are also highly valued, but please post them on the talk page of this proposal.

  • Community member: add your name and rationale here.

References edit

  1. George Weisz Professor of Social Studies of Medicine McGill University, 2005. Divide and Conquer : A Comparative History of Medical Specialization, Oxford University Press.
  2. Braun, L.M.M. et al., 2006. Avoiding literature overload in the medical domain. Studies in health technology and informatics, 124, pp.497–502. http://ebooks.iospress.nl/publication/9734
  3. Davenport, T.H. & John, G., Just-in-Time Delivery Comes to Knowledge Management - Harvard Business Review. Harvard business review. Available at: http://hbr.org/2002/07/just-in-time-delivery-comes-to-knowledge-management/ar/1%20
  4. Smith, R., 1996. What clinical information do doctors need? BMJ, 313(7064), pp.1062–1068. http://www.bmj.com/content/313/7064/1062
  5. Cucina, R.J. et al., 2001. Empirical formulation of a generic query set for clinical information retrieval systems. Studies in health technology and informatics, 84(Pt 1), pp.181–185. http://ebooks.iospress.nl/publication/19460
  6. Ely, J.W. et al., 2005. Answering physicians' clinical questions: obstacles and potential solutions. Journal of the American Medical Informatics Association : JAMIA, 12(2), pp.217–224. http://jamia.bmj.com/content/12/2/217
  7. Ely, J.W. et al., 2002. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. BMJ, 324(7339), p.710. http://www.bmj.com/content/324/7339/710
  8. Law, M.R., Mintzes, B. & Morgan, S.G., 2011. The sources and popularity of online drug information: an analysis of top search engine results and web page views. The Annals of pharmacotherapy, 45(3), pp.350–356. http://www.ncbi.nlm.nih.gov/pubmed/21343404
  9. Giles, J., 2005. Internet encyclopaedias go head to head. Nature, 438(7070), pp.900–901. http://www.nature.com/nature/journal/v438/n7070/full/438900a.html
  10. Clauson, K.A. et al., 2008. Scope, completeness, and accuracy of drug information in Wikipedia. The Annals of pharmacotherapy, 42(12), pp.1814–1821. http://aop.sagepub.com/content/42/12/1814.short
  11. Sandars, J. & Schroter, S., 2007. Web 2.0 technologies for undergraduate and postgraduate medical education: an online survey. Postgraduate Medical Journal, 83(986), pp.759–762. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2750915/pdf/759.pdf
  12. Hughes, B. et al., 2009. Junior physician's use of Web 2.0 for information seeking and medical education: A qualitative study. International Journal of Medical Informatics, 78(10), pp.645–655. http://science.icmcc.org/2009/07/07/junior-physicians-use-of-web-20-for-information-seeking-and-medical-education-a-qualitative-study/
  13. a b Muhlen, von, M. & Ohno-Machado, L., 2012. Reviewing social media use by clinicians. Journal of the American Medical Informatics Association, 19(5), pp.777–781. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3422846/
  14. Schmidt, T., 2013. Informal education of medical doctors on the Internet. Studies in health technology and informatics, 190, pp.92–94.
  15. Oehler, R 2012. An Overview of Social Media in the Practice of Infectious Diseases pp.1-6 http://journals.lww.com/infectdis/Abstract/2012/11000/An_Overview_of_Social_Media_in_the_Practice_of.3.aspx
  16. Metcalfe, D. & Powell, J., 2011. Should doctors spurn Wikipedia? JRSM, 104(12), pp.488–489. http://jrs.sagepub.com/content/104/12/488.long