Grants:PEG/Consumer Reports/Wikipedian in Residence/Report

Report accepted

This report for a Project and Event grant approved in FY 2011-12 has been reviewed and accepted by the Wikimedia Foundation.

To read the approved grant submission describing the plan for this project, please visit Grants:PEG/Consumer Reports/Wikipedian in Residence.

You may still comment on this report on its discussion page, or visit the discussion page to read the discussion about this report.
You are welcome to Email grants at wikimedia dot org at any time if you have questions or concerns about this report.

Compliance and completion

Did you comply with the requirements specified by WMF in the grant agreement?: YES

Is your project completed?: YES

Did you use any of the grant funds?: YES

Activities and lessons learned

This section describes what the grantee did, and what the grantee learned from implementing the project. This section should be useful to others implementing similar projects and is an opportunity for the grantee to reflect on the project's performance.

Activities

Provide a detailed list of activities performed to complete this project, descriptions of these activities, and the amount of time spent on each activity. This section should also include a list of participants, or a link to pictures, blog posts, or videos from the project or event.

To carry out this project, Consumer Reports hired Lane Rasberry as Wikipedian in Residence, beginning work on April 30, and continuing for six months on the Wikipedian-in-Residence project from May through October 31.

Overall, the CR Wikipedian-in-Residence project had two primary goals:

Providing Wikipedia expertise to increase public access to the healthcare information produced by the Choosing Wisely program, a special educational campaign supported by Consumer Reports. (Although this goal was not part of the grant request, we are proud of the results, and note that they support Wikimedia's strategic goals of engaging experts in Wikipedia, and of improving the quality of Wikipedia content; for a more detailed description please see this blog post.)
Investigating ways that Consumer Reports might engage with the Wikipedia community, including promotion of interaction between Consumer Reports staff and Wikipedians in other consumer topic areas.

This grant was sought in furtherance of the second goal. During the course of the residency, the Wikipedian in Residence led and supported a wide range of activities to explore and promote CR-Wikipedia community engagement in new topic areas. Through these efforts, a range of opportunities were identified for future followup.

Future Public Education Initiatives and Public Policy Articles: In addition to Choosing Wisely, Consumer Reports is sponsoring several other major public education outreach and public policy projects, in areas such as product safety, food safety, financial services and telecommunications. Through the residency, Consumer Reports was able to clarify that the most promising areas for future engagement would likely be additional public service topics such as patient safety, food safety or product safety, where CR has important information for consumers and policymakers that it seeks to widely disseminate. The organization now is able to see the demonstrated impact of Wikipedia engagement on health care issues, and can see a range of possibilities for applying this model to other public policy issues and education campaigns.

Consumer Reports is also exploring further possibilities of improving Wikipedia content with scholarly academic articles on public policy, as well as developing articles on US federal consumer protection law. Rasberry also advised staff about opportunities for preparing or contributing to other articles on other public policy issues, including food safety and product safety. He emphasized the importance of appropriate sourcing and the inclusion of other points of view e.g. business, policymaker or other stakeholder perspectives.

Possible Future Health Education Campaigns: In addition to Choosing Wisely, Consumer Reports is running several other major education outreach efforts on health care issues. Lane helped identify Wikipedia engagement opportunities for three other campaigns and educational initiatives: the Safe Patient Project, Consumer Reports Best Buy Drugs and High Value Care:

Best Buy Drugs
Best Buy Drugs is a consumer drug comparison educational outreach program. The program publishes plain-language patient guides to pharmaceutical treatments by condition. The Wikipedian in Residence investigated pilot conditions for integrating Consumer Reports content into Wikipedia, including diabetes, Alzheimer's, and ADHD, and made recommendations for how this could be achieved. The recommendations for this project were modeled after those for the Choosing Wisely campaign.
Safe Patient Project
The Safe Patient Project is a Consumer Reports effort to support grassroots community activists in doing whatever they want to do to increase patient safety in healthcare. Many participants in this program have experienced a problem which could have been prevented by higher standards in healthcare provision and are now advocating for increased regulation. Characteristic causes of interest to this group are reduction of hospital-acquired infection, stopping unnecessary medical treatment, and increasing safety of medical devices. In November, Rasberry spoke at a national Safe Patient Project conference attended by 40 activists, and provided training and resources for sharing information on Wikipedia.
High Value Care(CR Partner initiative)
High Value Care is an initiative of the American College of Physicians (ACP) to educate physicians and the public about the cost-effectiveness of tests, treatments, and procedures used in Internal Medicine. Consumer Reports supports the initiative by translating clinical documents and guidelines into plain English and working on outreach to patients and consumers. Rasberry advised ACP on ways to engage medical residents in improving Wikipedia articles on topics relevant to ACP's work, including a Wikipedia module for the ACP curriculum in this area. This draws on learnings from the Wikipedia Education Program.

Consumer Movement History: Toward the end of Rasberry's residency, CR began to investigate and consider opportunities to share materials about consumer movement history and the organization's own history. The history of the US consumer movement, in which Consumer Reports played a significant role, and the development of both government and marketplace policies and practices to improve consumer welfare, are an important part of U.S.history that could receive fuller exploration on Wikipedia. CR also has archival materials and photographs, some of which can potentially be made available on Wikipedia. While no decision has been made yet, the organization now sees the potential value of more widely disseminating material through Wikipedia, and is considering ways to facilitate this. In addition, CR will consider ways to share information about Wikipedia with consumer organizations in other countries, through its relationship with Consumers International, a global consumer organization.

Educational Presentations: With logistical and promotional support from CR External Relations, Rasberry gave 10 lunchtime presentations to CR staff on Wikipedia culture and community guidelines. The typical format was a 20-minute talk followed by 40 minutes for questions. Presentations demonstrated some function or location on Wikipedia then described the significance of that place. The average attendance was 7 persons, with some meetings drawing 15 people. Approximately half of the people who attended any presentation also attended a second one. In contrast to the more popular “editathon” model for presentations, at these events the attendees were encouraged to talk and ask questions rather than become active participants. Through this process, we cultivated staff interest across a many different organizational departments and functional areas. In addition to these group trainings, all Consumer Reports staff who requested individual training received it. At least 20 persons requested and received such training.

Wikimania 2012: Rasberry contributed key elements to the planning and execution of GLAM Night Out, a reception and panel discussion at the Newseum which drew more than 250 guests from the Wikimania 2012 conference in Washington DC. The reception was hosted in partnership with non-profit OCLC and the GLAM-Wiki community. It celebrated the launch of the GLAM Wiki US consortium, highlighting the value of building bridges between Wikipedia and cultural institutions. Participants included GLAM-focused Wikimedians and people interested in consumer health issues. The event helped to build goodwill around Consumer Reports as a valuable partner in fulfilling the Wikimedia mission, and around the concept of Wikipedians in Residence outside the GLAM sector. The event was introduced by Pete Forsyth, Lori Byrd Phillips, and Andrew Lih -- leaders in the Wikimedia community's efforts to engage with like-minded organizations.

The panel discussion, entitled “The Sponsored Point of View: Financial Conflicts of Interest in Health Care and Science, featured Charlie Ornstein, senior reporter at ProPublica, Dr. Roy Poses of the Foundation for Integrity and Responsibility in Medicine, and Dr. Daniel Mietchen, Wikipedian-in-Residence at the Open Knowledge Foundation, and Dr. John Santa, director of the Health Ratings Center at Consumer Reports. In addition to Rasberry, three CR staff members attended Wikimania 2012 to learn more about Wikipedia and develop relationships with members of the community.

Wikimedia NYC: During his residency, Rasberry participated in at least six events sponsored by Wikimedia NYC, discussed his activities and distributed CR publications to participants. As a result of these contacts, a group of five NYC-area Wikipedians toured CR's laboratories and offices in January 2013.

Lessons learned

What lessons were learned that may help others succeed in similar projects? Consider the following questions and respond with 1 - 2 paragraphs.

What went well?

This project succeeded partly because of careful prior planning for the overall focus of the residency, which ensured strong internal ownership of the project and a clear path for collaboration between CR and Wikipedia in at least one key subject area (evidence-based health care). Second, the project succeeded because we attracted a very strong candidate who was an excellent match for the organization's internal needs and capabilities. The candidate’s broad interests facilitated easy exploration of a broad range of other consumer topic areas, and he was able to provide very specific examples of how engagement with Wikipedia could benefit CR and Wikipedia users. Further, he related well to wide range of people from different departments and functional areas within Consumer Reports. He participated in a wide range of internal meetings and activities, such as the organization’s internal Hackathon, and contributed ideas for ways that CR information could be shared in novel ways. Finally, he was able to facilitate complex discussions relating to appropriate sourcing, conflict of interest and internal social media policies, among other issues.

What did not go well?

Because of the difficult state of the economy, CR employees are under pressure to focus on core responsibilities, and it is somewhat more difficult to engage in new collaborations and projects that otherwise might be the case. As many new organizations may also find, the learning curve with Wikipedia appears to be steeper for staff members and volunteers than many of us originally expected. Therefore, to deepen engagement, we would likely need to provide more systematic training to a targeted number of people with responsibilities in specific collaborations.

Also, while we had originally anticipated doing more to explore collaborations in content areas of greater interest to women, which might attract more women contributors or editors to Wikipedia, because of time constraints, we have not yet been able to explore that issue in depth. (However we would note that a majority of family health care decisions are made by women, and many of our patient safety activists are women, so we would expect we could have some impact in closing the gender gap in editing by emphasizing those issues, as a first step in this area.) We will therefore put this issue on our followup list, and consider ways that CR can contribute more in this area.

We have also not done as much as originally anticipated to document the success of our project and tell its story to the outside world. Going forward, we do plan to expand the scope and nature of materials provided, to the extent that they may be useful as a model for other organizations.

What would you do differently if you planned a similar project?

For the sponsoring organization that is new to Wikipedia, it is difficult to say with hindsight what we could have done differently, recognizing that "you don’t know what you don’t know." We think we succeeded in part because we carefully considered how to allocate staff time and internal resources to support the planned collaborations and engagement, based upon a strong mission or business case for a specific content area. We would expect this to continue to be the case going forward, for other consumer or public health education campaigns. Looking beyond that, we now have a better idea of what resources are needed, what capacity and what type of relationships with Wikipedians we would want to have in place at the end of residency. During the residency, where possible, we should identify as specifically as possible staff members, volunteers or other stakeholders would could be trained to support the collaboration. We should also systematically consider ways we can engage and train volunteers and allied organizations who may be working with us on a particular issue. In the case of Choosing Wisely, we received substantial interest from professional medical societies with a similar interest in the appropriate use of treatments and procedures. We realized through this process that a broader project to engage several organizations with similar interests could be very effective.

Project goal and measures of success

This section should reference the project goals and measures of success described in the approved grant submission. See Grants:PEG/Consumer Reports/Wikipedian in Residence to review the goals and metrics listed in the approved submission.

Project goal

Provide the project goal here.

The immediate goal of this project is to enable wider recruitment; longer term, recruiting the right applicant will support the development of the Wikipedian in Residence model within non-profit organizations like Consumer Reports.

Did you achieve your project goal? How do you know your goal was achieved? Please answer in 1 - 2 paragraphs.

YES. As outlined above, we identified numerous promising opportunities for future CR Wikipedia engagement on public education and policy isssues, and developed a strengthened internal capacity to appreciate and understand where collaboration with Wikipedia can advance organizational goals and the public interest. We received a strong favorable response from our communications partners and identified broader opportunities for future work on health education campaigns.

Overall impact of Wikipedian in Residence at Consumer Reports: Rasberry's residency has been widely welcomed at Consumer Reports, from the Board level to the staff level. Lane has raised the organization's consciousness about the mission and value of Wikipedia and identified many areas of common interest, particularly around serving the public interest through increasing access to unbiased information. Consumer Reports has been inspired to think in more innovative ways regarding issues such as open access to information, collaboration with non-traditional partners, and community-building. The organization has become more aware and appreciative of the Wikipedia culture and the generation of volunteers that work to improve Wikipedia and expand knowledge online. We have also been gratified by Rasbarry's contribution to making the Wikipedia community more aware of Consumer Reports' mission and work and by the community's mostly positive response to our work to educate the public. Rasberry has been the perfect fit for our organization and we appreciate the opportunity to be an innovator in the nonprofit consumer world by hosting a Wikipedian in Residence.

Context of funded work: In the main part of his residency (i.e., not the portion funded by Wikimedia), Rasberry worked closely with Consumer Reports staff and the Wikipedia community to help expand access to evidence-based consumer health information, including information produced through Choosing Wisely, a health education campaign led by the ABIM Foundation and Consumer Reports. The initiative aims to provide evidence-based information about medical tests, treatments, and procedures to both physicians and consumers. The Choosing Wisely information is based on consensus recommendations from top-level medical professional societies in the United States about medical tests, treatments and procedures that may be overused. The medical societies provide the information, and lists treatments or procedures that might merit careful consideration by physicians and patients. The ABIM Foundation provides expert review of this information. Consumer Reports' role is to disseminate the information in such a way that it benefits the public.

One of Rasberry’s primary goals was to integrate health information into health articles. We measured the number of articles into which Choosing Wisely content was integrated, and the number of times those articles were accessed. Choosing Wisely content went into 58 articles and was requested and viewed 11 million times in a three month period, and since the content is stable, presumably it will perpetually be viewed as frequently in every future quarter. A new set of Choosing Wisely content (90 new topics) will be available to the community starting February 21, 2013 and Rasberry is helping Consumer Reports work with the community to include it in Wikipedia.

Measures of success

List the measures of success exactly as provided in the approved grant submission, and evaluate your project according to each measure listed there.

We will consider this project a success if we attract applicants with skills and interests relevant to continuing to build the Wikipedian in Residence model. Some of these skills, such as excellent written and verbal communication skills, will overlap well with the requirements for the Choosing Wisely project. Skills that are more specifically appropriate to the development of a future Wikipedian in Residence position include:
Ability to plan and conduct a meeting in which the primary objective is to understand a department's purpose and strategic objectives;

Ability to tell compelling stories in written form (blogs and case studies) based on one's personal experience;

Creative, outside-the-box thinking; ability to design projects that will meet both a department's goals and Wikipedia's goals; ability to communicate such proposed projects convincingly.

The writing elements (blogging and case study composition) will document the progress of the Choosing Wisely residency. The self-documentation of Wikipedians in Residence in the GLAM sector has been an important tool in building interest both within GLAMs and among Wikipedians in these sorts of projects; here is just one recent example of how a GLAM Wikipedian in Residence documented how an activity (applying the CC-BY-SA license to a photo) reflected and advanced trust between a museum and Wikipedians. By documenting a new type of residency, we expect this Wikipedian in Residence will similarly contribute to the opening of doors for further residencies within Consumer Reports and with similar public interest organizations.

The Wikipedian in Residence will not be alone in these activities, but will have the guidance of Consumer Reports and Wiki Strategies staff who have already begun these explorations. If we receive applications reflecting abilities and interests in these areas, we will consider this aspect of the residency recruitment a success. A more definitive and observable measure of success (but one that cannot be guaranteed) would be the emergence of at least one paid Wikipedian in Residence position, to be funded by Consumer Reports, by the end of these initial two residency periods.

Provide an overall assessment of how your project went according to these measures.

The measures of success outlined in the grant application were met. The CR Wikipedian-in-Residence position was posted by Consumer Reports in its online hiring system, and publicized to the Wikipedia community through emails and personal contacts. Three candidates with strong qualifications and extensive Wikipedia experience were interviewed for the position (most notably Lane Rasberry, the successful candidate). The successful candidate had the appropriate subject background, professional interests and mix of skills and experience to successfully engage in all activities anticipated by the job description and grant. Although we originally anticipated hiring two Wikipedians-in-Residence for a term of three months each, Lane was an excellent fit for the position and CR decided to extend his term for the entire six months.

Throughout the grant period, Rasberry made substantial headway in advancing the notion of Wikipedian in Residence style collaborations between Wikipedia and mission-aligned organizations like Consumer Reports, and the medical societies working in the Choosing Wisely initiative. Rasberry initiated, led and supported many productive discussions about opportunities to engage with Wikipedia in other content areas.

Rasberry also shared information about his work at CR with other Wikipedia user through Wikipedia forums, particularly WikiProject Medicine. He created pages such as these to facilitate interaction between contributors to health care articles: Health Article Review Project, Consumer Reports project page on Wikipedia, Choosing Wisely project page on Wikipedia.

The project was also promoted and commented on Twitter, for example, and included a couple of times in CR communications such as our email newsletter and website. We also issued a press release announcing his residency. Early results of the project were presented to health professionals at the Tufts University Medical School during their Summer Institute in July 2012 by Tara Montgomery, Director of Health Impact for CR, and Pete Forsyth of Wiki Strategies.

Rasberry presented to the Consumer Health Communications Summit hosted by Consumer Reports in May 2012, where he explained Wikipedia's health work to 60 leaders from consumer organizations such as AARP, SEIU, etc.

As a result of Rasberry's ongoing consultations with various departments at Consumer Reports, at the end of the six-month project, Consumer Reports elected to extend the Wikipedian in Residence program for an additional six months. Several promising areas for future collaboration and future possible residencies have been identified.

If you were to plan a similar project, would you measure it differently? If yes, please explain how.: NO.

Impact

This section ties this project to Wikimedia's broader goals, and shows what the project accomplished.

What impact did this project have on WMF's mission and the strategic goals? Please answer in 1 -2 paragraphs and include specific measures of impact such as the number of readers or editors reached by a particular project, or the number of articles edited or improved.

~~Stabilize infrastructure~~
Increase participation: The Wikipedian in Residence presented on Wikipedia to numerous experts within Consumer Reports, and among the partner organizations of Choosing Wisely. (see #Activities section) Some have already begun to contribute to Wikipedia, and all will be better able to evaluate future opportunities to do so. (20 CR staff in one-on-one trainings; dozens through presentations within CR and at partner conferences)
Improve quality: 58 English Wikipedia articles relating to medical tests and procedures were improved, and infrastructure for improvement of further articles was established.
~~Increase reach~~
Encourage innovation: This project overall has been an in-depth exploration of how an organization like Consumer Reports -- a non-profit entity dedicated to disseminating factual information -- can engage with Wikipedia in ways that are effective from its own perspective, and ethical and culturally compatible from the Wikipedia perspective. The Wikipedian in Residence has fostered an environment of innovation through all stages of the engagement, including both CR staff and Wikipedia volunteers.

Reporting and documentation of expenditures

This section describes the grant's use of funds

Documentation

Did you send documentation of all expenses paid with grant funds to grants at wikimedia dot org, according to the guidelines here? Answer "Yes" or "No".: YES

Expenses

Please list all project expenses in a table here, with descriptions and dates. Review the instructions here.

$1000/month for two Wikipedian in Residence positions [these were both held by the same person], each running for three months (consecutively), to supplement $3000/month supplied by Consumer Reports. Total grant amount was $6,000. The Wikipedian in Residence earned $4000/month for six months.

Total project budget (from your approved grant submission): $6,000

Total amount requested from WMF (from your approved grant submission, this total will be the same as the total project budget if the WMF grant is your only funding source): $6,000

Total amount spent on this project (this total should be the total calculated from the table above): $6,000

Total amount of WMF grant funds spent on this project (this total will be the same as the total amount spent if the WMF grant is your only funding source): $6,000

Are there additional sources of revenue that funded any part of this project? List them here.: Consumer Reports contributed .75 FTE ($18,000) of the Wikipedian-in-Residence position, $6,000 in Wikimedia Foundation funds were used as a salary allocation for .25 FTE for 6 months, at a rate of $1,000 per month.

Remaining funds

Are there any grant funds remaining?: NO

Please list the total amount (specify currency) remaining here. (This is the amount you did not use, or the amount you still have after completing your grant.)

$0