Grants:IEG/Open Access Reader

This project is funded by an Individual Engagement Grant

status: selected

project:

Open Access Reader

project contact:

ed@wikimanialondon.org

participants:

join this idea!

grantees: Edward Saperia
volunteers: Katherine Bavage, Claudia Koltzenburg
advisors: Stuart Lawson, Simon Knight, Michelle Brook, Daniel Mietchen, Blue Rasberry (talk)

summary:

Open Access Reader is a project to systematically ensure that all significant open access research is cited in Wikipedia.

engagement target:

English Wikipedia

strategic priority:

Improving Quality, Increasing Participation

total amount requested:

6,550 USD

2014 round 1

Project idea

What is the problem you're trying to solve?

The 2007 research study What open access research can do for Wikipedia, identified a gap between available open publications and article content and proposed a model to improve this.

The Wikimedia movement has made considerable inroads in this broad area in the following ways:

• Elevating the coverage of academic publications use on the projects
• Raising awareness of open access publications more broadly WikiProject Open Access and WikiProject Open
• Defining the standard information collected about publications and the instance that relates to topic metadata Wikidata - Periodicals Task Force

Despite this gap between a wealth of resource and citations substantiating and improving WP article content persists – most recently discussed in a session at Wikimania 2013

What is your solution?

The Open Access Reader project ultimately aims to deliver a complete workflow from online open resource to editor, allowing editors to access highly relevant open research sources in the topic they are interested in developing in article space. For detail of the entire project, see the Open Access Reader Project Page

This grant proposal proposes scoping out and beginning to lay infrastructure for such a project, and other possible projects in the OA space:

1. Evaluating existing open access research aggregators
2. Evaluating existing open metrics for papers (e.g. citations)
3. Evaluating open research topic metadata
4. Evaluating existing content suggestion mechanisms for editors.

Project goals

By funding time for product management and a couple of software development sprints, this grant enables an exploration of the open access scholarship ~ wikimedia space. We intend to:

• Engage with members of the community who either historically or currently worked across Wikimedia projects in this area, including
  • the Wikidata periodicals task force and engineering team around past attempts at delivering customizable content based on topic (e.g. WikiFeed, SuggestBot)
  • Wikimedia or academic community members who have worked with programmatic manipulation of OA papers.
• Identify and evaluate as many open access paper aggregators with appropriate APIs as we can find. In general, try and lay groundwork and create useful infrastructure for any future projects that would like to work with OA resources, including:
  • Developing and delivering a specification for a filtering logic and programme that could screen available data and define notability of content. A suggested option has been how often a given paper is cited in other papers as an indicator of significance. There may be other filters that are relevant and consultation in both the academic and wikimedia communities would help feed into developing this specification.
  • Paper metadata is of variable quality. We'll attempt to create a filter to categorising research papers by topic, which can then supported by an on-wiki system to reference metadata labels. These can then be matched to appropriate WikiProjects and it is possible that Wikidata: Periodicals task force instance labels are already beginning to reproduce this function
• If there is budget remaining, try prototyping various tools using this infrastructure in concert with editors to aid their workflow, and if they prove useful, promote these tools both within and without wikimedia.

Project plan

Scope

Activities

Product manager

• Research, including identifying relevant internal (community) and external (academic and publishing) groups to engage in the four phases of the project
• Incorporating feedback generated into specifications for filter, metadata lookup labels and template design and implementation
• Identify and contract with software developer(s) who will have relevant expertise to work on this project now and in future phases
• Schedule and collaborate on two sprint sessions (see below)
• Develop and update specifications for each project phase based on feedback and sprint activities

Developers

• Design database for aggregation of open access sources
• OA Repository API integration.
• Web front-end, if specification calls for it.
• Integrates with mediawiki api, if specification calls for it.
• Develop template and integration of metadata labels to project (NB this could be something developed by volunteers if interest generated)

Budget

Total amount requested

$6,550 USD

Budget breakdown

Hourly rates based off WMF HR benchmarks as supplied by Siko in an email 25/05/14:

• 40 hours of developer time spread across 2 sprints budgeted for equivalent of $50/hr: 2,000 USD
• 182 hours product management for 7 hours a week over six months for equivalent of $25/hr: 4,550 USD

Intended impact

Target audience

• English Wikipedia contributors in WikiProjects where there is a pressing need to develop a density of citations in order to improve article quality.
• English Wikipedia contributors in topics where there has been a persistent lack of success increasing the number vital articles as opposed to featured or good.

Community engagement

• Open drafting of sprints schedule and specifications
• Linking these to appropriate Wikimedia project communities as identified in scoping phase for feedback
• Promoting the development of the tool to internal and external audiences throughout Wikimania 2014 & pre-event hackathons, particularly the Open Scholarship Hack Weekend.
• Work with chapter groups and thematic orgs (e.g. Wikimedia UK Technology Committee and Wiki Education Foundation

Fit with strategy

• Increase Participation: This project serves the movement's strategic goal to 'Increase Participation' because of the potential to leverage WikiProjects as a channel for editor engagement by providing them with fresh content, reactivating lapsed editors and supporting existing editors. We also believe that the OAR workflow may be a significant tool for recruiting new editors, e.g. in an undergraduate teaching context.
• Improve Quality: It is also significantly supportive of the strategic goal to 'Improve Quality' by virtue of providing better sourced content, with sources that can be routinely checked against newer research in a given topic area as a result of being open access.

Sustainability

• The project goal is to provide a prototype workflow that is delivering as envisioned. This is most manageable if after the first phase of development (Aggregator and API) the specifications for a filter, matching metadata to instance labels and WikiProject templates are initially focused on a small number of publications and corresponding WikiProjects. Delivering this successfully would act as proof of concept of the specifications developed, allow feedback to be incorporated and then a new specification developed based on new challenges faced in different topic areas.
• Currently this workflow mechanism is exclusively focused on delivering content to English language Wikipedia projects. However, once developed there would be scope to advise and collaborate with the international community if they wished to replicate the process for their in-country languages, wikis, and wiki projects.
• The further development of proactive notification or delivery of content to editors beyond availability on project pages or portals - perhaps by developing bots to cross reference user box interest and new content in a given metadata label, or subscription to a regular talk page delivery announcing new content.

Measures of success

• The engagement of at least 20 individuals from within related Open Access and Academic communities in notability filter specification design and feedback
• The engagement of at least 50 individuals from within the editing communities in metadata keyword matching specification design and feedback
• At least 50 articles show improvement with relevant content supported by the new citations

Participant(s)

• Project Lead: EdSaperia (talk)
• Grantwriter: Katherine Bavage
• Open Access consultants
  • Stuart Lawson
  • Simon Knight
  • Michelle Brook
  • Daniel Mietchen
  • Blue Rasberry (talk)
• Identified Developers
  • Tool Labs/Mediawiki familiarity: User:A930913
  • Web frontend: Kimi Lawrie

Discussion

Community Notification

UK Community mailing list

Endorsements

Do you think this project should be selected for an Individual Engagement Grant? Please add your name and rationale for endorsing this project in the list below. Other feedback, questions or concerns from community members are also highly valued, but please post them on the talk page of this proposal.

• Community member: add your name and rationale here.

1. See Open Access Reader Project Page for more endorsements.
2. Jodi.a.schneider (talk) 08:52, 1 April 2014 (UTC) Making open access content easier to find and cite would be beneficial, as the resulting references could be read and accessed by *all* readers.
3. This would help us a great deal. ----Rsrikanth05 (talk) 19:35, 22 April 2014 (UTC)
4. By allowing Wikipedians to find and city Open Access content easier, this project would strengthen the Open Access ecosystem. --Frank Schulenburg (talk) 21:21, 24 April 2014 (UTC)
5. Endorse Jane023 (talk) 08:34, 29 April 2014 (UTC)
6. Endorse --Aubrey (talk) 08:38, 16 October 2014 (UTC)

Committee Feedback

Scoring criteria (see the rubric for background)	Score 1=weak alignment 10=strong alignment
(A) Impact potential Does it fit with Wikimedia's strategic priorities? Does it have potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	7
(B) Innovation and learning Does it take an Innovative approach to solving a key problem? Is the potential impact greater than the risks? Can we measure success?	6.9
(C) Ability to execute Can the scope be accomplished in 6 months? How realistic/efficient is the budget? Do the participants have the necessary skills/experience?	5.6
(D) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support? Does it support diversity?	7.1
Comments from the committee: A well-advised project by trusted volunteers. The project leader has organizational experience (however is involved in another large project with Wikimania2014 until August). Open Access is a big theme of this Wikimania - in the course of my organisational duties, I can conveniently meet and talk to many of the leading people in the field, which I expect will be pretty useful in putting this project together. Not to mention many prominent community members! EdSaperia (talk) 12:26, 22 May 2014 (UTC) Will both promote open access content, while potentially improving the number of sourced articles on Wikipedia across projects. Could have a significant impact in improving quality, which is one of the strategic priorities. Could be scaled and adapted. Innovative approach in developing a ranking system devised by number of citations. The measures of success are brief but well-aimed. Project is trying to solve a traditional problem about citation, so there are low risks because the idea is well developed. Somewhat unclear on the technical implementation, which is not explained in detail. Project's success seems to be highly dependent on the software developer(s) who is to be determined later. While this appears to be a worthwhile project, it is rather lacking in specific details, with important decisions yet to be made and no timeline of completion of interim steps. If this proposal was held over to the next round, it could include more specific timeline, and it would be clearer what the product would be and how Wikipedia editors would interact with it. This is a product design & management project; Designing the best user experience (which includes user acquisition) is part of the project scope, and after the "paper significance ranking" infrastructure is built I expect to produce several prototypes that will guide the development of the tool. This may involve wikiprojects, user talk pages, article talk pages, standalone tools, webservices, social media, or other things not considered yet. As such, it's hard to predict ahead of time exactly what technical skills will be required. Would benefit from a use case scenario to look for before considering the specifications. For example, highly skilled editors might not need such a tool when looking for papers in the domain they are familiar with, since they can find open access papers without assistance. Highly skilled editors are a very precious resource; I'm trying to produce tools that allow less skilled editors to take over some of their workload - the same argument could be made of e.g. Visual Editor. Also, a skilled editor may be able to find open access papers relevant to a given domain, but that doesn't imply that all significant papers for all domains are cited. EdSaperia (talk) 12:26, 22 May 2014 (UTC) Concerned about the dependency on Wikiprojects, which tend to be fairly unresponsive across the board. There are very few editors who patrol those WP talk pages. As stated above, different channels will be experimented with; here's a talk giving an overview of some possible avenues: http://www.slideshare.net/generalising/community-communications-slides EdSaperia (talk) 12:26, 22 May 2014 (UTC) Hope that proposers plan to leverage existing resources such as CORE, DOAJ, and OAD and connect with The Wikipedia Library and Wikiproject Open Access. Certainly will be! Most of those are already mentioned in the project plan. EdSaperia (talk) 12:26, 22 May 2014 (UTC) The community has shown considerable interest in this project, and if this interest is maintained community engagement should be high. Since this is planned only for the English Wikipedia it won't be particularly useful to support diversity, although if successful it could be a model for other projects that will. The initial scope is only for English Wikipedia, localisation could be included in future phases (and consideration will be given to making it easy to integrate later). The budget is very significant. The hourly rate of $36.50/hr is high and if continued into future rounds would become a large investment. Funding a less expensive version would be attractive.

As discussed on the talk page, I am happy to accept any WMF SF benchmark rates for developers, but I don't want to risk the failure of the project because I didn't have the budget to hire suitably skilled contractors. EdSaperia (talk) 12:26, 22 May 2014 (UTC)

Thank you for submitting this proposal. The committee is now deliberating based on these scoring results, and WMF is proceeding with it's due-diligence. You are welcome to continue making updates to your proposal pages during this period. Funding decisions will be announced by the end of May. — ΛΧΣ²¹ 00:01, 13 May 2014 (UTC)