Toolhub/Roadmap
This roadmap is an attempt to prioritize and organize the work that will go into developing Toolhub. The roadmap is meant to reflect the development team's current understanding of how to best approach the problem of tool discovery. It also represents a vision of what Toolhub should be, based on an effort to synthesize the research and learn from it. This is a binding plan as of writing, but it is not meant to be permanent or inflexible. The plan can change as new situations arise and lessons are learned. As the plan changes, this page should be updated as well.
We invite you to get involved in this process by following our progress reports and to discuss the project on the talk page. If you suspect something is going in the wrong direction, it is better to speak up sooner rather than later.
Data model edit
- Status: Done
- Toolhub version: 1.0.0
The data model refers to the pieces of information used to describe tools. This includes the basic facts about the tool, such as its name and who created it, as well as information that helps people who are looking for it.
Our starting point will be the data model used by Hay's Tool Directory, a volunteer-developed directory service. This data model is a good start but it has some drawbacks. Keywords are freeform text, meaning that there is significant term duplication, with terms in English and other languages treated as separate concepts instead of the same (image vs. images vs. media vs. multimedia vs. fotografia). There is also no guidance as to what kind of information to provide, or how. This hinders tool discovery.
Tool records in Hay's directory are represented through JSON files provided by the tool developers. Each tool record contains a unique tool ID, tool name, description, URL, keywords, author, and a link to the repository where the code is hosted. These files, called toolinfo files, are placed somewhere on the web and then downloaded by the directory software to populate the directory. This approach assumes that records are not centralized. Despite this assumption, over 60% of tool descriptions are provided through two central sources: the Toolforge admin console and an on-wiki list meant to be community maintained. This suggests that even though decentralized tool metadata is valuable, in practice the data is significantly centralized. In fact, the creation of a wiki page to serve as an all-purpose toolinfo record suggests there is at least some demand for the ability to just create tool descriptions. Because of this, the data model will be split into two parts:
- Basic information, including the tool's name, authors, and license. This data can be submitted through a toolinfo file that is crawled by Toolhub, or the information can be submitted directly through Toolhub.
- Tool annotations that cannot be submitted through toolinfo files and can only be submitted or modified on Toolhub.
The Wikimedia Tool JSON schema will include basic information expected of each tool. For reference, the current data model used by Hay's directory has been codified as version 1.0.0 of the Wikimedia Tool JSON schema. In May 2018 a draft of version 1.1.0 was published for public comment. Version 1.1.0 will expand the current data model while being completely backwards compatible, creating a smooth migration path. The final version will be published by 30 June 2018. There may also be work on a backwards-incompatible version 2.0.0, depending on project needs. Note that although this data is described using a JSON file, it will be possible to instead submit it directly to Toolhub, without the need to create a JSON file.
Once Toolhub has ingested a tool record, either through crawling a toolinfo file or by receiving data directly, it will be possible to add additional tool attributes. These attributes will make use of a controlled vocabulary, reducing the risk of keyword duplication and making it possible to translate terms into different languages. Hosting the data in Toolhub itself ensures that the community will always be able to edit it, even if the tool description is otherwise contained in an un-editable toolinfo file. It will also help keep the toolinfo files small and abstract away implementation details that would make the files difficult to edit by hand.
(Note that while the data model may be in two parts, the UI will "hide" this distinction for ease of use.)
Initial API and UI; toolinfo crawler edit
- Status: Done
- Toolhub version: 1.0.0
Central to Toolhub will be the API, on top of which the user interface will be built. Aside from it being good practice to separate data transactions from data presentation, this also reflects our priority of gathering, organizing, and distributing high quality data. Part of realizing this vision is being able to submit and request information about tools in more places than just the official UI. For instance, once the API is online, a Wikidata gadget could be written to create an automatically updated list of Wikidata gadgets, allowing Wikidata users to learn about helpful tools without leaving the site. One hundred percent of Toolhub business should be possible through the API alone.
The API and the user interface will be developed in tandem; as API methods are implemented, so will parts of the user interface. The top priority is to achieve feature parity with Hay's directory. This includes:
- Submitting a URL of a toolinfo.json file to be crawled
- Retrieving a list of URLs of toolinfo.json files that are regularly crawled
- Retrieving a list of all tools
- Retrieving records about an individual tool
Direct tool registration edit
- Status: Done
- Toolhub version: 1.0.0
Once the submission of annotations is supported, the next step is to allow tools to be registered directly through Toolhub (as opposed to crawled over the web), including API methods for registering tools, editing tool records (other than those not submitted directly to Toolhub), and deleting tool records.
Search edit
- Status: Done
- Toolhub version: 1.0.0
This milestone will focus on implementing ElasticSearch support in Toolhub, allowing tools to be searched based on certain criteria. This includes exposing a search API and adding a search box to the UI.
Content moderation and patrolling edit
- Status: Done
- Toolhub version: 1.0.0
Toolhub will not allow anonymous edits, all edits to Toolhub content will be associated with Wikimedia users. We hope that this greatly reduces the need for content moderation. We do still however need to provide tools for removing or suppressing inappropriate content which may be introduced via toolinfo records, their associated annotations, and lists. The decision record on moderation describes the minimum functionality that will be completed via this milestone.
Lists edit
- Status: Done
- Toolhub version: 1.1.0
Curating lists of tools used by particular communities or groups or for particular tasks, as well as personal lists that serve as bookmarks, is expected to be a widely used feature of Toolhub. Creating, retrieving, and editing lists will be possible via both the API and the UI.
For the 1.0.0 release, we will had the ability to create and edit lists via the UI, but did not yet have an interface for viewing history and doing patrolling actions. This functionality was available via the Toolhub API.
Annotations edit
- Status: Done
- Toolhub version: 1.1.0 (partial)
Annotations are tool metadata that extend the basic metadata provided during initial tool registration. They're additional pieces of information that aid in tool discovery. Because it is important to be able to make edits to annotations, they cannot be submitted through toolinfo files; they must be managed directly through the Toolhub API or UI. (However, the UI will abstract away this distinction.) This milestone will focus on the addition of API methods and corresponding UIs for editing tool annotations.
Informed User Decisions edit
Today users only have word-of-mouth to rely on as to the "quality" of a tool. It is very difficult to determine the level of support and maintenance a tool has. Will it keep up with changes that happen on-wiki (policies, standards)? Will it keep up with infrastructure changes (API updates, hosting platform changes, wiki hosting changes)? Does it even have active maintainers? Is there a "best" way to get help? Toolhub will provide more insights in these questions, and increasingly help users assess the quality of tools on multiple levels.
Research on this topic is ongoing, but implementation will likely include features similar to:
- Endorsements: a simple, lightweight way of expressing approval of a tool
- Metrics about tools, including usage
- Recommendations for tools to use based on criteria such as endorsements and metrics
Future milestones edit
There are additional ideas for making Toolhub useful. At this point they are just ideas, with no commitment to deliver them. We are interested in knowing what you think of them, and we also invite you to suggest other ideas.
- Requests for new tools to be developed
- Automatically generated tool records based on web crawling activities, with workflows to integrate "crawled" tools into the directory proper