Recommendation API

The Wikimedia Recommendation API is an open-access HTTP API, currently under development, that will provide personalized recommendations for a variety of use-cases. The initial use-case for this API is ContentTranslation where it recommends articles to translate from one language to another by using traffic trends, activity logs, and other signals to identify gaps in coverage and to match those to editors based on interest. The API is intended to be flexible enough to serve as a basis for new features by third-party developers or researchers.

Contact us

The best way to contact us in the #wikimedia-research channel on Libera Chat IRC.

Consumers

The Translation Recommendation Type is being used by ContentTranslation as part of their "Suggestions" feature.

Type Endpoints

Type	RESTBase		Production		Cloud VPS		Experimental
	API	GapFinder	API	GapFinder	API	GapFinder	API	GapFinder
Translation
Missing Sections
Related Articles

Service overview

Recommendation API

the service that we’re trying to productionize

• Provides endpoints for all the Recommendation Types and the Recommendation Tool
  • These can live independently of each other

Recommendation Type

an endpoint of the Recommendation API that provides a type of recommendation

Translation

• Recommend articles for translation that exist in a source language wiki but are absent from a target language wiki
• Spec: https://recommend.wmflabs.org/types/translation/spec
• Ex query: /types/translation/v1/articles?source=en&target=de&seed=Apple
  • Returns articles that exist in enwiki but are missing from dewiki that are related to the article “Apple”
• Currently uses morelike search, Pageview API, and Wikidata queries to provide recommendations
  • Build list of candidate articles
    • Use morelike if a seed is provided, else most popular articles based on Pageview API
  • Apply filters to prune candidate articles
    • https://github.com/wikimedia/research-recommendation-api/blob/master/recommendation/api/types/translation/filters.py
• Currently can use the experimental Related Articles Recommendation Type for building the list of candidate articles in place of morelike

Related Articles

• experimental
• Waiting for results from study
  • https://phabricator.wikimedia.org/T142009
• Recommend articles that are related to a seed article in a given language
• Spec: https://recommend-related-articles.wmflabs.org/types/related_articles/spec
• Ex query: /types/related_articles/v1/articles?source=en&seed=Apple
  • Returns articles that are related to “Apple” that exist in enwiki
• Ex query: /types/related_articles/v1/items?seed=Q89
  • Returns wikidata items that are related to Q89
• Uses Wikidata queries
• Uses computed matrix file
  • Size: measured in Gigabytes
  • Computed offline approx. every month
    • Needs less than a day to build

Missing Sections

• experimental
• Recommend articles and sections to add to them
• Code: https://github.com/schana/recommendation-missing-sections
• Spec: https://recommend-experimental.wmflabs.org/types/missing_sections/spec
• Ex query: /types/missing_sections/v1/articles
  • Returns articles and sections that could be added to them
• Ex query: /types/missing_sections/v1/articles?seed=Apple
  • Returns articles related to "Apple" and sections that could be added to them

Recommendation Tool

the UI component that can exercise the API

• GapFinder
  • Tool that helps editors fill content gaps in Wikipedia
  • Currently uses the Translation Recommendation Type
    • Research:Increasing_article_coverage/Tool
  • Code lives in the main Recommendation API repo
• There is a different version in development to support different recommendation types
  • https://github.com/schana/recommendation-tool

Current service

• Lives in Cloud VPS
  • nginx/uwsgi
  • https://github.com/wikimedia/research-recommendation-api/blob/master/recommendation/data/labs_setup.sh
  • https://recommend-related-articles.wmflabs.org/
  • https://recommend.wmflabs.org/
• Python 3.4

Future service

• Based on ServiceTemplateNode
• https://github.com/wikimedia/mediawiki-services-recommendation-api
• Deployment task: phab:T165760
• Sequence Diagram

Our intentions for productization

To support the current use case of ContentTranslation, we want to productionize the Recommendation API providing only the Translation Recommendation Type endpoint at this time, without relying on any experimental functionality (that is, don’t use the Related Articles Recommendation Type in place of morelike). Without needing the Related Articles Recommendation Type, the Recommendation API will mostly consist of I/O bound calls to mw/pageview/wikidata endpoints. The Recommendation API should have a publicly accessible endpoint.

Beyond this initial use case, more Recommendation Types can be added to the Recommendation API as they’re vetted. These may have varying technical requirements and will likely require further collaboration before production infrastructure can support them. For example, the Related Article Recommendation Type needs a large computed matrix file.

See also

• repository on Gerrit
• workboard on Phabricator
• Research behind the article recommendation service
• Research project
• GapFinder documentation
• Blog post
• Cloud VPS project