Community Wishlist Survey 2021/Warn when linking to disambiguation pages

Disambiguation Wish Team
Disambiguation Wish Team
	Warn when linking to disambiguation pages
Group:	Community Tech
Team members:	Nicolas Ayoub, Harumi Monroy, Leon Ziemba, Emanuele Leoni, Sam Wilson, Szymon Grabarczuk
Lead:	Natalia Rodriguez (product owner)
Updates:	Status Updates

Hello all, and thanks for coming to read more details about Warn when linking to disambiguation pages, the #2 wish in the Community Wishlist Survey 2021. This article will outline our approach to building a solution of this wish and ask for your feedback and insight so that we may deliver the best possible experience.

This page documents a project the Wikimedia Foundation's Community Tech team has worked on or declined in the past. Technical work on this project is complete.

We invite you to join the discussion on the talk page.

Wish Objective Summary: Reduce undesirable links to disambiguation pages on the wiki(s)

Original Wish

Background & Problem Space

For those unfamiliar with disambiguation pages on the encyclopedias, please note that disambiguation pages are articles associated with a generalized term that link to more specific articles on the topic. Our current user interface makes it difficult for editors to differentiate when linking to a disambiguation page or a specific article. For the purposes of this doc, link to disambiguation pages that are meant to be links to specific articles are unwanted disambiguation links.

At a high level, we can summarize the problem of unwanted disambiguation links the following way:

How do we provide editors the ability to link relevant & specific knowledge when creating an article?

Let’s make the problem space concrete with a topic an editor may want to link to when creating an article: For example, the word "Mercury" can refer to several different things, including an element, a planet, and a Roman god. Since only one Wikipedia page can have the generic name "Mercury", unambiguous article titles are used for each of these topics: Mercury (element), Mercury (planet), Mercury (mythology). There must then be a way to direct the reader to the correct specific article when the ambiguous word "Mercury" is referenced by linking, browsing or searching; this is what is known as disambiguation. In this case it is achieved using Mercury as the title of a disambiguation page.

If an editor wanted to link to Mercury while writing an article on Mars, they’d be faced with the disambiguation link as their top recommendation in VisualEditor when linking to related knowledge:

From a product standpoint, allowing editors to link to specific content (non-disambiguated content) would allow us to:

Improve the reader experience by increasing relevance in linked knowledge
Generate a more accurate content map of articles on the wikis

Within this broader problem space, we can explore the following set of user problems:

Example User Problems
Editor ("it's hard for me to...")	Reader ("I want...")
... to differentiate a disambiguation link from a regular link	...to read about the associated and specific knowledge inside an article
...to understand the most relevant and specific article to cite the knowledge I want to link to in my article	… to navigate to the specific page on a topic linked to from another article

Proposed Solutions

The following are a set of design requirements that we hypothesize will mitigate unwanted disambiguation links. We will explore each before deciding on which makes the most sense to prioritize. We'd love your input.

As an Editor using the link toolbar to search for knowledge to reference in my article, I can:

See specific articles as the first search results
See a warning before publishing if I have introduced a disambiguation link in my article
Visibly differentiate disambiguation links as a different type of content than articles that come up on my suggestions

Explicit Scope for our Solution: We will be limiting the scope of our changes to the link helper that appears when you try to use a link button in the toolbar. Solutioning for wikitext links would be too complex given the team bandwidth and other wishes to fulfill. We also understood from the proposal and its comments, that a script already exists that provides a warning to prevent one from adding links to the encyclopedia. We understand the issue of adding unwanted disambiguation link to be a problem for new editors face, a group that we know to use VisualEditor and not wikitext.

Open Questions: We want to hear from you!

The solutions above are proposed and in early stages, we'd love to hear your feedback on the talk page. Your insight can help us understand other approaches, risks, and potential solutions.

Some questions on our mind as we progress through design research:

As we try to move away from warning editors AFTER they've made a mistake, and shift towards a pattern that prevents them from doing so, what are your thoughts on still warning editors if they add a disambiguated term?
Are there any additional use cases to deepen our understanding of the problem space? Could there be downstream impact if we move the disambiguated terms to be the last result rather than the first?
What are some circumstances in which a user would want to link to a disambiguated page from one article to the next?
What other feedback do you have about our proposed solutions?

Data Investigations

We are currently working on answering the following questions which will help deepen our understanding of the problem:

How many disambiguated links are being generated per day on the project(s)?
Which editor UI is being used to introduce most of the disambiguated links in the project(s)?
What is the breakdown of total users for each of these editors https://www.mediawiki.org/wiki/Editor?
- Quick preview from turnilo for the last 30 days: out of 30m edits, 1.5m are from VE, 26.1m from "other". This might include API edits. Bots are excluded. Going by change tags: https://quarry.wmflabs.org/query/56216 (22% VisualEditor over past three days on English Wikipedia)

Why and how did we accept this wish?

This wish scored high in our prioritization process for 2021 because it was very popular in terms of votes, impactful in terms of users, and had a relatively low complexity estimate. Please read about our full process here.

Status Updates

Jan 12, 2022: Turning on the changes for all Wikis

Hello everyone,

Since our last update, we've been able to build and release our last proposed solution to a group of wikis that agreed to be early adopters of the feature. This included releasing the change to ensure editors could see a warning before publishing if they introduced a disambiguation link in a page. The changes applied to editors using the 2010 wikitext editor.

We worked with the following pilot wikis to release the changes:

huwiki

cawiki
plwiki
viwiki
fawik
fiwiki

The changes went live for the pilot wikis in November, and have been live and active for over the duration of all of December. We've monitored the changes for feedback and bugs and are ready to turn on for the rest of the wikis.

Thanks to everyone who provided feedback about the usability of this new feature by being part of our early adopter wikis, and thanks to all folks who tested the new functionality and gave us feedback during Talk to Us Hours, inside the Talk Page, or in Phabricator. We're looking forward to seeing the changes go live for the all of the wikis! This concludes the work on this wish.

Nov 8, 2021: User Testing Findings

Hello everyone,

Many thanks for your great support and feedback on the proposed user experience and designs for this wish. This update is regarding the usability testing, which was conducted in parallel with community members on the talk page, as well as the first “Talk to Us hours” video call last September. Thanks again for your valuable inputs!

The goal of this round of usability tests was to determine if our proposed designs were successful in warning users about the potential risks of adding a link to disambiguated page in the 2010 wikitext editor without inhibiting users from continuing to edit. We conducted the test on usertesting.com with 5 editors total.

Here are the insights we gathered from the tests:

2/3 of users successfully noticed the new notification warning them about the disambiguation link they added. Two users were not able to see the notifications due to complications with the testing interface and instructions.
All users who saw the notification found it helpful and easy to understand. When users noticed the disambiguation notification and understood its purpose, most of them stated that it was useful without being disruptive to their edit flow.

Acknowledgments: This test was conducted on beta-wiki with a limited content pages. Before we could push some recent user experience and engineering improvements for this wish, including the new link selector widget for the Wikitext editor. We are hoping that these changes will mitigate some of the difficulties encountered by the users during this usability test.

Not all of these editors from this panel are experienced users. Although this wish is intended to be helpful for every user – we assume that less experienced editors will tend to use the VisualEditor over the wikitext editor, thus making this new feature less relevant for them.

We are grateful for your feedback on this talk page and the Talk to Us for our team to get a better understanding of your needs as experienced contributors. Thanks so much for your feedback on the talk page!

August 18, 2021: Design Feedback

Progress with the design

Hello everyone, and thanks to those of you who have been active on our talk page and tested the proof of concept script for displaying a warning. We have moved forward with designing for the last two requirements in our proposed solutions and wanted to share them with you so that we could hear your input!

As an Editor using the link toolbar to search for knowledge to reference in my article, I can:

See a warning before publishing if I have introduced a disambiguation link in my article
Visibly differentiate disambiguation links as a different type of content than articles that come up on my suggestions

For the first solution above, we are proposing displaying this yellow box warning when wikitext users add a link:

For the second solution above, we are proposing changing the copy to better describe what a disambiguation page means in the wikitext link button and in the visual editor link pop-up:

Open questions

We'd love to hear your thoughts on our proposed designs. We'd especially love feedback on:

Making the copy accessible and making it easier to glean what a disambiguation page is
Making sure the warning for wikitext editors feels preventative without coming off as discouraging-- editors should still be able to publish even if they have introduced faulty links

We're looking forward to hearing your thoughts on our proposed designs!

August 2, 2021: The first solution

The first part is about the following improvement:

As an Editor using the link toolbar to search for knowledge to reference in my article, I can see specific articles as the first search results.

This effects the link search function in VisualEditor and the 2017 wikitext editor.

The change should go live this week. In the meantime, if you would like to preview the changes, you can try it out on the beta cluster. For instance, when adding a link and you type out the term "New York", New York City and New York (state) appear above the disambiguation page New York in the search results.

We are looking forward to hearing your feedback!

After this change is released, we will be monitoring the numbers inside The Daily Disambig to see if it has any impact on the number of unwanted dab links being added (thanks for pointing us in the direction of this page).

July 23, 2021: The first steps

We have begun design planning on this wish. Our designer, NAyoub is working on setting up some user tests where we ask new editors to link from the Jupiter article to the Mars article and see if they fall for the disambiguated page since it's the first result. We will take notes and write up next steps based on the results. Our engineers are also investigating the feasibility of moving search results for disambiguated terms to the bottom of the results. Read more