Community Wishlist Survey/Historical Metrics 2023

Future of the Community Wishlist Survey

• Introduction
• Redesign Conversations
• Potential Modifications

Contact

Krishna Chaitanya Velaga

Product Analytics, Wikimedia Foundation

Duration:  2023-08 – 2023-11

---

Research:Projects

This page documents a completed research project.

Introduction

The Community Wishlist Survey (hereinafter CWS) is an annual survey that allows contributors to the Wikimedia projects to propose and vote for tools and platform improvements. The survey had eight iterations since its inception in 2015, the last one being in 2023. The Community Tech team usually handles the technical development of the selected wishes, along with running the survey itself. In some cases, wishes have also been completed by volunteers or staff from other teams with little to no input from the Community Tech team.

Once the process begins, community members make proposals in various categories of technical wishes (the categories themselves have evolved over the years). Some of the categories include bots & gadgets, editing, multimedia & commons, moderation tools etc. The proposal phase is followed by a voting process, where community members again vote. Until 2020, the Community Tech team used to commit to the top 10 wishes (by the number of votes), however, since 2021 the Prioritization Framework has been used to select the wishes to be worked upon.

---

Before the next iteration of the survey, the Community Tech is working on analysing the feedback from previous years to understand how this process could be improved to keep up with the growing needs of the movement, referred to as the Future of the Wishlist. The historical metrics analysis is intended to support this process by surfacing historical trends of survey. The majority of the report focuses on community participation i.e. who have been participating, where the users come from (home wiki), their experience level etc.

Definitions & Notes

• proposal refers to a technical wish proposed by the community.
• proposer refers to the user who proposed the wish.
• discussant refers to a user who takes part in discussion of a proposal.
• voter refers to a user who takes part in voting of a proposal.
• home-wiki refers to the wiki where a user had the highest number of edits during the two preceding years of a survey.
• newcomer refers to users who participated for the first time in CWS, but not a new contributor to Wikimedia projects

---

Notes
The goal of the analysis and this report is to understand how CWS evolved and what has happened historically. This is not an evaluation report, but rather an exploratory data report. Although there was a survey conducted in 2018, the team changed the naming approach to state the year during which the teams intends to work on the wishes instead of the year in which the survey is held, which created a gap. In several metrics and graphs, you might see a huge difference for the year 2020, for example, drop in number of participants, this was due to the team's special focus on non-Wikipedia content projects for that year. The traditional structure was resumed from 2021. All the metrics, unless otherwise mentioned, are about the accepted proposals only. Starting with 2017, only selected proposals go into the voting phase, and others get rejected for various reasons, discussed later in the report. A combination of data available in MediaWiki databases, results from parser API and web scraping of CWS pages was used to gather the required data. For accepted proposals, data gathering failed for 1% of the proposals, and when rejected proposals are included, the failure rate is at 3%. However, as the goal is to understand broad trends, this won't affect any of the metrics. For consistency and easier comparison during analysis, some categories were were either merged or renamed. Please refer to the footnote for information about them.

Report

Proposals and Categories Over Years

Fig 1: Proposals by Year

The number of proposals submitted consistently increased by ~24% (median) from 2015 to 2022, with the exception of 2020. This was followed by sharp decline in 2023, by ~40% compared to 2022.

Accepted proposals followed the same trend as submitted ones, since the practice was started in 2017, on average ~35% of the proposals were rejected. The three most common reasons for wishes being rejected from entering the voting phase:

• Too big, broad or out of scope for Community Tech team
• Suggests a social/cultural change rather than a technical feature
• Insufficient information provided or idea not explained well

Categories

Editing, Multimedia and Commons, Wikidata, Moderation tools, and Navigation tools, are the top categories (in descending order) by the number of proposals, collectively accounting for 55% of the proposals. Followed by Watchlists, Mobile apps, Bots / gadgets, Reading, and Wikisource, and along with the top five categories, they account for 80% of the proposals. In the year 2020, by number of proposals, Wikisource was at the top accounting for 39% of the proposals, collectively with Wiktionary and Wikiversity, account for 82% of the proposals.

Overall Participation

Fig 2: Overall Participation by Year & Roles

There has been consistent increase in user participation at a rate 22% annually from 2015 to 2021, excluding 2020. During these years, while voters and discussants participation grew by 25% annually, number of people proposing wishes only grew by 15%.

Post-2021, the participation has been declining at 10% annually until 2023. When analysed by roles, number of people participating in discussing the proposals have the highest decline with 25% annually. However, the data is insufficient to predict the future trend.

Despite the decline, the participation levels of 2022 and 2023 have reached similar to that of 2019. When the participation data is compared with editors and content key product metrics, during the months of November-December 2020 (when the 2021 survey was conducted), Wikimedia projects had the highest number of active editors (~93,000), compared to the same period for 2019 and 2022-2023 where the number of active editors on average was ~88,000. As the survey gets promoted using CentraNotice banners, and number of editors on our projects has an impact on the participation.

Roles

On average, of all participants

• proposers account for 7-9%
• discussants account for 20-25%
• voters account for 70-75%

Roles across Categories

Top 10 categories of participation by roles

in descending order; coloured cell indicates notable change in rank of a category across roles

Role	Category	Role	Category	Role	Category
proposers	Editing	discussants	Editing	voters	Editing
	Multimedia and Commmons		Multimedia and Commmons		Moderation tools
	Wikidata		Moderation tools		Multimedia and Commmons
	Moderation tools		Wikidata		Bots and gadgets
	Navigation tools		Reading		Reading
	Watchlists		Bots and gadgets		Navigation tools
	Mobile and apps		Navigation tools		Wikidata
	Reading		Watchlists		Citations
	Bots and gadgets		Mobile and apps		Mobile and apps
	Citations		Citations		Anti-harassment
Across all roles, the top 10 categories collectively account for 80% of the participation.

The top categories for proposers are similar to the top categories by the number of proposals received. While a higher number of proposals and proposers can likely indicate the need for more features related to a category, the top categories by voting can likely indicate higher interest/need for the implementation of features related to a category.

Participation from Wikimedia projects

Number of uniques wikis, by role

Role	# Wikis
Proposers	86
Discussants	126
Voters	227

Home wiki

Across all roles, the highest number of participants has been from English Wikipedia, accounting for 30% of all participants. This was followed by Wikimedia Commons, Wikidata, German Wikipedia and French Wikipedia for proposers and voters. For discussants, more participants came from Meta-Wiki instead of French Wikipedia. Together, the top five wikis account for 60% of participation across all roles.

Across all roles, contributors from Wikimedia Commons and Wikidata had the highest participation in proposals related to Multimedia & Commons and Wikidata respectively. Whereas, contributors from English, German, French, Spanish and Russian Wikipedias, had the highest participation in Editing related proposals. Interestingly, contributors from the Chinese Wikipedia have the highest participation in Moderation Tools related proposals.

From wikis with the highest participation (top 15), the top five categories account for 50% of all participation from those respective wikis.

By size rank

Across all years, with an exception to 2020, the top five large wikis account for 45% of participation, and the next five account for 22% of participation. In 2020, 66% of the participation was from outside of top ten large wikis. Although Spanish and Japanese Wikipedias are among the top five large wikis, they are not part of the top five wikis by participation. Similarly, Italian and Portuguese Wikipedias are among the top ten large wikis, but they are not part of the top ten wikis by participation.

Project groups

Fig 3: Participation Trends by Project Group, by Year

On average across all roles, Wikipedia, Wikimedia Commons, and Wikidata, account for 85% of the participation, with Wikipedia alone accounting for about 65%.

Wikipedia, Wikimedia Commons and Wikidata had similar participation trends from 2015 to 2019, growing at 30% annually. Similar to the overall participation trend, there was highest participation in 2021, and followed by a decline until 2023, reaching participation levels similar to that of 2019.

Compared to other project groups, Wikisource had the highest growth rate annually, at 50% until 2020. Unlike the overall participation trend, participation from Wikisource declined in 2021. Although there was a slight increase in 2022, it was followed by a sharp decline in 2023.

Wiktionary had consistent increase in participation until 2020, followed by a decline until 2022, and a slight increase in 2023.

Participation by Experience Levels

Editing experience

edit buckets used	0-99	100-999	1000-4999	5000+

Home wiki

Fig 4: Participation Trends by Edit Bucket on Home wiki, by Year

Across all roles, users with higher editing experience tend to participate more the in survey. 57% of the participation had been from users with 5000+ edits, followed by 1000-4999 and 100-999 edit buckets, accounting for 20% each.

Proposers tend to be more experienced than other participants; on average, across years, 85% of the proposers of belong to the 5000+ edit bucket. The 2020's survey is a notable exception to this trend, which saw increased participation from 100-999 & 1000-4999 edit bucket groups.

During the initial three years, participation from the 5000+ edit bucket group accounted for 75% of the participation; however, in the recent years (2021-2023), there has been a notable shift in the trend. During the last three years, 5000+ edit bucket group accounted for 60%. While the participation from the 1000-4999 edit bucket group remained the same, there has been a substantial increase in participation, from 5% during the initial years to 20% in the last three years, from the 100-999 edit bucket group.

However, this trend is largely influenced by changes in participation among voters. Among proposers and discussants, the 5000+ edit bucket group has accounted for a majority of the participation with 83% and 77% respectively.

Edit buckets and Categories

Across all roles, with a few exceptions, users belonging to the 0-99 group have the highest participation in Reading related wishes, whereas the 100-999 & 1000-4999 groups have the highest participation in Editing related wishes. Among users with 5000+ edits, Editing, Multimedia & Commons and Moderation tools, are the most popular categories.

Meta-Wiki

Most of the users who participated in the survey were not active participants on Meta-Wiki. 70% of the participants belong to the 0-99 edit bucket, 20% to the 100-999 edit bucket, and 5000+ edit bucket group only accounts for 5% of the participation. When considered only proposers and discussants, 50% belong to the 0-99 edit bucket and 30% to the 100-999 bucket.

User rights

categorisation of users rights

category	rights
advanced	abusefilter, sysop, interface-admin, checkuser, oversight, botadmin
intermediate	having at least one user right, but don't fall into either of other two categories
none	extendedconfirmed or ipblock-exempt or none

Fig 5: Participation Trends by User Rights, by Year

Overall, 28% of participants possessed advanced user rights, while 43% possessed at least one intermediate level user right (not including advanced rights). When observed by role, 78% of the proposers have at least one user right (including advanced rights).

Over the years, the trend is similar to that which was observed with edit buckets, where users from lower edit buckets have started engaging more in the process. In case of user rights, during the initial years, users with no rights accounted for only 18% of participation, while during the last three years, they accounted for 41% of the participation.

User Rights and Categories

Among proposers with advanced user rights, Moderation Tools has been the most popular category. Across all roles, for users with at least one user right, Editing and Multimedia & Commons have been the most popular categories. While Editing related wishes have also seen high engagement from participants with no specialized user rights, Reading has also been a equally popular category among voters.

User age

user age has been calculated as the difference between registration timestamp and the month a survey was conducted

Fig 6: Average User Age by, by Year

The average User age of participants has been increasing over the years, starting from 5 years in 2015 to 9 years in 2023. As seen Fig 6, although the average user age varied by role, all of them follow an upward trend. However, this does not necessarily imply that more veterans are participating in the survey than newcomers, especially when comparing with the trends of edit buckets and user rights.

As explained in the later sections, the survey has substantial percentages of both first-time and returning participants. For example, when a user from 2015 with user age of 5 years participated in 2023, their user age would be considered as 13 years. As the age of the Wikimedia movement increases with each year, it is likely that the average user age of participants will increase as well.

User age and Categories: Among all categories and role intersections, proposers participating in the Anti-harassment category have the highest account age, with 11 years on average.

Miscellaneous

Newcomer Engagement

Note: 2015 and 2016 were excluded from the analysis of newcomer engagement.

Fig 7: Distribution of returning and new participants, by role and year

On average, 55% of the participants were newcomers (i.e. participating in the survey in any role for the first time), the remaining 45% were returning participants. However, when observed by only proposers and discussants, 60% of them have been returning participants. With an exception for 2020, English Wikipedia, Wikimedia Commons, and German Wikipedia are the top Wikimedia projects where newcomers come from, accounting for 70% of all newcomers on average.

With an exception for 2020, since 2016, returning participants had the highest participation in wishes related to Moderation Tools, whereas for newcomers, it was a mix of Moderation Tools and Editing related proposals.

Cross-Proposal Engagement

Average Number of Proposals a Participant Engages with, by Role Role # Proposals Proposer 1 Discussant 2 Voter 6

Notably, Wikisource and Wikidata were two categories where each participant submitted two proposals on average (irrespective of 2020), whereas for all other categories it was one per participant.

YoY engagement

2015 2016 2017 2019 2020 2021 2022 2015 100% - - - - - - 2016 26% 100% - - - - - 2017 21% 30% 100% - - - - 2019 17% 24% 33% 100% - - - 2020 12% 11% 17% 18% 100% - - 2021 13% 16% 21% 26% 14% 100% - 2022 11% 13% 17% 19% 14% 26% 100% 2023 9% 11% 14% 15% 11% 20% 24%

Participants from recent iterations of the survey tend participate to more in a given year, rather than from older iterations. For example, in 2023, 9% of participants also participated in 2015, where 24% participants in 2022. Note: The percentages are not mutually exclusive within a given year. For example, a user who participated in both 2015 and 2022, would be counted at both the times.

Proposals Re-submission

Several approaches were considered to find the rate of re-submission of proposals, including exact title match, proposal title text similarity, and associated Phabricator tickets. Except for analysis based on exact title match, the other two approaches are unreliable and prone to a high error rate, and they are therefore not presented in the report.

Category	Proposal	Years
Wiktionary	Display definitions from Wikisource dictionaries	4
Wikisource	XTools Edit Counter for Wikisource	4
Wiktionary	Custom list for language learner	3
Communication_tools	Notify users when their revision has been approved or rejected	3
Wiktionary	Insert attestation using Wikisource as a corpus	3
Wikidata	Expand automatic edit summaries	3
Wikidata	Display reference in edit summary when a reference is added	3
Wikisource	ProofreadPage extension in alternate namespaces	3
Wikisource	Improve workflow for uploading books to Wikisource	3
Wikisource	Improve export of electronic books	3
Wikisource	Offer PDF export of original pagination of entire books	3
Wiktionary	Context-dependent sort key	3
Miscellaneous	Provide a tool to efficiently analyze the usage of a template	3
Multimedia_and_Commons	Support 360 photo viewing	3

Selected/won proposals

Selected/won proposals are proposals which have made it to the top based on voting, until 2020, and from 2021, the proposals which have been selected by the Community Tech based on the prioritization framework, or taken up by other WMF teams.

The trends of participation at selected/won proposals, based on home wiki and project group follow the overall participation trend, with no significant differences. However, when observed by edit bucket on home wiki, there has been increased participation from more experienced editors (i.e. higher edit bucket) at selected/won proposals. Notably, 87% of all selected/winning proposals were created by editors with 5000+ edits, collectively with editors having 1000+ edits, they account for 99%.

Implementation

Until 2020, from the top 10 proposals by number of votes received each year, 67% (30) have been completed, 27% (12) were declined, and 4% (2) were partially completed. Among the proposals have been successfully completed, 83% were developed by the Community Tech team, and 13% by other WMF teams, and the rest by volunteers.

From 2021 to 2023, where the prioritization framework has been used, 10 wishes were completed by the Community Tech team, 8 were partially completed, and one was declined. During the same period, 30 wishes were completed by other WMF teams, and 7 were partially completed. In addition, 15 wishes were completed by volunteers developers.

---

The list includes top 10 proposals until 2021. From 2022, it includes all proposals for which status information is available on the results page of the respective year (as 20 November 2023).

Year	Proposal	Status	Developed by	Link
2015	Migrate dead external links to archives	Completed	Community Tech	link
2015	Improved diff comparisons	Completed	Community Tech	link
2015	Central repository for gadgets, templates and Lua modules	Declined	misc WMF team	link
2015	Cross-wiki watchlist	Declined	None	link
2015	Numerical sorting in categories	Completed	Community Tech	link
2015	Pageview Stats tool	Completed	Community Tech	link
2015	Global cross-wiki user talk page	Declined	None	link
2015	Improve the plagiarism detection bot	Completed	Community Tech	link
2015	Add a user watchlist	Declined	None	link
2015	Allow categories in Commons in all languages	Completed	misc WMF team	link
2016	Global gadgets	Declined	misc WMF team	link
2016	Edit summary length for non-Latin languages	Completed	misc WMF team	link
2016	Section heading URLs for non-Latin languages	Completed	Community Tech	link
2016	Global preferences	Completed	Community Tech	link
2016	Rewrite XTools	Completed	Community Tech	link
2016	Wikitext editor syntax highlighting	Completed	Community Tech	link
2016	Warning on unsuccessful login attempts	Completed	Community Tech	link
2016	Automatic archive for new external links	Declined	None	link
2016	Fix Mr.Z-bot's popular pages report	Completed	Community Tech	link
2016	User rights expiration	Completed	Volunteer developers	link
2017	Kartographer improvements	Completed	misc WMF team	link
2017	Ping users from the edit summary	Completed	Community Tech	link
2017	Development of the programs and events dashboard	Completed	Community Tech	link
2017	Blame tool	Completed	Community Tech	link
2017	Template wizard	Completed	Community Tech	link
2017	Deploy Article Alerts to other languages	Declined	Community Tech	link
2017	Auto-save feature in Visual Editor and WikiText Editor	Completed	misc WMF team	link
2017	Allow 'thanks' notification for a log entry	Completed	Community Tech	link
2017	SVG-Translate	Completed	Community Tech	link
2017	Commons deletion notification bot	Completed	Community Tech	link
2019	Page Curation and New Pages Feed improvements	Completed	Community Tech	link
2019	Night mode	Declined	None	link
2019	Add an option to require email address and username to reset password	Completed	Community Tech	link
2019	Improve export of electronic books	Completed	Community Tech	link
2019	VisualEditor: Allow references to be named	Declined	None	link
2019	Put mw.toolbar back	Declined	None	link
2019	Watchlist item expiration	Completed	Community Tech	link
2019	Article reminders	Declined	None	link
2019	Include section name in the diff	Partially completed	Community Tech	link
2019	2FA available for all concerned editors	Declined	None	link
2020	Improve export of electronic books	Completed	Community Tech	link
2020	New OCR tool	Completed	Community Tech	link
2020	Migrate Wikisource specific edit tools from gadgets to Wikisource extension	Completed	Community Tech	link
2020	Inter-language link support via Wikidata	Partially completed	Volunteer developers	link
2020	Insert attestation using Wikisource as a corpus	Not completed	Community Tech	link
2020	Ebook Export Improvement	Completed	Community Tech	link
2020	Migrate Wikisource specific edit tools from gadgets to Wikisource extension	Completed	Community Tech	link
2020	Insert attestation using Wikisource as a corpus	Not completed	Community Tech	link
2021	Warn when linking to disambiguation pages	Completed	Community Tech	link
2021	Copy and paste from diffs	Completed	Community Tech	link
2021	Realtime Preview	Completed	Community Tech	link
2021	(Un)delete associated talk page	Completed	Community Tech	link
2021	Bibliographic bot for Wikidata	Declined	Community Tech	link
2021	Allow global user rights to expire automatically	Completed	Volunteer developers	link
2021	Global events calendar	Completed	misc WMF team	link
2021	Link Wikipedia redirects to Wikidata items	Completed	misc WMF team	link
2021	Inline Audio-Player for pronunciations and other usages	Completed	Community Tech	link
2021	Mobile editnotices	Partially completed	Volunteer developers	link
2022	Select preview image	Partially completed	misc WMF team	link
2022	Better diff handling of paragraph splits	Completed	Community Tech	link
2022	Notifications for user page edits	Completed	misc WMF team	link
2022	Tool that reviews new uploads for potential copyright violations	Partially completed	misc WMF team	link
2022	Improve SVG rendering	Partially completed	misc WMF team	link
2022	IPA audio renderer	Partially completed	Community Tech	link
2022	Enhanced Move Logs	Not completed	Volunteer developers	link
2022	Change default number of search results displayed	Completed	Volunteer developers	link
2022	Autosuggest linking Wikidata item after creating an article	Partially completed	Community Tech	link
2022	Show editnotices on mobile	Partially completed	Volunteer developers	link
2022	Table sorting on mobile	Completed	misc WMF team	link
2022	Categories in mobile app	Completed	misc WMF team	link
2022	Get WhatLinksHere's lists in alphabetical order	Partially completed	misc WMF team	link
2022	Mass-delete to offer drop-down of standard reasons, or templated reasons.	Completed	Volunteer developers	link
2022	Gadget: Who is active	Completed	misc WMF team	link
2022	Enable negation for tag filters	Completed	misc WMF team	link
2022	Full page editing	Completed	misc WMF team	link
2022	Enable Thanks Button by default in Watchlists and Recent Changes	Completed	misc WMF team	link
2022	Show edit count at Special:Contributions	Completed	Volunteer developers	link
2022	Missing LaTeX capabilities for math rendering	Partially completed	Community Tech	link
2022	Kartographer icon improvements	Completed	misc WMF team	link
2022	Audio links that play on click	Completed	Community Tech	link
2022	Allow adding sitelink to redirect in Wikipedia section	Completed	misc WMF team	link
2022	Show nearby or related articles in maps	Partially completed	misc WMF team	link
2022	Do not follow sitelink redirects when redirect badge is used	Completed	misc WMF team	link
2022	Finish deployment of videojs (and remove kultura)	Completed	Volunteer developers	link
2022	Alert View	Completed	misc WMF team	link
2022	Improve plain-text change tag selector	Completed	misc WMF team	link
2022	Preference to set default watchlist expiry	Partially completed	Community Tech	link
2022	Watchlist in app	Completed	misc WMF team	link
2022	No option to enable/disable automatic signing in DiscussionTools	Completed	misc WMF team	link
2022	Load images in metered networks only on demand	Completed	misc WMF team	link
2022	Search within article on iOS app	Completed	misc WMF team	link
2023	Allow citations to be edited in the references section with VisualEditor	Completed	misc WMF team	link
2023	Notifications for user page edits	Completed	Volunteer developers	link
2023	Extend "Who Wrote That?" tool to more wikis	Completed	Community Tech	link
2023	Auto-save feature	Partially completed	Community Tech	link
2023	Warn when adding a url reference that matches the SpamBlacklist	Partially completed	Community Tech	link
2023	Layering/timing of blocks	Partially completed	Community Tech	link
2023	Add link to CentralAuth on Special:Contributions	Completed	Community Tech	link
2023	Add ability to share QR code for a page in any Wikimedia project	Partially completed	Community Tech	link
2023	Search bar at preferences	Completed	misc WMF team	link
2023	Native SVG support	Partially completed	Volunteer developers	link
2023	Linking of citing tool with Wikidata	Partially completed	Volunteer developers	link
2023	Update Vega to the latest build	Partially completed	misc WMF team	link
2023	Utility to attach acccount to all wikis	Completed	Volunteer developers	link
2023	Create a button to expand all collapsible elements on a page	Partially completed	misc WMF team	link
2023	Editable gallery captions in Visual Editor	Completed	misc WMF team	link
2023	Return to the article when closing the MediaViewer	Completed	misc WMF team	link
2023	Make the AbuseFilter edit window resizable and larger by default	Completed	Volunteer developers	link
2023	New users should see the deletion log when visiting a deleted page on English Wikipedia	Completed	misc WMF team	link
2023	Support ASGI on Toolforge	Completed	misc WMF team	link
2023	Links for preferences	Not completed	None	link
2023	Fix Wikipedia IFTTT integration	Completed	misc WMF team	link
2023	Change information about the number of watchers on a page	Completed	Community Tech	link
2023	Fix security key (WebAuthn) support	Partially completed	Volunteer developers	link
2023	Add "search results per page" at the top of Special:Search	Completed	misc WMF team	link
2023	Quickly visit template's page in Visual Editor	Completed	misc WMF team	link
2023	Change year range shown in date selection popup	Completed	misc WMF team	link
2023	Watchlist edit - "check all" checkbox	Completed	misc WMF team	link
2023	Button to mark a single change as read in the global watch list	Completed	misc WMF team	link
2023	Make Special:Search on Commons show all requested thumbnails	Partially completed	Volunteer developers	link