Community Wishlist Survey/Historical Metrics 2023

Future of the Community Wishlist Survey
This page documents a completed research project.

Introduction

edit

The Community Wishlist Survey (hereinafter CWS) is an annual survey that allows contributors to the Wikimedia projects to propose and vote for tools and platform improvements. The survey had eight iterations since its inception in 2015, the last one being in 2023. The Community Tech team usually handles the technical development of the selected wishes, along with running the survey itself. In some cases, wishes have also been completed by volunteers or staff from other teams with little to no input from the Community Tech team.

Once the process begins, community members make proposals in various categories of technical wishes (the categories themselves have evolved over the years). Some of the categories include bots & gadgets, editing, multimedia & commons, moderation tools etc. The proposal phase is followed by a voting process, where community members again vote. Until 2020, the Community Tech team used to commit to the top 10 wishes (by the number of votes), however, since 2021 the Prioritization Framework has been used to select the wishes to be worked upon.


Before the next iteration of the survey, the Community Tech is working on analysing the feedback from previous years to understand how this process could be improved to keep up with the growing needs of the movement, referred to as the Future of the Wishlist. The historical metrics analysis is intended to support this process by surfacing historical trends of survey. The majority of the report focuses on community participation i.e. who have been participating, where the users come from (home wiki), their experience level etc.

Definitions & Notes

edit
  • proposal refers to a technical wish proposed by the community.
  • proposer refers to the user who proposed the wish.
  • discussant refers to a user who takes part in discussion of a proposal.
  • voter refers to a user who takes part in voting of a proposal.
  • home-wiki refers to the wiki where a user had the highest number of edits during the two preceding years of a survey.
  • newcomer refers to users who participated for the first time in CWS, but not a new contributor to Wikimedia projects

Notes
  1. The goal of the analysis and this report is to understand how CWS evolved and what has happened historically. This is not an evaluation report, but rather an exploratory data report.
  2. Although there was a survey conducted in 2018, the team changed the naming approach to state the year during which the teams intends to work on the wishes instead of the year in which the survey is held, which created a gap.
  3. In several metrics and graphs, you might see a huge difference for the year 2020, for example, drop in number of participants, this was due to the team's special focus on non-Wikipedia content projects for that year. The traditional structure was resumed from 2021.
  4. All the metrics, unless otherwise mentioned, are about the accepted proposals only. Starting with 2017, only selected proposals go into the voting phase, and others get rejected for various reasons, discussed later in the report.
  5. A combination of data available in MediaWiki databases, results from parser API and web scraping of CWS pages was used to gather the required data. For accepted proposals, data gathering failed for 1% of the proposals, and when rejected proposals are included, the failure rate is at 3%. However, as the goal is to understand broad trends, this won't affect any of the metrics.
  6. For consistency and easier comparison during analysis, some categories were were either merged or renamed. Please refer to the footnote[1] for information about them.

Report

edit

Proposals and Categories Over Years

edit
 
Fig 1: Proposals by Year

The number of proposals submitted consistently increased by ~24% (median) from 2015 to 2022, with the exception of 2020. This was followed by sharp decline in 2023, by ~40% compared to 2022.

Accepted proposals followed the same trend as submitted ones, since the practice was started in 2017, on average ~35% of the proposals were rejected. The three most common reasons for wishes being rejected from entering the voting phase:

  • Too big, broad or out of scope for Community Tech team
  • Suggests a social/cultural change rather than a technical feature
  • Insufficient information provided or idea not explained well

Categories

edit

Editing, Multimedia and Commons, Wikidata, Moderation tools, and Navigation tools, are the top categories (in descending order) by the number of proposals, collectively accounting for 55% of the proposals. Followed by Watchlists, Mobile apps, Bots / gadgets, Reading, and Wikisource, and along with the top five categories, they account for 80% of the proposals. In the year 2020, by number of proposals, Wikisource was at the top accounting for 39% of the proposals, collectively with Wiktionary and Wikiversity, account for 82% of the proposals.

Overall Participation

edit
 
Fig 2: Overall Participation by Year & Roles

There has been consistent increase in user participation at a rate 22% annually from 2015 to 2021, excluding 2020. During these years, while voters and discussants participation grew by 25% annually, number of people proposing wishes only grew by 15%.

Post-2021, the participation has been declining at 10% annually until 2023. When analysed by roles, number of people participating in discussing the proposals have the highest decline with 25% annually. However, the data is insufficient to predict the future trend.

Despite the decline, the participation levels of 2022 and 2023 have reached similar to that of 2019. When the participation data is compared with editors and content key product metrics, during the months of November-December 2020 (when the 2021 survey was conducted), Wikimedia projects had the highest number of active editors (~93,000), compared to the same period for 2019 and 2022-2023 where the number of active editors on average was ~88,000. As the survey gets promoted using CentraNotice banners, and number of editors on our projects has an impact on the participation.

Roles

edit

On average, of all participants

  • proposers account for 7-9%
  • discussants account for 20-25%
  • voters account for 70-75%

Roles across Categories

edit
Top 10 categories of participation by roles
in descending order; coloured cell indicates notable change in rank of a category across roles
Role Category Role Category Role Category
proposers Editing discussants Editing voters Editing
Multimedia and Commmons Multimedia and Commmons Moderation tools
Wikidata Moderation tools Multimedia and Commmons
Moderation tools Wikidata Bots and gadgets
Navigation tools Reading Reading
Watchlists Bots and gadgets Navigation tools
Mobile and apps Navigation tools Wikidata
Reading Watchlists Citations
Bots and gadgets Mobile and apps Mobile and apps
Citations Citations Anti-harassment
Across all roles, the top 10 categories collectively account for 80% of the participation.

The top categories for proposers are similar to the top categories by the number of proposals received. While a higher number of proposals and proposers can likely indicate the need for more features related to a category, the top categories by voting can likely indicate higher interest/need for the implementation of features related to a category.

Participation from Wikimedia projects

edit
Number of uniques wikis, by role
Role # Wikis
Proposers 86
Discussants 126
Voters 227

Home wiki

edit

Across all roles, the highest number of participants has been from English Wikipedia, accounting for 30% of all participants. This was followed by Wikimedia Commons, Wikidata, German Wikipedia and French Wikipedia for proposers and voters. For discussants, more participants came from Meta-Wiki instead of French Wikipedia. Together, the top five wikis account for 60% of participation across all roles.

Across all roles, contributors from Wikimedia Commons and Wikidata had the highest participation in proposals related to Multimedia & Commons and Wikidata respectively. Whereas, contributors from English, German, French, Spanish and Russian Wikipedias, had the highest participation in Editing related proposals. Interestingly, contributors from the Chinese Wikipedia have the highest participation in Moderation Tools related proposals.

From wikis with the highest participation (top 15), the top five categories account for 50% of all participation from those respective wikis.

By size rank

Across all years, with an exception to 2020, the top five large wikis account for 45% of participation, and the next five account for 22% of participation. In 2020, 66% of the participation was from outside of top ten large wikis. Although Spanish and Japanese Wikipedias are among the top five large wikis, they are not part of the top five wikis by participation. Similarly, Italian and Portuguese Wikipedias are among the top ten large wikis, but they are not part of the top ten wikis by participation.

Project groups

edit
 
Fig 3: Participation Trends by Project Group, by Year

On average across all roles, Wikipedia, Wikimedia Commons, and Wikidata, account for 85% of the participation, with Wikipedia alone accounting for about 65%.

Wikipedia, Wikimedia Commons and Wikidata had similar participation trends from 2015 to 2019, growing at 30% annually. Similar to the overall participation trend, there was highest participation in 2021, and followed by a decline until 2023, reaching participation levels similar to that of 2019.

Compared to other project groups, Wikisource had the highest growth rate annually, at 50% until 2020. Unlike the overall participation trend, participation from Wikisource declined in 2021. Although there was a slight increase in 2022, it was followed by a sharp decline in 2023.

Wiktionary had consistent increase in participation until 2020, followed by a decline until 2022, and a slight increase in 2023.

Participation by Experience Levels

edit

Editing experience

edit
edit buckets used 0-99 100-999 1000-4999 5000+

Home wiki

edit
 
Fig 4: Participation Trends by Edit Bucket on Home wiki, by Year

Across all roles, users with higher editing experience tend to participate more the in survey. 57% of the participation had been from users with 5000+ edits, followed by 1000-4999 and 100-999 edit buckets, accounting for 20% each.

Proposers tend to be more experienced than other participants; on average, across years, 85% of the proposers of belong to the 5000+ edit bucket. The 2020's survey is a notable exception to this trend, which saw increased participation from 100-999 & 1000-4999 edit bucket groups.

During the initial three years, participation from the 5000+ edit bucket group accounted for 75% of the participation; however, in the recent years (2021-2023), there has been a notable shift in the trend. During the last three years, 5000+ edit bucket group accounted for 60%. While the participation from the 1000-4999 edit bucket group remained the same, there has been a substantial increase in participation, from 5% during the initial years to 20% in the last three years, from the 100-999 edit bucket group.

However, this trend is largely influenced by changes in participation among voters. Among proposers and discussants, the 5000+ edit bucket group has accounted for a majority of the participation with 83% and 77% respectively.

Edit buckets and Categories

Across all roles, with a few exceptions, users belonging to the 0-99 group have the highest participation in Reading related wishes, whereas the 100-999 & 1000-4999 groups have the highest participation in Editing related wishes. Among users with 5000+ edits, Editing, Multimedia & Commons and Moderation tools, are the most popular categories.

Meta-Wiki

edit

Most of the users who participated in the survey were not active participants on Meta-Wiki. 70% of the participants belong to the 0-99 edit bucket, 20% to the 100-999 edit bucket, and 5000+ edit bucket group only accounts for 5% of the participation. When considered only proposers and discussants, 50% belong to the 0-99 edit bucket and 30% to the 100-999 bucket.

User rights

edit
categorisation of users rights
category rights
advanced abusefilter, sysop, interface-admin, checkuser, oversight, botadmin
intermediate having at least one user right, but don't fall into either of other two categories
none extendedconfirmed or ipblock-exempt or none
 
Fig 5: Participation Trends by User Rights, by Year

Overall, 28% of participants possessed advanced user rights, while 43% possessed at least one intermediate level user right (not including advanced rights). When observed by role, 78% of the proposers have at least one user right (including advanced rights).

Over the years, the trend is similar to that which was observed with edit buckets, where users from lower edit buckets have started engaging more in the process. In case of user rights, during the initial years, users with no rights accounted for only 18% of participation, while during the last three years, they accounted for 41% of the participation.

User Rights and Categories

Among proposers with advanced user rights, Moderation Tools has been the most popular category. Across all roles, for users with at least one user right, Editing and Multimedia & Commons have been the most popular categories. While Editing related wishes have also seen high engagement from participants with no specialized user rights, Reading has also been a equally popular category among voters.

User age

edit
user age has been calculated as the difference between registration timestamp and the month a survey was conducted
 
Fig 6: Average User Age by, by Year

The average User age of participants has been increasing over the years, starting from 5 years in 2015 to 9 years in 2023. As seen Fig 6, although the average user age varied by role, all of them follow an upward trend. However, this does not necessarily imply that more veterans are participating in the survey than newcomers, especially when comparing with the trends of edit buckets and user rights.

As explained in the later sections, the survey has substantial percentages of both first-time and returning participants. For example, when a user from 2015 with user age of 5 years participated in 2023, their user age would be considered as 13 years. As the age of the Wikimedia movement increases with each year, it is likely that the average user age of participants will increase as well.

User age and Categories: Among all categories and role intersections, proposers participating in the Anti-harassment category have the highest account age, with 11 years on average.

Miscellaneous

edit

Newcomer Engagement

edit
Note: 2015 and 2016 were excluded from the analysis of newcomer engagement.
 
Fig 7: Distribution of returning and new participants, by role and year

On average, 55% of the participants were newcomers (i.e. participating in the survey in any role for the first time), the remaining 45% were returning participants. However, when observed by only proposers and discussants, 60% of them have been returning participants. With an exception for 2020, English Wikipedia, Wikimedia Commons, and German Wikipedia are the top Wikimedia projects where newcomers come from, accounting for 70% of all newcomers on average.

With an exception for 2020, since 2016, returning participants had the highest participation in wishes related to Moderation Tools, whereas for newcomers, it was a mix of Moderation Tools and Editing related proposals.

Cross-Proposal Engagement

edit
Average Number of Proposals a Participant Engages with, by Role
Role # Proposals
Proposer 1
Discussant 2
Voter 6

Notably, Wikisource and Wikidata were two categories where each participant submitted two proposals on average (irrespective of 2020), whereas for all other categories it was one per participant.

YoY engagement

edit
2015 2016 2017 2019 2020 2021 2022
2015 100% - - - - - -
2016 26% 100% - - - - -
2017 21% 30% 100% - - - -
2019 17% 24% 33% 100% - - -
2020 12% 11% 17% 18% 100% - -
2021 13% 16% 21% 26% 14% 100% -
2022 11% 13% 17% 19% 14% 26% 100%
2023 9% 11% 14% 15% 11% 20% 24%

Participants from recent iterations of the survey tend participate to more in a given year, rather than from older iterations. For example, in 2023, 9% of participants also participated in 2015, where 24% participants in 2022.

Note: The percentages are not mutually exclusive within a given year. For example, a user who participated in both 2015 and 2022, would be counted at both the times.

Proposals Re-submission

edit
Several approaches were considered to find the rate of re-submission of proposals, including exact title match, proposal title text similarity, and associated Phabricator tickets. Except for analysis based on exact title match, the other two approaches are unreliable and prone to a high error rate, and they are therefore not presented in the report.
Category Proposal Years
Wiktionary Display definitions from Wikisource dictionaries 4
Wikisource XTools Edit Counter for Wikisource 4
Wiktionary Custom list for language learner 3
Communication_tools Notify users when their revision has been approved or rejected 3
Wiktionary Insert attestation using Wikisource as a corpus 3
Wikidata Expand automatic edit summaries 3
Wikidata Display reference in edit summary when a reference is added 3
Wikisource ProofreadPage extension in alternate namespaces 3
Wikisource Improve workflow for uploading books to Wikisource 3
Wikisource Improve export of electronic books 3
Wikisource Offer PDF export of original pagination of entire books 3
Wiktionary Context-dependent sort key 3
Miscellaneous Provide a tool to efficiently analyze the usage of a template 3
Multimedia_and_Commons Support 360 photo viewing 3

Selected/won proposals

edit
Selected/won proposals are proposals which have made it to the top based on voting, until 2020, and from 2021, the proposals which have been selected by the Community Tech based on the prioritization framework, or taken up by other WMF teams.

The trends of participation at selected/won proposals, based on home wiki and project group follow the overall participation trend, with no significant differences. However, when observed by edit bucket on home wiki, there has been increased participation from more experienced editors (i.e. higher edit bucket) at selected/won proposals. Notably, 87% of all selected/winning proposals were created by editors with 5000+ edits, collectively with editors having 1000+ edits, they account for 99%.

Implementation

edit

Until 2020, from the top 10 proposals by number of votes received each year, 67% (30) have been completed, 27% (12) were declined, and 4% (2) were partially completed. Among the proposals have been successfully completed, 83% were developed by the Community Tech team, and 13% by other WMF teams, and the rest by volunteers.

From 2021 to 2023, where the prioritization framework has been used, 10 wishes were completed by the Community Tech team, 8 were partially completed, and one was declined. During the same period, 30 wishes were completed by other WMF teams, and 7 were partially completed. In addition, 15 wishes were completed by volunteers developers.


The list includes top 10 proposals until 2021. From 2022, it includes all proposals for which status information is available on the results page of the respective year (as 20 November 2023).
Year Proposal Status Developed by Link
2015 Migrate dead external links to archives Completed Community Tech link
2015 Improved diff comparisons Completed Community Tech link
2015 Central repository for gadgets, templates and Lua modules Declined misc WMF team link
2015 Cross-wiki watchlist Declined None link
2015 Numerical sorting in categories Completed Community Tech link
2015 Pageview Stats tool Completed Community Tech link
2015 Global cross-wiki user talk page Declined None link
2015 Improve the plagiarism detection bot Completed Community Tech link
2015 Add a user watchlist Declined None link
2015 Allow categories in Commons in all languages Completed misc WMF team link
2016 Global gadgets Declined misc WMF team link
2016 Edit summary length for non-Latin languages Completed misc WMF team link
2016 Section heading URLs for non-Latin languages Completed Community Tech link
2016 Global preferences Completed Community Tech link
2016 Rewrite XTools Completed Community Tech link
2016 Wikitext editor syntax highlighting Completed Community Tech link
2016 Warning on unsuccessful login attempts Completed Community Tech link
2016 Automatic archive for new external links Declined None link
2016 Fix Mr.Z-bot's popular pages report Completed Community Tech link
2016 User rights expiration Completed Volunteer developers link
2017 Kartographer improvements Completed misc WMF team link
2017 Ping users from the edit summary Completed Community Tech link
2017 Development of the programs and events dashboard Completed Community Tech link
2017 Blame tool Completed Community Tech link
2017 Template wizard Completed Community Tech link
2017 Deploy Article Alerts to other languages Declined Community Tech link
2017 Auto-save feature in Visual Editor and WikiText Editor Completed misc WMF team link
2017 Allow 'thanks' notification for a log entry Completed Community Tech link
2017 SVG-Translate Completed Community Tech link
2017 Commons deletion notification bot Completed Community Tech link
2019 Page Curation and New Pages Feed improvements Completed Community Tech link
2019 Night mode Declined None link
2019 Add an option to require email address and username to reset password Completed Community Tech link
2019 Improve export of electronic books Completed Community Tech link
2019 VisualEditor: Allow references to be named Declined None link
2019 Put mw.toolbar back Declined None link
2019 Watchlist item expiration Completed Community Tech link
2019 Article reminders Declined None link
2019 Include section name in the diff Partially completed Community Tech link
2019 2FA available for all concerned editors Declined None link
2020 Improve export of electronic books Completed Community Tech link
2020 New OCR tool Completed Community Tech link
2020 Migrate Wikisource specific edit tools from gadgets to Wikisource extension Completed Community Tech link
2020 Inter-language link support via Wikidata Partially completed Volunteer developers link
2020 Insert attestation using Wikisource as a corpus Not completed Community Tech link
2020 Ebook Export Improvement Completed Community Tech link
2020 Migrate Wikisource specific edit tools from gadgets to Wikisource extension Completed Community Tech link
2020 Insert attestation using Wikisource as a corpus Not completed Community Tech link
2021 Warn when linking to disambiguation pages Completed Community Tech link
2021 Copy and paste from diffs Completed Community Tech link
2021 Realtime Preview Completed Community Tech link
2021 (Un)delete associated talk page Completed Community Tech link
2021 Bibliographic bot for Wikidata Declined Community Tech link
2021 Allow global user rights to expire automatically Completed Volunteer developers link
2021 Global events calendar Completed misc WMF team link
2021 Link Wikipedia redirects to Wikidata items Completed misc WMF team link
2021 Inline Audio-Player for pronunciations and other usages Completed Community Tech link
2021 Mobile editnotices Partially completed Volunteer developers link
2022 Select preview image Partially completed misc WMF team link
2022 Better diff handling of paragraph splits Completed Community Tech link
2022 Notifications for user page edits Completed misc WMF team link
2022 Tool that reviews new uploads for potential copyright violations Partially completed misc WMF team link
2022 Improve SVG rendering Partially completed misc WMF team link
2022 IPA audio renderer Partially completed Community Tech link
2022 Enhanced Move Logs Not completed Volunteer developers link
2022 Change default number of search results displayed Completed Volunteer developers link
2022 Autosuggest linking Wikidata item after creating an article Partially completed Community Tech link
2022 Show editnotices on mobile Partially completed Volunteer developers link
2022 Table sorting on mobile Completed misc WMF team link
2022 Categories in mobile app Completed misc WMF team link
2022 Get WhatLinksHere's lists in alphabetical order Partially completed misc WMF team link
2022 Mass-delete to offer drop-down of standard reasons, or templated reasons. Completed Volunteer developers link
2022 Gadget: Who is active Completed misc WMF team link
2022 Enable negation for tag filters Completed misc WMF team link
2022 Full page editing Completed misc WMF team link
2022 Enable Thanks Button by default in Watchlists and Recent Changes Completed misc WMF team link
2022 Show edit count at Special:Contributions Completed Volunteer developers link
2022 Missing LaTeX capabilities for math rendering Partially completed Community Tech link
2022 Kartographer icon improvements Completed misc WMF team link
2022 Audio links that play on click Completed Community Tech link
2022 Allow adding sitelink to redirect in Wikipedia section Completed misc WMF team link
2022 Show nearby or related articles in maps Partially completed misc WMF team link
2022 Do not follow sitelink redirects when redirect badge is used Completed misc WMF team link
2022 Finish deployment of videojs (and remove kultura) Completed Volunteer developers link
2022 Alert View Completed misc WMF team link
2022 Improve plain-text change tag selector Completed misc WMF team link
2022 Preference to set default watchlist expiry Partially completed Community Tech link
2022 Watchlist in app Completed misc WMF team link
2022 No option to enable/disable automatic signing in DiscussionTools Completed misc WMF team link
2022 Load images in metered networks only on demand Completed misc WMF team link
2022 Search within article on iOS app Completed misc WMF team link
2023 Allow citations to be edited in the references section with VisualEditor Completed misc WMF team link
2023 Notifications for user page edits Completed Volunteer developers link
2023 Extend "Who Wrote That?" tool to more wikis Completed Community Tech link
2023 Auto-save feature Partially completed Community Tech link
2023 Warn when adding a url reference that matches the SpamBlacklist Partially completed Community Tech link
2023 Layering/timing of blocks Partially completed Community Tech link
2023 Add link to CentralAuth on Special:Contributions Completed Community Tech link
2023 Add ability to share QR code for a page in any Wikimedia project Partially completed Community Tech link
2023 Search bar at preferences Completed misc WMF team link
2023 Native SVG support Partially completed Volunteer developers link
2023 Linking of citing tool with Wikidata Partially completed Volunteer developers link
2023 Update Vega to the latest build Partially completed misc WMF team link
2023 Utility to attach acccount to all wikis Completed Volunteer developers link
2023 Create a button to expand all collapsible elements on a page Partially completed misc WMF team link
2023 Editable gallery captions in Visual Editor Completed misc WMF team link
2023 Return to the article when closing the MediaViewer Completed misc WMF team link
2023 Make the AbuseFilter edit window resizable and larger by default Completed Volunteer developers link
2023 New users should see the deletion log when visiting a deleted page on English Wikipedia Completed misc WMF team link
2023 Support ASGI on Toolforge Completed misc WMF team link
2023 Links for preferences Not completed None link
2023 Fix Wikipedia IFTTT integration Completed misc WMF team link
2023 Change information about the number of watchers on a page Completed Community Tech link
2023 Fix security key (WebAuthn) support Partially completed Volunteer developers link
2023 Add "search results per page" at the top of Special:Search Completed misc WMF team link
2023 Quickly visit template's page in Visual Editor Completed misc WMF team link
2023 Change year range shown in date selection popup Completed misc WMF team link
2023 Watchlist edit - "check all" checkbox Completed misc WMF team link
2023 Button to mark a single change as read in the global watch list Completed misc WMF team link
2023 Make Special:Search on Commons show all requested thumbnails Partially completed Volunteer developers link

Footnotes

edit
  1. The categories Admins_and_stewards, Admins_and_patrollers, and Moderation_and_admin_tools are categorized as Moderation_tools. The categories Commons and Multimedia are categorized as Multimedia_and_Commons. The categories Notifications,_Watchlists_and_Talk_Pages, Notifications, Talk_pages, and Special_pages are categorized as Communication_tools. Lastly, the categories Search_and_Categories, Search, and Categories are categorized as Navigation_tools.