Wikimedia Meta-Wiki

Community Wishlist Survey 2021/Bots and gadgets

< Community Wishlist Survey 2021
Community Wishlist Survey 2021
Bots and gadgets
15 proposals, 296 contributors, 514 support votes
The survey has closed. Thanks for your participation :)



Allow loading gadgets from wikitext

  • Problem: Some templates depend on JavaScript gadgets: enhanced search fields, flyout menus, and so on. Currently, if an interface administrator wants to get the gadget loaded on pages that use this template, the only option is to run a small amount of JavaScript (determining whether the full gadget should be loaded) on each and every page load, for everyone. Maybe the template is used only on one or two pages out of the tens or hundreds of thousands of pages on the wiki, yet all page loads are slowed down by it.
  • Who would benefit: Readers by not having unnecessary JavaScript slowing page loads, interface admins (and even other tech-savvy users without IA right) having more straightforward way of controlling gadget loading.
  • Proposed solution: Have a parser tag similar to TemplateStyles that includes a ResourceLoader module indicated in its argument.
  • More comments: Care should be taken in order not to allow arbitrary code to be loaded by untrusted users (the parser tag in wikitext can be added and edited even by anonymous users). While the contents of ResourceLoader modules can be changed by only select people, probably the safest solution is to create a “namespace” (e.g. ext.gadget.includeable.*) for this purpose within gadgets, and disallow loading any other RL module. Disallowed modules can be loaded by making them dependencies of allowed gadgets. (Dependency is safe, as gadget dependencies can be edited only by trusted interface admins.)
  • Phabricator tickets: phab:T17075 from 2008, T241524
  • Proposer: Tacsipacsi (talk) 00:49, 26 November 2020 (UTC)Reply[reply]

Discussion

  • An alternative could be to have one default hidden gadget with the classnames/criteria -> script loading sequences... —TheDJ (talk • contribs) 12:53, 26 November 2020 (UTC)Reply[reply]
    • @TheDJ: Do you mean a gadget that decides on client side whether it should load other gadgets? This is exactly the small amount of JavaScript I mentioned and that I’d like to eliminate. (To be exact, I meant whatever way that small amount of JS is run: one default gadget, many default gadgets, Common.js etc.) This may be a small amount per gadget, but it’s not zero, and especially on Wikibooks and Wiktionaries can be many different such gadgets. —Tacsipacsi (talk) 23:51, 26 November 2020 (UTC)Reply[reply]
  •   Comment I think as proposed this would make security and performance very difficult to uphold. However the idea of allowing a gadget to be enabled "by default" but limited to a subset of pages makes sense. We already support narrowing the default by skin, platform, and user rights. We could for example add support for limiting to categories. Then to initiate it from a template one would add the article to a (hidden) category. This would place the security control with the gadget configuration (the gadget definition specifies its categories, the page does not directly require a gadget), and would also encourage more re-usable logic with less effort (e.g. many articles and templates already have existing categories that you can make use of). It also means gadgets can be merged, split or renamed without their loading breaking as there would not be a direct connection between the two. Perhaps the proposal could be edited to focus more focussed on the problem and use case, and leave the implementation details to be decided by developers and other stakeholders if/when it is worked on. --Krinkle (talk) 20:30, 11 December 2020 (UTC)Reply[reply]
    Adding a ResourceLoader module from a parser function doesn't seem problematic to me performance-wise; a number of parser hooks already do that. In terms of security, the only issue I can think of is that it must not allowed to work on "safe" pages like login (via a system message; probably it shouldn't work in system messages at all).
    Categories would be nice for tracking where a script is used, but we could also just (ab)use the templatelinks table like we do for TemplateStyles. Tgr (talk) 08:51, 13 December 2020 (UTC)Reply[reply]
    @Krinkle: If we were to use categories to decide whether or not to load a gadget, renaming gadgets would be easier, but renaming categories would be more difficult—and the consequences are far less obvious, fixing a typo in a category name would stop the gadget from working for no apparent reason. Furthermore, if one renames a category, they probably won’t be able to adjust the gadget configuration afterwards (as they’re not necessarily an interface admin). Also, gadget naming is an internal detail completely transparent for users, while category names are user-visible, so they’re much more likely needed to be changed. I think gadget renaming is already mostly impossible, as there’s no other permanent identifier for gadgets, so renaming a gadget would likely revert it back to its initial state for everyone (i.e. turn it on in case of default gadget, turn it off otherwise). I see your concerns (although I agree with Tgr in that they’re unlikely to be actual issues), but the point of easing renames is simply false IMO. Split and merger is probably really more convenient with the category-based system, but it still doesn’t outweigh the renaming issues. Also, I don’t like polluting the categories with such technicalities.
    Security-wise: what issue do you think would appear that isn’t handled by the proposed tag-based system, but would be handled by your category-based one? IAs are already in control of what could be possibly loaded (with the namespace system), while what’s actually loaded can be controlled by anyone anyway (through adding either the parser tag or the category).
    @Tgr: Entirely disabling it in messages could be the way to go, but Common.js is also disabled in these contexts, so there should be a way in MW core to determine whether risky RL modules can be loaded. (There are some MediaWiki messages like s:MediaWiki:proofreadpage_index_template that are practically templates stored in the MediaWiki namespace, and may benefit from this feature. I don’t know if these can be programmatically distinguished from the cookie prompt on the login page, which should certainly not have anything like this.) —Tacsipacsi (talk) 21:43, 14 December 2020 (UTC)Reply[reply]
  • I think it would be better to implement phab:T204201.
    • That has much broader capabilities and gives more benefit at various circumstances.
    • The common approach is to look for elements with certain classes after DOM has been loaded, then proceed. If that could be narrowed to a particular namespace it would save a lot of client execution time already.
    • Using a category, even a hidden maintenance category, might distract attention of editors. Equipping some elements with a class is very silent.
    • However, the improvements suggested in phab:T204201 may introduce more Cirrus filters like incategory and hastemplate (actually: transcludes). They should use underscore page titles and leave spaces as separators.
    • The target is to reduce the amount of gadgets to be loaded in a page but could never be useful under predictable and easily detectable conditions.
    • I don’t think it is promising to introduce new Wikisyntax, new elements, new parser functions for making page contents triggering gadget execution. There are page properties and various criteria in T204201 which will deliver the same result but with less efforts and wider applicability.
      --PerfektesChaos (talk) 14:21, 18 March 2021 (UTC)Reply[reply]

Voting

Automatically import

  • Problem: Some projects often use sites with compatible Creative Commons licenses. Usually these projects use a bot, but this takes time, some stop operating, new sites start using the Creative Commons license and others stop using it. In addition, there are many smaller wikis that are abandoned or forgotten.
  • Who would benefit: Mainly Wikiquote and Wikinews, I think. But the tool would not be available to everyone, to avoid vandalism.
  • Proposed solution: I am proposing the creation of a tool that allows users (not all obviously) to automatically copy content from other sites. This would help many projects to grow.
  • More comments: I don't know if it has been proposed before, I await comments…
  • Phabricator tickets:
  • Proposer: Edu! (talk) 15:51, 18 November 2020 (UTC)Reply[reply]

Discussion

  • I think this will be highly useful. I know several compatible wikis that are used as sources of content. However, not being currently able to do the import as described in the proposal mean we can't get the best attribution for the content. We rely on third parties to track the individual work on the article. If I recall correctly, there was a problem with non-Wikimedia wikis due to different user bases (if you import edition ABCD done by User:XYWZ, the software considers the user:XYWZ in the end wiki instead of the source wiki). However, I think it should be technically feasible to adapt to wikis using a different repository of users.--FAR (talk) 10:04, 29 November 2020 (UTC)Reply[reply]
  • How is this different from Special:Export/Import and related functionality? --Tgr (talk) 06:53, 14 December 2020 (UTC)Reply[reply]

Voting

  •   Support UncleMartin (talk) 01:31, 9 December 2020 (UTC)Reply[reply]
  •   Support Omda4wady (talk) 07:25, 9 December 2020 (UTC)Reply[reply]
  •   Support If these sites use Creative Commons as a license, the proposal is excellent! WikiFer msg 22:47, 10 December 2020 (UTC)Reply[reply]
  •   Support Strong support. Can help a lot to save time. BoldLuis (talk) 10:40, 11 December 2020 (UTC)Reply[reply]
  •   Support Anonyme314159 (talk) 18:36, 15 December 2020 (UTC)Reply[reply]
  •   Support Sounds good but a bit broad. Are we talking imports to Wikisource? Commons? A frequent usecase is when a site provider has just re-licensed their images or whatever under a more restrictive license; since they can't cancel the old license we can rescue the material and re-host it. HLHJ (talk) 00:15, 20 December 2020 (UTC)Reply[reply]
  •   Support Juan90264 (talk) 03:08, 21 December 2020 (UTC)Reply[reply]
  •   Support S8321414 (talk) 14:08, 21 December 2020 (UTC)Reply[reply]

Artificial Intelligence Spam hunter

  • Problem: Spam is replacing vandalism as our biggest problem
  • Who would benefit: Everyone. But probably first the English Wikipedia
  • Proposed solution: Use the articles and edits that have been deleted as spam to train an Artificial Intelligence bot to identify spam, tag it for deletion or revert it where it is certain it is spam and bring it to human attention where it is less confident
  • More comments:
  • Phabricator tickets:
  • Proposer: WereSpielChequers (talk) 22:07, 29 November 2020 (UTC)Reply[reply]

Discussion

Voting

DabFix

  • Problem: The DabFix tool for rapidly populating disambiguation pages is no longer available.
  • Who would benefit: Editors who create disambiguation pages.
  • Proposed solution: Create an in-house version of DabFix capable of populating disambiguation pages with appropriate information (e.g., birth and death dates of human subjects, and short descriptions of their significance).
  • More comments:
  • Phabricator tickets:
  • Proposer: BD2412 T 00:24, 25 November 2020 (UTC)Reply[reply]

Discussion

Yes, none of these tools seem to work anymore, and they have always been erratic. Too bad, because they were very helpful tools: Dabfix, Dabsolver, Commonfixes.
Vmavanti (talk) 18:39, 8 December 2020 (UTC)Reply[reply]

I'd add Dablinks, which is the tool I use most. (Example page: [1].) Certes (talk) 16:45, 9 December 2020 (UTC)Reply[reply]
Code for Dablinks could be reused to warn when linking to disambiguation pages. Certes (talk) 17:11, 9 December 2020 (UTC)Reply[reply]

The tools are not currently available via their usual IP address. Even if they return, they may be unstable, which increases the priority of this wish. I have a backup of the source code, but no licence to use or deploy it. Certes (talk) 18:21, 25 December 2020 (UTC)Reply[reply]

Voting

Improve Auto-wiki-browser

  • Problem: Auto-wiki-browser's resolution is very low and cannot display pages in visual diff mode.
  • Who would benefit: AWB users
  • Proposed solution: Add support for visual diff mode and visual editor in AWB and improve it's resolution. It would also be great if a web-app version is available.
  • More comments:
  • Phabricator tickets:
  • Proposer: WikiAviator (talk) 06:25, 17 November 2020 (UTC)Reply[reply]

Discussion

  • Isn't that maintained by users, not MediaWiki/WMF? DemonDays64 (talk) 09:07, 17 November 2020 (UTC)Reply[reply]
    It is, however it has never been an obstacle for CommTech. A more serious problem is that AWB is developed on Windows while WMF engineers only use Mac/Linux. Max Semenik (talk) 10:29, 17 November 2020 (UTC)Reply[reply]
    Oh come on, I'm sure we can fund some cloud instances with Windows from an external provider.
    Although I remember seeing a web port at some point. Don't remember where I saw it and if it shares some codebase with the Windows app. Maybe working on that would be helpful?--Strainu (talk) 11:43, 17 November 2020 (UTC)Reply[reply]
    Yeah I think so. For working on CommTech, I don't see it as a problem because WMF can always modify it (assuming it is released on an open license) and make an officially-maintained version of it. Also, it would be better if more users could use AWB.WikiAviator (talk) 13:36, 17 November 2020 (UTC)Reply[reply]
    AWB is a community maintained, yes. I have been trying to rework it, but its codebase is so outdated, and there's a huge lack of comments in general in the code which makes it hard to see what does what. However, I am trying to swap out the Internet Explorer elements with Chromium (CEFSharp), but I can't make any promises with how feasible it is, and whether or not it'd just be better for somebody to start it all again from scratch. Ed6767 (talk) 14:46, 17 November 2020 (UTC)Reply[reply]
    I think we can start this from scratch as a Web-based application (for easier access) based on Auto-Ed, that would be easier for the developers in the community as well as the WMF. WikiAviator (talk) 06:14, 18 November 2020 (UTC)Reply[reply]
    .NET 5 is open source and can now be downloaded for Mac and Linux.[2]. I assume there is a compiler available for Mac/Linux development. (And why does WMF only use Mac/Linux? Is that even relevant if we have at least one volunteer compiling to Windows? Completely offtopic, I know. :) --Izno (talk) 05:48, 18 November 2020 (UTC)Reply[reply]
    Historically Windows is not a very developer-friendly OS (considering everything we deploy to is Linux-based). But times have changed, as I understand it. Windows can now do Linux-y things too, and I know several WMF and WMDE devs who use Windows. Regardless, I don't think we should perpetuate AWB as a Windows application by adding more features to it. If it's true .NET can be used cross-platform, as you say, getting that working is step #1, but it sounds like a complete rewrite is the better long-term option which is surely out of scope :(
    That said, @WikiAviator have you tried JWB, as Fastily mentions below? That is a web-based version that I think would be much easier for us to contribute to (if the maintainer is open to outside contributions). MusikAnimal (WMF) (talk) 22:03, 21 November 2020 (UTC)Reply[reply]
    I've tried it lately and it has problems making lists (can't make list for new pages), but the foundation is solid (it works smoothly overall). I think we can collaborate with the maintainer so we can revamp and publicly release it as official commtech (like Huggle and AWB). I agree that the web version would be easier for us compared to the Windows version (running discontinued IE). If JWB is stable enough, we can abandon AWB altogether. WikiAviator (talk) 13:29, 23 November 2020 (UTC)Reply[reply]
    The core of .NET 5 is open-source and cross-platform, but this doesn’t mean everything built on this core is as well. Windows Forms, used by AWB, is still Windows-only in .NET 5. Although it’s possible in theory to get current AWB work on Linux (with Mono and some other additional software), I haven’t been able to do so yet. GTK# is an open-source GUI alternative, but it looks like a dead project at a glance. As far as I understand, .NET MAUI is truly cross-platform and under heavy development, but this heavy development means that its general availability is planned in a year, so while porting AWB to it may be the way to go, it probably won’t happen in 2021. —Tacsipacsi (talk) 00:24, 26 November 2020 (UTC)Reply[reply]
  • There is a browser based version of AWB: w:User:Joeytje50/JWB -FASTILY 05:11, 18 November 2020 (UTC)Reply[reply]
Tracked in Phabricator:
Task T157271

Voting

API for deleted title search

  • Problem: Some time ago, it became possible to search the archive table for titles of deleted pages - https://en.wikipedia.org/w/index.php?prefix=%28company%29&title=Special%3AUndelete&fuzzy=1 (admins only). This functionality is not accessible through the API and therefore cannot be incorporated into bots and gadgets used by administrators. As a consequence, this limits the discoverability of reposts under a slightly different title (a very common abuse tactic).
  • Who would benefit: Admins, patrollers, bot operators
  • Proposed solution: See above. A module is added to the Action API that allows searching for deleted titles.
  • More comments:
  • Phabricator tickets: T192023
  • Proposer: MER-C (talk) 17:47, 21 November 2020 (UTC)Reply[reply]

Discussion

Voting

Inform gadget developers of errors in their gadgets/scripts

  • Problem: Wikimedia projects now have Logstash which allows us to log client-side errors experienced by our users. Since the information is in logstash, only LDAP users can view this information. My proposal is to allow gadget developers to see these errors for their gadgets in anonymized form.
  • Who would benefit: All gadget developers will have a better insight into how their gadgets are working and users using the gadgets will experience fewer bugs.
  • Proposed solution: With LDAP credentials I can filter file url results to show only errors relating to my gadget. Without LDAP however, I can't see this. One option would be to automate the table creation manually being created onUser:Jdlrobson/User_scripts_with_client_errors#Top_errors
  • More comments:
  • Phabricator tickets:
  • Proposer: Jdlrobson (talk) 16:14, 18 November 2020 (UTC)Reply[reply]

Discussion

Voting

Bot to input the proper class for all wiki projects

  • Problem: That most wikiprojects aren’t put in their proper class or importance
  • Who would benefit: Editors who want to know what needs improving upon, in addition to users who search through projects
  • Proposed solution: A bot should be created in order to fill out the classes and importance of all wikiprojects, mainly the classes because it can be read off other wikiprojects and it is already automated in.
  • More comments:
  • Phabricator tickets:
  • Proposer: Bigmike2346 02:30, 17 November 2020 (UTC)Reply[reply]

Discussion

  • @Bigmike2346: Hello and thanks for participating in the survey! I have corrected the formatting of your proposal. If you could, please fill in the "Problem" and "Who would benefit" questions. Thanks! MusikAnimal (WMF) (talk) 03:28, 17 November 2020 (UTC)Reply[reply]
  • I believe this is a matter for specific projects to discuss on a case-by-case basis, as each WikiProject has a different purpose. If the proposal is to automatically assess class then many editors would object, thinking that only a human can properly assess each class criterion. If the proposal is just to copy over classes/importances where they occur for other projects on the same talk page then this again is flawed, as importances can vary across projects and WikiProjects may even have different levels of strictness for different classes. A user-maintained bot could serve whatever action is required, rather than WMF involvement. — Bilorv (talk) 17:55, 17 November 2020 (UTC)Reply[reply]
    Class at least is pretty universal for most pages. In that regard we used to have a bot on en.WP at least that would duplicate classes from one banner to another. (So I agree with the final comment.) --Izno (talk) 19:25, 17 November 2020 (UTC)Reply[reply]

Voting

Gadgets improvements

  • Problem: Gadgets are hard to develop and maintain, even more so for smaller communities and non-English wikis
  • Who would benefit: Anyone who uses or develops gadgets
  • Proposed solution: See below
  • More comments:
  • Phabricator tickets: a bunch
  • Proposer: DannyS712 (talk) 06:39, 23 November 2020 (UTC)Reply[reply]

Work on Gadgets 2.0 is stalled, and the Gadget and Gadget definition namespaces are registered and reserved, but not used for anything. The following parts of the old roadmap (mw:Extension:Gadgets/Roadmap#Gadgets 2.0) and tracking task (phab:T31272) should be prioritized

  • "No more manual editing of gadgets definition, everything should have its GUI to change the underlying JSON definition" - add an interface to manage the gadget definitions (rights required, scripts and styles to load, messages, default enabled, whether the gadget is hidden, etc.)
  • Gadget code and definitions should move out of the mediawiki namespace to the dedicated gadget and gadget definitions namespace (see the example use of the gadget definitions namespace at mw:Extension:Gadgets#Using Gadget Definition Namespace, though as noted above it shouldn't need to be edited manually as json, but rather via a GUI)
  • "Structured localization framework for gadgets" - phab:T238386

Discussion

Is this complete and deploy Gadgets 2.0? "Gadgets improvements" suggests broader topic but the proposal seems limited to the extension. – Ammarpad (talk) 04:36, 24 November 2020 (UTC)Reply[reply]

Essentially, yes, but since its not very clear how "Gadgets 2.0" is currently scoped/defined, I listed the key issues that I thought should be addressed DannyS712 (talk) 04:58, 24 November 2020 (UTC)Reply[reply]
Much wanted. It had promising development in the past but then it silenced.
For localization improvements, Community Wishlist Survey 2021/Bots and gadgets/Easy and effective way to translate gadgets and userscripts was also proposed. --Matěj Suchánek (talk) 08:32, 24 November 2020 (UTC)Reply[reply]
Gadgets 2.0 includes a real localization and translation system. Kaldari (talk) 18:38, 8 December 2020 (UTC)Reply[reply]
The Gadgets 2.0 code is pretty much complete. The main part that needed to be finished was the code for migrating old gadgets to Gadgets 2.0, and also just more testing and polish to make sure such a big feature change goes smoothly. Since Gadgets 2.0 never had a Product Manager or QA Engineer those last steps never happened. I bet CommTech could get it finished though! Kaldari (talk) 18:38, 8 December 2020 (UTC)Reply[reply]

Voting

Talk page archiving bot updating incoming links

  • Problem: When a talk page thread gets archived, all incoming links into the thread get broken, making it very difficult to follow old threads, which often contain useful information for current discussions. In the English Wikipedia we have ClueBot III which claims to fix up links while archiving (en:User:ClueBot_III#Keeping_linked), but in reality it unfortunately does this only to a very limited extent (links from other pages). The more important links from the same page (from other threads or from inside the very thread to be archived, as well as links from threads already archived) don't get updated.
  • Who would benefit: All Wikipedians participating in talk page discussions
  • Proposed solution: Enhance ClueBot III or provide a new universal archiving bot fixing up all links while archiving automatically. While detecting links from the same page might be more difficult than to detect links from other pages, it certainly isn't impossible.
  • More comments:

Discussion

  • As an alternative, there's also w:User:SD0001/find-archived-section which very conveniently searches archives when following a broken link. It's available as a gadget on English Wikipedia. I'm not sure something like this belongs in MediaWiki and/or an extension, but we could globalize/localize the gadget to make it more easily available to all wikis. MusikAnimal (WMF) (talk) 14:44, 26 November 2020 (UTC)Reply[reply]
Yeah, thanks for providing the link. Unfortunately, scripts like this cannot be used by people who (must) have JavaScript disabled for security reasons (or are using browsers not supporting it). So, a more general solution is still needed, ideally one which fixes the problem at its root (that is, in the moment when the problem is created), not at a later point in time when it may be difficult to put the pieces together reliably.
--Matthiaspaul (talk) 14:52, 26 November 2020 (UTC) (updated 12:59, 13 December 2020 (UTC))Reply[reply]
Thanks for the link, had no idea this existed. — Bilorv (talk) 00:44, 11 December 2020 (UTC)Reply[reply]

Voting

Yes, this is exactly what this proposal is about. --Matthiaspaul (talk) 12:53, 13 December 2020 (UTC)Reply[reply]

InternetArchiveBot for Wikidata

  • Problem: Dead links are not managed in an automated way in Wikidata
  • Who would benefit: Wikidata and all projects using it
  • Proposed solution: Adapt InternetArchiveBot to Wikidata and add archival URLs to dead links for URL and external identifiers datatypes.
  • More comments:
  • Phabricator tickets:
  • Proposer: Ayack (talk) 10:29, 26 November 2020 (UTC)Reply[reply]

Discussion

Voting

Importing data from IMDb

  • Problem: When creating new pages for actors, directors, etc. much of the information is available in a structuring format on their IMDb page but it is time-consuming to transfer manually to Wikipedia.
  • Who would benefit: Editors creating, improving, or updating pages that use IMDb for information.
  • Proposed solution: A tool that can auto-import IMDb fields (e.g. year, title of movie/show, role, etc.) and a bot that will update articles when there is an addition to their IMDb page.
  • More comments:
  • Phabricator tickets:
  • Proposer: GeneralBelly (talk) 22:23, 24 November 2020 (UTC)Reply[reply]

Discussion

  • @GeneralBelly: There are already Mix-n-match catalogues for IMDb (for example, for actors]; find them all by searching 'imdb' in the catalogue search form), to help with the importing part of this. Is that sufficient? The work then is to integrate this data into Wikipedia articles, the technical functionality for which already exists. —Sam Wilson 03:18, 25 November 2020 (UTC)Reply[reply]
Hi Samwilson, I'm not familiar with Mix-n-match but I had a look at it and couldn't figure out a way to use it. Basically, I was hoping for a tool where I could identify an IMDb entry for an actor, e.g. Michelle Yeoh https://www.imdb.com/name/nm0000706/ and it would give me a table of her work in wiki markup which I could add directly to the article. Is that possible? --GeneralBelly (talk) 16:27, 30 November 2020 (UTC)Reply[reply]
  • @GeneralBelly: Sounds good. Mix-n-match helps with getting the actors' and movies' details into Wikidata, from where they can be added to Wikipedia articles. This can be done dynamically with Lua in some cases (for individual items) or en masse via a bot such as {{Wikidata list}}, or via wikitext exports. The details can be figured out after voting. —Sam Wilson 04:03, 1 December 2020 (UTC)Reply[reply]

Voting