Wikimedia Meta-Wiki

Community Wishlist Survey 2022/Admins and patrollers/Expose ORES scores in AbuseFilter

< Community Wishlist Survey 2022‎ | Admins and patrollers

Expose ORES scores in AbuseFilter

  • Problem: AbuseFilters are a great way of preventing problematic edits before they happen. However, guessing "problematic" is currently done using user segmentation, common phrases used by vandals etc. We have a much better tool to determine if an edit is destructive: ORES. If we were able to prevent all edits above a certain threshold, the workload on patrollers would be significantly reduced and, possibly, would prevent some communities from requesting and all-out IP-editing ban.
  • Proposed solution: Expose the raw ORES "damaging" score as a variable in AbuseFilter
  • Who would benefit: Patrollers and admins would have less work
  • More comments: Exposing ORES levels from the Special:RecentChanges interface (very likely, likely, less likely, unlikely) would also be OK.
  • Phabricator tickets: phab:T123178
  • Proposer: Strainu (talk) 16:13, 11 January 2022 (UTC)[reply]

Discussion

  • I'm not an ORES architecture expert, but I think this would be a major timing issue. AF has to be real-time to work, having to wait for ORES processing would likely be a huge bottleneck on every edit/action made - since between clicking publish and your save committing the data would need to go in to and back our of ORES, then in to AF before AF can do anything with it. — xaosflux Talk 19:21, 11 January 2022 (UTC)[reply]
    Since all edits pass through ORES, this should not increase global processing time. If you’re assumption is right, this feature would require to queue edits (so they could take several seconds to be live), but it is not blocking in my opinion.

    Related feature request: Suggesting AbuseFilter by machine learning. Pols12 (talk) 19:45, 11 January 2022 (UTC)[reply]

    @Pols12: AF is an interrupt, it will prevent saving an edit or present a warning to the user - it can't wait for ORES to process and also still do this. We're not going to leave our user sitting at a "processing" screen after they click publish - and once they left that screen it is too late to present them a warning. Now perhaps AF could add scores that ORES could use for things like deferred edits, but the reverse doesn't seem feasible. — xaosflux Talk 22:49, 11 January 2022 (UTC)[reply]
    We have to increase the processing speed on the ORES, but machine learning is still possible. Thingofme (talk) 00:39, 12 January 2022 (UTC)[reply]
    The bigger problem is that right now ORES is working on revisions, not diffs. But since ores is good enough for en.wp RC stream, I expect latency issues only for the biggest changes. But if I'm wrong, maybe other solutions, such as auto-revert, can be considered. Strainu (talk) 05:11, 12 January 2022 (UTC)[reply]
    @Strainu: Am I missing something? Doesn't abusefilter processing occur prior to a revision being created, and ORES happens after a revision is created? — xaosflux Talk 14:09, 12 January 2022 (UTC)[reply]
    You're not missing anything, that's what I also meant by "ORES is working on revisions, not diffs". I suspect making an ORES API that can receive a diff will be the main software change of this project.
    However, I don't expect this to bring along any latency issues. If ORES can work in near-real-time at en.wp scale, likely it can also be scaled to handle (in the worse case scenario) double the requests. Note that everything will happen locally (i.e. the same datacenter). A few hundred milliseconds of additional delay seems acceptable to me. Strainu (talk) 15:54, 12 January 2022 (UTC)[reply]
    Well, I have heard from the Performance team in the past that AbuseFilter is one the biggest slow-downs and causes of concern. I strongly suspect even a few hundred milliseconds is asking too much. As Strainu says, ORES would need to be first be able to accept raw content rather than just a revision ID. That alone I think makes this proposal out of scope for Community Tech, but it could deserve a spot in the Larger suggestions category (intended for things we can't do or promise, but still should have visibility to the broader movement). MusikAnimal (WMF) (talk) 23:25, 12 January 2022 (UTC)[reply]
    @CAlbon (WMF) I'm not sure if you're the right person to ping about ORES, but if not maybe you know who we could talk to? I'm trying to find out whether this proposal is feasible. The questions are:
    • Is it possible to give ORES some wikitext and it gives us a score (i.e. before the edit has been saved)?
    • If it is not possible, how hard would it allow ORES to accept arbitrary wikitext rather than a revision ID?
    • In either case, can ORES be potentially a bit slow? Several hundred milliseconds, or longer?
    Thanks for any information you can provide! MusikAnimal (WMF) (talk) 17:15, 13 January 2022 (UTC)[reply]
    The primary problem is speed, and I strongly believe the lack of speed is from I/O. When a request for a prediction for a revision ID is received, ORES hits the mediawiki API to get the wikitext, parses it, converts it into a feature vector, then serves it to the model to get a prediction. That is slow, obviously. Right now that slowness is hidden by pre-caching scores.
    That said, with some changes to we could deploy a version of the model that accepts wikitext and scores it. I'll create a ticket for that and we can do a spike on it. CAlbon (WMF) (talk) 17:05, 18 January 2022 (UTC)[reply]
    Ticket created! https://phabricator.wikimedia.org/T299436 CAlbon (WMF) (talk) 18:25, 18 January 2022 (UTC)[reply]
  • This or similar would be excellent. --Izno (talk) 00:18, 17 January 2022 (UTC)[reply]

Voting

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Community_Wishlist_Survey_2022/Admins_and_patrollers/Expose_ORES_scores_in_AbuseFilter&oldid=22816721"