Grants:IdeaLab/Bot to detect and tag advocacy editing

Bot to detect and tag advocacy editing
A bot that flags "advocacy edits" (broadly defined)
idea creator
Jytdog
developer
Esquivalience
this project needs...
volunteer
join
endorse
created on19:39, 7 March 2016 (UTC)


Project idea edit

What is the problem you're trying to solve? edit

Maintaining the integrity and reliability of content.

What is your solution? edit

A bot that flags "advocacy edits" (broadly defined). This should build on other bots, like en:User:COIBot. See the similarly aimed Grants:IdeaLab/Fast and slow new article review, which is directed more to new article reviewi than individual edits, as this one is.

Goals edit

  • Identify criteria defining an "advocacy edit" (an edit that reflects advocacy broadly defined - that promotes or denigrates something or someone)
  • implement criteria in a bot
  • train the bot
  • test the bot
  • get consensus to release the bot and if obtained, release it

Possible additional ability of bot

  • see if the bot can look at other edits by an editor whose edit was flagged, and can further identify a pattern of advocacy editing
  • discuss in the community what we would do with such a bot
  • if we can get consensus on how to use it, implement it

Get Involved edit

About the idea creator edit

I work on editing content about health in Wikipedia, and on a broader level work on conflict of interest/advocacy issues. I have a set of mental "filters", or criteria you could say, that flags "advocacy editing" (promotional or denigrating) in articles I monitor, as I review changes to articles on my watchlist. If that flag goes off, I look at contribs by the editor and see if there is a pattern of advocacy editing, and if I find that, I open a discussion with the editor on his or her Talk page. I wonder how translatable my criteria (which I have reflected on and are somewhat articulate-able) are to a bot that could flag edits that express advocacy - that violate the en:WP:PROMO and/or en:WP:NPOV content policies.

Participants edit

  • Developer I can assist in developing the bot. I have already started working on a similar bot that uses simple machine learning techniques to detect promotional new pages. Esquivalience (talk) 01:16, 15 March 2016 (UTC)

Endorsements edit

  • Endorse We needed this years ago! There were previous discussions here (and elsewhere in the thread) and here. There's no shortage of edits to train it with, but we need somebody who can do the difficult part of writing a program to score diffs and new articles. It seems to be beyond our volunteer capabilities. Smartse (talk) 21:09, 7 March 2016 (UTC)
  • Endorse This would assist the current community at en:WP:COIN. Doc James (talk · contribs · email) 00:12, 8 March 2016 (UTC)
  • This is really useful. We have plenty of articles like this. Indeed, we might fall in this type of articles sometimes when writing about a close topic/project. Nicoguaro (talk) 00:41, 15 March 2016 (UTC)
  • Endorse I think that with some work turning User:Jytdog's criteria into features (en:feature engineering) and gathering representative set of "advocacy"/non-"advocacy" edits (en:supervised learning), we could stand up a classifier prediction model that would be relatively high fitness in m:ORES so that anyone could make use of it. I'd be happy to support anyone who wanted to do this engineering work with the revscoring library. I have some examples (see feature engineering & basic classifier training) that should help get us started. --EpochFail (talk) 03:29, 15 March 2016 (UTC)
  • Endorse Agree, this could be useful as a multiplier of human effort for the COIN volunteers and others. Please note my suggestions about what might be useful classifier cues at Phabricator T120170. Brianhe (talk) 07:54, 20 March 2016 (UTC)

Expand your idea edit

Would a grant from the Wikimedia Foundation help make your idea happen? You can expand this idea into a grant proposal.

Expand into an Individual Engagement Grant
Expand into a Project and Event Grant