Research talk:Vandal fighter work load

This is really fantastic work. I'd like to put this together with the work that Stuart's been doing so we can get a full picture of reversions. I have a few ideas I'd like to share -- I'll stop by early next week. Howief 22:40, 1 July 2011 (UTC)

Sounds great. Staeiou and I have already started picking out Huggle and Twinkle edits to be able to tie the pictures together. --EpochFail 23:27, 1 July 2011 (UTC)

Yes, nice work, I think I can put some context on this. But in "I also performed the analysis with raw number of revertes" should that be reverts or reverters? WereSpielChequers 09:14, 4 July 2011 (UTC)

"Reverts"... I just made the edit. Thanks for catching that. --EpochFail 16:34, 8 July 2011 (UTC)

Pattern

This work excludes bots, and the story of vandalfighting in recent years has been the rise of the editfilters and vandalfighting bots so that less manual reversion of vandalism is needed today than it was a few years ago. I suspect that what we are seeing is the crowding out of the watchlisters and other editors from vandalfighting as the recent changes patrol and the hugglers become more and more efficient at reverting vandalsim - certainly that has been my experience. A couple of years ago I could easily find vandalism to revert just by looking at my watchlist, but today I need to run elaborate reports to find vandalism that has snuck past the recent changes patrol. If that is the case then we should see that the average time to revert vandalism has continued to fall - and if so that implies that wikipedia is getting increasingly healthy at least in the way it handles vandalism. This may boost the theory that we have plenty of potential editors out there, they just don't get as many opportunities to fix vandalism as they used to. But it would be interesting to see if the remaining vandalfighters are relatively longstanding editors compared to the community in general, and if the ones who've remained are the grumpier ones or if there is a tendency for those who revert vandalism to get more cynical about newbies over time. WereSpielChequers 09:25, 4 July 2011 (UTC)

This would be interesting for a follow-up. Gathering the mean time to revert over time would be a really interesting plot. Sadly, I don't have the data at my finger tips, but this is one I could probably produce pretty easily or with a subsequent sprint. --EpochFail 16:34, 8 July 2011 (UTC)

Looking at the 22 non admins in the 2007 batch I found 9 former admins, so in those days the core group of vandalfighters were admins or possibly about to become so, in 2007 the first non-admin I came to was in 14th place. I doubt if many of these top fifties have made admin after they did their vandalfighting stint, I recognise at least one unsuccessful candidate from RFA though it is possible there are more as I don't remember every RFA. By 2010 we see only five admins in the top 13, so what seems to be happening is that the most active vandalfighters are burning out and being replaced by non-admins. WereSpielChequers 07:06, 12 July 2011 (UTC)

reversion of non-vandal edits

My experience is that the community or at least some editors have become less and less tolerant of unsourced amendments. My suspicion is that if you are looking at reversion of other people's edits you will see a rising proportion of "revert unsourced" type edits, especially if you are ignoring bot reversion of edits. The assumptions behind this particular research may miss this.WereSpielChequers 11:12, 4 July 2011 (UTC)

Can you give me an example of how this would affect the findings? --EpochFail 16:35, 8 July 2011 (UTC)

Well redirect fixer was as I understand it a bot that fixed redirects rather than reverted vandalism. But there is a broader issue, vandalism and especially the proportion that gets past the vandalfighting bots is a very different thing to unsourced edits. Vandalism is badfaith editing and you are encouraged to use Rollback to deal with it. Unsourced edits are goodfaith edits and you aren't allowed to use Rollback on them. My suspicion is that the very active vandalfighters deal with obvious vandalism and couldn't achieve their editing figures if they were also doing manual reverts of unsourced edits. So I suspect if you filtered out the revert unsourced type of reversions you would mostly lose more of the occasional reverters from the analysis - and that would further exaggerate the picture of a declining cadre of vandalfighters, who so far have managed to keep energising enough replacements into the core of very active vandalfighters to keep up their incredibly high service standards. WereSpielChequers 06:57, 12 July 2011 (UTC)

workload

Measuring workload can be complex - the hugglers who look at the latest changes are going to perceive things very differently from the watchlisters who look at a particular set of article every few hours or days. Having discussed this at the London meetup with a couple of hugglers, I suspect that the hugglers are evaluating things both in terms of the amount of vandalism they find in a particular session, and also in terms of the number of times they are ready to logoff, but another huggler hasn't arrived to relieve them. And it is the latter scenario that is likely to happen if we have a decline in both the amonut of vandalism that is getting past the bots and in the number of vandalfighters. WereSpielChequers 11:49, 4 July 2011 (UTC)

In any setting where these results are presented, this point should be made clear. Using the revert as a measurement of work has the flaw that it can be temporally biased (a revert used to take longer to do than it does now). On the other hand, if reverts take less time to do now than they did before, this would suggest that my results are still sound (that time spent reverting per editor is going down), but that the reduction might be more extreme than I think. Does that sound right? --EpochFail 16:38, 8 July 2011 (UTC)

The reduction in workload may well be more extreme as the individual hugglers will tend to get faster kit and more practice. But I suspect that a different dimension of this is the need for a group of volunteers to maintain 24/7 365.25 cover of the pedia. My suspicion is that the fewer the vandalfighters the more times someone stays on an extra hour or three until they are relieved. If you cover a position with full timers you need 6 people by the time you allow for breaks and holidays, with volunteers you need far more because they work when they want and for as little or as long as they want. So my prediction is that the stress comes from maintaining the 24/7 coverage not keeping up with the vandalism when they are huggling. WereSpielChequers 18:52, 8 July 2011 (UTC)

RFA

One thing that I'm curious about is the interconnection on EN wiki between the changes at RFA and the antivandal patrollers. Since the unbundling of Rollback in early 2008 "Good Vandalfighter" has no longer been enough on its own to get an editor through RFA, and one of our most active hugglers retired a few months ago after rejection at RFA. Current expectations at RFA are that however accurate and active a vandalfighter is, they also need to have contributed content; It would be interesting to se how this is altering the vandalfighting community and whether it is indeed becoming evermore dependent on a few vandalfighting admins who got through RFA before the drought of the last three and a quarter years. WereSpielChequers 11:49, 4 July 2011 (UTC)

Former bots

I notice http://en.wikipedia.org/wiki/User:DASHBotAV is in two of the top fifties, and http://en.wikipedia.org/wiki/User:Redirect_fixer is in another. Is it possible that this work only screens out currently flagged bots, or even relies on most bots ending their usernames with "Bot" if so you might want to screen out those with bot flags andthese. Also there are a few Wikimedians whose usernames predate the rule reserving "bot" for usernames that are bots. WereSpielChequers 14:38, 6 July 2011 (UTC)

Thanks for catching those. My approach to removing bots is to use the list you provided, the currently flagged bots and usernames that might violate the username policy by looking like bots. This isn't perfect, but since my work is focused on looking at the relative workload of the top 50 non-bots, I think the minuscule error should wash out. With that in mind, I have no idea why DashBot is still there. If we go (more) public with these lists, I'll perform the cleanups you suggest and do some spot checking to make sure there aren't more. --EpochFail 16:46, 8 July 2011 (UTC)

Removing a few of these that are bots would only make the message stronger, so though it is worth doing and I'd like to put this in signpost, I don't think that overlooking one or two minor bots greatly alters the picture. However redirect fixer as I understand it was cleaning up redirects not reverting vandalism, so I wonder if that means that possibly more of your vandalfighters were doing something other than revert vandalism? WereSpielChequers 18:41, 8 July 2011 (UTC)

The top vandal fighters are based on reverts (for vandalism or otherwise). If revert fixer is doing edits that look like reverts, they will be caught. --EpochFail 21:24, 12 July 2011 (UTC)

OK thanks for the explanation. I'm not sure if any other bots that revert non-vandalism would get included in your non-bot vandalfighters, but anecdotally reversion of goodfaith edits has been increasing, so counting those in with vandalism may be giving us a distorted picture. I wonder how different this would be if you were able to only measure non-bot reversion of vandalism? WereSpielChequers 11:06, 13 July 2011 (UTC)

dupe

File:Vandal fighter activity.2010.no bots.png and File:Vandal fighter activity.2009.no bots.png look remarkably similar, any chance of some stray timewarp effect? I was wondering why Marek69 hadn't appeared but I suspect he would only be in one of these years. WereSpielChequers 12:57, 7 July 2011 (UTC)

Good catch. I suppose that is an uploading error on my part. I'll have a look. --EpochFail 16:47, 8 July 2011 (UTC)

It turns out that I uploaded the right image, but the server cache isn't updating. --EpochFail 16:52, 8 July 2011 (UTC)

Still trying to fix this. --EpochFail 16:52, 8 July 2011 (UTC)

The monkeys inside the servers are presenting the correct image now. Marek69 is present. :) --EpochFail 16:54, 8 July 2011 (UTC)

Pattern

Apart from the 2009/2010 similarity the other years seem to show transition at the top with the most active vandalfighters having similar dominance but there being considerable turnover between the 5,000 plus reverters and the rest of the reverters. Are we seeing an element of burnout here? I'm certainly aware of some people who were at the top of those charts and who no longer edit. WereSpielChequers 12:57, 7 July 2011 (UTC)

Admins/nonadmins

I think it might be instructive if we could subdivide the most active vandalfighters by wikigeneration and also by whether or not they are admins. A wikigeneration analysis would tell us whether we are still recruiting into this group, and an admin/nonadmin analysis would tell us the impact that the RFA drought is having on this (admin vandalfighters can block editors instead of just reporting them to AIV so it makes a big difference whether or not they have the mop). WereSpielChequers 12:57, 7 July 2011 (UTC)

The first pass at this gives us 22 non admins in 2007, 26 in 2008, 27 in 2009 and again 27 in 2010. but 9 of the 22 in 2007 are former admins. I think I see a pattern here. In 07 the most active non-admin vandalfighter was in 14th place but nowadays many of the most active are non-admins. So the core of very active vandalfighters is renewing itself, but there has been an increased dependence on non-admins. Whether you are an admin or not makes no difference to the reversion of vandalism, but it does make a big difference when blocking vandals and some vandals don't stop till they are blocked. So we have three potential stress thresholds to consider. Whether we have continuous Admin cover at AIV, whether we have continuous huggler cover at recent changes and whether we have sufficient hugglers on line during the busy periods. Any one of these could cause stress, and with the declining number of active admins I'd be particularly concerned at the coverage of AIV - if there is delay there it could be very stressful for non-admin hugglers. WereSpielChequers 11:01, 13 July 2011 (UTC)

Who is busier?

My reading of this and my discussions with other editors lead me to suspect that we have several patterns here. Firstly some vandalfighters are definitely less busy as vandalfighters (though they may be spending time doing other things on wiki), if like me you fight vandalism using your watchlist you are likely to be reverting less vandalism than a couple of years ago. I suspect the pattern is that the hugglers and others at recent changes are picking up a higher proportion of vandalism, leaving less vandalism for others to find on their watchlists. A time analysis of vandalism reversion would probably confirm this by showing that we are picking up vandalism more quickly. I can still pick up some new vandalism that gets past the recent changes team by using tools like poop patrol, but for vandal fighters who aren't looking at recent changes the pickings are thin, and that of course is very good news. Where things are busier are at recent changes and I suspect at certain hours there is likely to be a mismatch between our hugglers and the amount of vandalism. Recent changes patrol is 24/7 and we need to cover every minute of the day. With a volunteer team we can't be sure that our volunteers will turn up at the times when they are most needed, hence the userboxes, IRC and so forth. To confirm the theory that the recent changes patrol is busier it would be interesting if someone had a tally as to how the frequency of IRC and other calls for reinforcement has changed over time. Or indeed if longterm hugglers could say whether they find themselves busier in terms of the number of times they are huggling and waiting to be relieved. WereSpielChequers 14:39, 12 July 2011 (UTC)

Checksum

Why did you use regular expressions instead of checksum to identify reverts? If I remember correctly, the statistics by Erik Zachte showed that using checksums is easier and gives different (better) results. Moreover, it would be easier to generate statistics for other language editions and projects. --Nemo 14:09, 17 September 2011 (UTC)