Grants:Project/Rapid/Strainu/Enhance and secure the future of Romanian projects' critical infrastructure

statusfunded

Strainu/Enhance and secure the future of Romanian projects' critical infrastructure

Ease the pressure on experienced members of the Romanian communities by developing new tools and externalizing the infrastructure they run on to the WMF.

targetall existing Romanian projects

start dateMarch 15

end dateSeptember 15

budget (local currency)2431.14€

budget (USD)2720

grant typeindividual

grantee• Strainu

contact(s)• Strainu

give feedback

friendly space expectations

ucc

privacystatement

Review your report

Project Goal

Briefly explain what are you trying to accomplish with this project, or what do you expect will change as a result of this grant. Example goals include, "recruit new editors", "add high quality content", or "train existing editors on a specific skill".

This project's goal is to improve and develop the code for the critical tools I maintain and deploy them on the Wikimedia infrastructure in a way that allows them to run unsupervised for long periods of time (e.g. years)

By critical tool, I understand bots that have taken over maintenance tasks and which the community is relying upon to free up time for other activities. This project does not include user scripts or other code running on-wiki; while that code is also critical and needs to be maintained, the infrastructure it runs on is stable and they are extensivly tested because of the Wikipedia scale. It also does not include other robots run by different members of the community for individual projects, such as the monuments bot, settlment bot etc. as those are often tweaked and improved, needing constant oversight.

Since 2016, the number of daily users of the Romanian Wikipedia has gone up from 80 to 95, with the number of daily edits also going up from a low of 791 to 951 edits per day in 2021 [1]. However, this has not brough up a new generation of community leaders. In the same period, the number of administrators has decreased by 2 [2] and the number of technical administrators is just 2, both with over 16 years of wiki experience.

In the same period, the number of automated tools has increased and there are now 6 tools that I deem as critical infrastructure, out of which I run the last 4:

Notifier for users who ask for an unblock (shared a lot of code with the previous one)
An archiver bot
Notifier for users responding to image warnings
A bot which converts old diacritics into new ones (this is specific to Romanian language, see ro:WP:DVN).
A bot which automatically lists for delition articles who's notability has been questioned for over 3 months
A proposed automatic reverter based on ORES

The tools listed above have run until now on the private servers of the technical administrators. However, in November, a hardware failure has brought down the archiver and unblock notifier for over a month, causing issues for some users because of the large size of the discussion pages. Also, the automated reverter, because of the large estimated footprint (over 1% of all edits on ro.wp) needs to be especially reliable both as code and as infrastructure.

Note about the reverter bot: While I'm aware that another reverter bot exist in the community (e.g. en:User:ClueBot NG), its training and maintenance is a 3-person job, which cannot be replicated for ro.wp. Instead, I plan to use the ORES score, which is readily available from the MediaWiki API. At the end of the score spectrum, ORES is excellent at predicting reverts (over 95% of all patrolled edits with an ORES resolution of "very likely damaging" have been reverted on ro.wp in the last month - details can be found, in Romanian, in the table from the bot request).

Project Plan

Activities

Tell us how you'll carry out your project. What will you and other organizers spend your time doing?

Planning

In the planning phase, I will consider the feedback received on the revert bot RFB discussion, as well as identifying the correct hosting solution from WMF (ToolForge or CloudVPS). This phase will be done before the grant begins.

The planning for the upgrades to the other bots is already done - they are detailed in the following sections.

Writing code

All coding activities include writing the code, the tests and locally testing it. The following activities are planned, together with the estimated hours they will take:

Writing the revert bot according to the bot request and comments revieved: 40h
Creating a notification framework that allows following the bot status on-wiki. This will include a semaphore-like status page for all the bots using it, including the notifier itself: 4h
Update the diacritics converter to allow several actions, such as notifying instead of moving. This is more appropriate to the current status of the Wiki: 2h
Update the deletion proposal bot with several deletion approaches. The current approach creation deletion requests for all the articles with the notability tag from a certain month at once. This crowds the proposal page and discourages people from participating in discussions: 4h

Deployment to Wikimedia Cloud Services

Depending on the solution for hosting chosen (ToolForge or CloudVPS), there might be some variation around the timings mentioned here. The variation has been include in the "reserve" budget line.

Creating the needed environment on the new hosting: 2h
Deploying each of the robots to the new hosting: 4x0.5h

Analysis and bugfixing

All code has bugs, and even if 3 of the 4 tools have been running for a few years, I expect that after deployment I will encounter various setup issues. Normally, the ratio between development and bugfixing is 1:1 or worse, but a lot of the effort is already included in the development tasks (which also include unit-tests). I estimate the remaining effort to 20% of the development time:

Bugfixing and statistics gathering for the revert bot: 8h
Bugfixing for the rest of the tasks: 2h

Documentation and dissemination

It's useful to have automation, but if nobody knows about it, the automation will quietly die when the maintainer will be unable to work on it. I therefore intend to reorganize ro:Proiect:Cod Wikipedia and document how to re-enable the tools from existing source-code.

Also, the code I develop could be useful to other communities. I plan to re-request bot flags on the rest of the Romanian project in order to be able to run the diacritics and revert bots.

For communities except Romanian, I will submit presentations to regional conferences happening in 2022, such as the CEE Meeting.

How will you let others in your community know about your project (please provide links to where relevant communities have been notified of your proposal, and to any other relevant community discussions)? Why are you targeting a specific audience?

The bot request for the revert bot is here and has been closed on Jan 31st. I also announced this grant request in the same section. I will communicate the changes to the bots at the village pump as the project progresses.

What will you have done at the end of your project? How will you follow-up with people that are involved with your project?

As described in the sections above, by the end of the project, all critical bots (including the new Revert bot) used on Romanian projects will be running from the Wikimedia Cloud Infrastructure is a way that ensures unsupervised running for years. All generated code will be published under a free license in repos at the rowiki organization on github

Are you running any in-person events or activities? If so, you will need to complete the steps outlined on the Risk Assessment protocol related to COVID-19. When you have completed these steps, please provide a link to your completed copy of the risk assessment tool below:

No

Impact

How will you know if the project is successful and you've met your goals? Please include the following targets and feel free to add more specific to your project:

Functional revert bot
Functional tools dashboard on Wiki
Number of tools running from Wikimedia Cloud Services: 4
Updated documentation at ro:Proiect:Cod Wikipedia

Another personal target is to have a presentation at the CEE Meeting. However, I did not include that in the project time-line as it would extend it with almost 3 months (or 50%) without any other activity planned in that period.

Resources

What resources do you have? Include information on who is the organizing the project, what they will do, and if you will receive support from anywhere else (in-kind donations or additional funding).

The code for the existing bots has already been written and is publicly available under free licenses at https://github.com/rowiki/wikiro/tree/master/robots/python

What resources do you need? For your funding request, list bullet points for each expense:

I mostly need funding to allow me to dedicate time to developing the described tooling, as detailed in the following table.

Activity	Hours	Rate	Total ammount
Planning	0^*	$40^***	$0
Development and maintenance of the revert bot	48 h		$1920
Development and maintenance of the existing bots	12 h		$480
Setting up the cloud instance	4 h		$160
Documentation and dissemination	0^**		$0
Reserve fund	4 h^****		$160
Total			$2720

^* This will happen before the project start and is thus not included in the budget

^** This will be done during my volunteer time, regardless of the project funding

^*** The average rate for a programmer in Romania is between $31 and $51, higher in Bucharest and Cluj

^**** Software development estimates are inherently inexact - bugs can appear, new requirements can come up. I have requested a limited reserve fund to cover for such cases

Endorsements

Strong support. --NGC 54 (talk / contribs) 01:20, 26 January 2022 (UTC)
Strong support. -- Accipiter Gentilis Q. (talk) 18:16, 4 February 2022 (UTC)
Support This is a promising tool that would help a great deal the patrollers on rowiki (it will not magically solve all problems but will set a foundation for eventual new anti-vandalism tools). Based on Strainu's programming skills I'd give Strong support. My personal request to the developer @Strainu: please make sure you write a comprehensive report for this grant – even in the improbable event that nothing came out of your code. I saw an expensive grant request on a similar theme that has been funded a couple of years ago and the report was basically nothing. (I won't point fingers, but I privately let Strainu know what grant I am talking about.) On a similar note: AFAIK requesting a grant for paid work on a tool is a first for rowiki – as a result, I anticipate some resistance from the community. An honest progress report from Strainu is unlikely to change the hearts of the opposing people, but will place (leave) Strainu in the right. Good luck! Gikü (talk) 11:07, 7 February 2022 (UTC)