Grants talk:Programs/Wikimedia Community Fund/Rapid Fund/UTRS User Experience Development (ID: 22215192)

Comments from I JethroBT (WMF)

Hello AmandaNP, and thanks for this proposal to address needs in the UTRS system related to translation, ACC transfers, and continuous appeals. I'm only a little familiar with UTRS as an admin on enwiki in my voluntary role, and so it's helpful to see where you (and others) are seeing challenges for admins and blocked users needing to interact with the system. I appreciate that you are hoping that this development work would make important improvements in the accessibility and functionality of certain actions and review functions. I've approved the proposal with the following comments and questions:

• You have great metrics for the ACC transfer- and appeals-related outcomes, so thanks for thinking about these areas. I have a few other suggestions for the evaluation plan, but wanted to hear your feedback on them:
  • Given that translation is featuring heavily in one of the main tasks, could an outcome be to complete 1-2 translations of key documentation (e.g. say, Portuguese, because it's been heavily requested?)
  • The proposal starts with a broader goal to improve the process and hoops users have to jump through so it’s less time consuming for both administrators and editors. Is it possible to formally or informally follow-up with users (admins or blocked editors using UTRS) to assess if the changes to particular functions in UTRS feel easier / better / faster to use? This kind of information would help speak to the specific experiences of people regularly using UTRS. Maybe something like this could be added to the survey proposed?
• I'm not sure exactly what the work around the audit of tools messages would look like-- could you describe this audit further? Once the audit is done, do you think it will be plausible for UTRS to be translated more broadly to other Wikimedia projects?

When you're able, please respond to my questions above, and thanks again for your proposal! Take care, I JethroBT (WMF) (talk) 16:50, 6 August 2023 (UTC)

So the goal of the translation side is not to actually complete the translations (as I don't have the skillset), but to finalize the interface messages into a translatable format, and move to a new translation platform if needed to garner more translations (like translatewiki.net). I am also considering getting auto translation of user input, to the default language of their browser or section - though this would require a subscription and may be subject to a future grant request. It would allow better understanding for it to be clear people are able to communicate in their native language (if supported by the subscription).

In terms of follow up, admin follow up is good. We actually had feedback just the other week on w:WT:UTRS as this was still ongoing. Informal questions there could definitely work. As far as users go, I'm very hesitant to consider any contact with blocked users. They are blocked users for a reason, and if they are accessing UTRS, it's because they want unblock. I already decline to handle anything that is not a technical issue that they try and send by email as in no way am I a "supervisor" for these requests. Beyond that particular people, especially LTAs continuously appealing will subject such as survey to a full level of abuse. I think the better way to review is to gauge by statistical changes. And I could make it my first week to determine some KPIs that would be relevant.

As I slightly mentioned above, there are system messages (similar to "My watchlist" at the top of this page) that are not able to be translated because they are not listed in the translation file. I would need to review all pages of the interface using the system messages language, and test different states to ensure everything is ready for translation by others or automation - depending on which is available. That's it. -- Amanda (she/her) 04:25, 7 August 2023 (UTC)

@I JethroBT (WMF): While thinking further about things this week, I have planned efforts to try and better log analytics that come out of my grant work. I have created a task to make an analytics dashboard, and requested that ACC provide results of transfers back - though that may not be immediately implemented. -- Amanda (she/her) 17:50, 17 September 2023 (UTC)

@AmandaNP: Thanks for this update! The idea to build this dashboard is a great approach to showing what impact your work may be having. It's fine if the results from transfers aren't prioritized right away as you've said, I think there's plenty of other indicators on the dashboard here to support good evaluation. I JethroBT (WMF) (talk)