Talk:Incident Reporting System

Feedback on the Incident Reporting System Minimum Testable Product in Beta – November 10, 2023

Hello, thanks for testing, please proceed to give your feedback. You can use the following questions as prompts:

• What do you think about this reporting process? Especially what you like/don’t like about it?
• If you are familiar with extensions, how would you feel about having this on your wiki as an extension?
• Which issues have we missed at this initial reporting stage?

–– STei (WMF) (talk) 12:02, 10 November 2023 (UTC)

I just tried to test it, but I only got the option from Slide 2 [1] (report button in the tools menu). There was no overflow button even though I have discussion tools enabled (I'm using Vector 2022).

I like the interface, but it took a little bit to figure out not to write „User:Testing“ at „The user you would like to report“ but instead just write the username.

When I try to submit, it says „Your request could not be submitted due to an error. Please try re-submitting“, is that on purpose? According to your update page even this MVP is sending an email to T&S, but when opening the report tool it says „Your report will be sent to an admin to review.“

I know it's just an MVP, but is it possible to connect this tool with gadgets like Adiutor or en:WP:Twinkle so that users can decide whether to report UCoC violations privately or on the dedicated community pages, depending on the severity of the behaviour? Johannnes89 (talk) 13:02, 10 November 2023 (UTC)

Hi! Thanks for trying this out. Can you link to the page where you couldn't see the overflow button? I'd like to check what's going on there.

When using DiscussionTools (so the report button under the overflow) the tool should actually automatically fill in the username you'd like to report for you.

The error message is not on purpose, it should only appear if something is wrong. Maybe your email is not confirmed? You should be logged in and have a confirmed email address for this to work.

"Your report will be sent to an admin" is the language that we'd like to use when the tool will actually be live and the reports will go to the right entity that should process them. Right now since this is testing only the reports just go to the Trust and Safety Product team.

Thanks for the feedback re: Auditor and Twinkle. Ideally we would want this to act like a routing tool where depending on the type of report, it gets directed to the right channel. How the decision is made on what report should go where is a discussion that we need to have. MAna (WMF) (talk) 14:35, 10 November 2023 (UTC)

Hi @MAna (WMF), I was trying both User talk:Testing and Talk:African wild dog, my email is confirmed (I checked by writing someone a wikimail) and I am logged in.

I got an idea and left a comment on Talk:African wild dog – after saving the edit, the overflow button appeared (but still no overflow button at User talk:Testing where I didn't leave a comment).

Using the overflow button, I could successfully report STei. But using the report button on the same page it was still not possible to report STei or anyone else, still the same error message.

While I was writing this, after trying the report button and reloading the page a couple of times, the overflow button disappeared again. Johannnes89 (talk) 17:21, 10 November 2023 (UTC)

I just tried the tool on this page [2] and the overflow button only appeared after leaving the second comment and disappeard when I reloaded the page (in the meantime I was able to report my comment using both the overflow button and the report button).

Then I returned to Talk:African wild dog again, left a new comment and the overflow button appeared again on that page as well (after reloading it disappeared again). But for some reason now both the overflow and the report button showed the error message again. Johannnes89 (talk) 17:33, 10 November 2023 (UTC)

@Johannnes89, hi there. Thanks for testing this tool out and giving feedback. Can I ask if you have checked the box next to DiscussionTools on Spezial:Einstellungen#mw-prefsection-betafeatures? WBrown (WMF) (talk) 18:07, 10 November 2023 (UTC)

I ask this as the dewiki beta needs this checked for the overflow menu to show in all cases (as I have discovered in looking into your issues reported here). If you don't have this checked, could you try again and see if that issue is fixed?

The other issue you report is probably because you have submitted too many reports. The rate-limit is quite restrictive at the moment, and a generic error message is shown currently if you hit the rate limit. This means that if you have a new account on betawikis, you will only be able to submit 1 report a day per wiki.

If you feel comfortable doing so, the reason for the failure should be present in the response to the API request made by submitting the dialog which should be visible in 'Inspect element' (otherwise known as developer tools). Knowing this will help me diagnose the problem further. WBrown (WMF) (talk) 18:21, 10 November 2023 (UTC)

Ok I did some further testing across multiple language versions. When I initially reported the error message above at 13:02 UTC, I had discussion tools enabled at Special:Preferences#mw-prefsection-editing but not at Special:Preferences#mw-prefsection-betafeatures.

I changed that when I checked my settings, tried again and responded at 17:21 UTC, so you are right, it's likely the rate limit that was responsible for the error message still being shown. Indeed when testing again at enwiki beta and inspecting the page, it now says 429 (Too Many Requests) which confirms this.

I was confused for a second because I can still report multiple times at dewiki beta, but then remembered that this is due to my admin permissions -> noratelimit at dewiki beta.

I now tried to reproduce the initial error by deactivating the discussion tools at Special:Preferences#mw-prefsection-betafeatures at nlwiki beta and got 403 (Forbidden). Strangely after activating the discussion tools at beta features again, I still got the same error 403 (Forbidden) – I would have expected either to be able to report one time or to receive the 429 error code instead.

By the way: Is there are reason why the tool is currently limited to talk namespaces? Johannnes89 (talk) 19:21, 10 November 2023 (UTC)

@Johannnes89:

• A user can only make a report when they have made at least one edit. It seems you have made no edits on nlwiki beta. The 403 errors should go away once you've made an edit.
• All checks (such as edit count and email address checks) are made before the rate-limit check. This is so that a user who submits a form with invalid data (such as a username that does not exist) can have another go to submit the form.
• This is because on most standard installations, discussions will be kept on talk namespaces. A wiki with this installed could extend and/or modify the list of allowed namespaces as desired.

Thanks for the thoughts and testing, WBrown (WMF) (talk) 12:54, 13 November 2023 (UTC)

@WBrown (WMF) thanks for your explanation! You might want to reconsider the namespace limitation:

• The project namespace (namespace 4) is used in most projects for discussions e.g. discussion places for newcomers [3], general help desks [4], administrators' noticeboards [5], vandalism noticeboards [6] and many more pages (for enwiki e.g. [7]) – these are just some examples of highly used discussion pages outside the discussion namespaces which might require frequent use of the incident reporting system
• Severe UCoC violations can happen in any namespace, e.g. by abusing the edit summary or just misusing any page to harass someone. Some LTA deliberately try to insult Wikimedians or disclose their personal data using random Wikipedia articles (namespace 0)
• Ideally the tool should be working on Special:Log as well (namespace -1) in order to report logs of account/page creations by some LTA containing personal information or other kinds of harassment.
• Similarly there are abusive file uploads (namespace 6)... I could think of many more examples explaining why all namespaces should be covered by the incident reporting system per default

Johannnes89 (talk) 13:32, 13 November 2023 (UTC)

Thanks for the thoughts. Although I cannot speak for definite, I would think that the tool would be enabled on WMF production wikis for the project namespace because of your first point.

The tool itself in theory could be enabled for all namespaces, but I think some testing would be needed to check that it works for special namespaces (-1) as the tool currently sends the latest revision of the page with the report (which doesn't apply in the case of special pages).

I will ask about this with the team. WBrown (WMF) (talk) 14:45, 13 November 2023 (UTC)

Thanks for sharing this, very interesting! I think there's opportunity here to explore different entry points for the incident reporting system. For now we're starting with the most obvious places such as the article and user talk pages. As this progresses we will be thinking about other possible entry points. MAna (WMF) (talk) 18:16, 13 November 2023 (UTC)

I was unable to get the 3 dots method to work but was able to use the Tools menu option.

More substantively: I challenge the idea that the correct scope of an MVP is only private reports. In fact I think one of the most important things this tool could do is help users figure out when they need to make a private report and when they should do a public report and, in either scenario, to help them report it in a way that gives enough information to make it actionable. A huge worry I have is that this tool will overwhelm volunteer capcity to process reports regardless of the size of the project and by making it private only this concern is exacerbated. Speaking only from an enwiki perspective, it's also not true that administrators would see the report as enwiki policy says only Checkusers, Oversighters, and Arbitrators can handle private reports.

Beyond that here's some feedback based on the four criteria from the Enforcement Guidelines:

• ease-of-use: once I was in the tool easy to use. I would suggest that the menu tool be labeled "Report harassment" (vs reporting incorrect information or an edit war) and/or that this label be configurable on each project
• privacy and security: without being able to see the resulting reports I can't really comment on this but is reasonable from what I can see
• flexibility in processing: does not appear to be part of the MVP and beyond what I've commented above I'm not concerned about this not being part of the product
• transparency: doesn't even appear to have been prioritized/considered in this design process.

Barkeep49 (talk) 21:20, 13 November 2023 (UTC)

Thanks for adding your thoughts, this is very helpful. One of the end goals of the tool is exactly what you mentioned: guide the user through making a private or a public report depending on the scenario. We started with private reporting as a first step but there will be explorations around public reporting as we progress.

And yes, I absolutely agree with the worry of overwhelming volunteer capacity to process reports. This is something that we're thinking about. I'd love to discuss more about possible ways to mitigate this (public reporting will help some but I think we can explore more). MAna (WMF) (talk) 13:02, 14 November 2023 (UTC)

Also, when you're thinking about transparency, are you referring only to public reporting or are there other aspects that you think should be covered? MAna (WMF) (talk) 13:08, 14 November 2023 (UTC)

As long as there are no plans to launch in production without the public reporting piece figured out, starting with private feels fine. The UCOC Enforcement Guidelines Revision Committee had drafted the following (non-binding) recommendations which were supposed to have been passed to the team that ultimately worked on this tool. I can't help but wonder given the reorganization that happened at the foundation if this has never reached you. So here it is:

Privacy and anonymity

• Allow reports to be made either publicly (where all details of the case are viewable by the general public), or with varying degrees of privacy (for example, where the name of the reporter is hidden to the public; where the username of any individuals involved in the reported behaviour are hidden to the public; and other potential examples)
• Permit reports to be made whether logged in or logged out

Processing

• Allow reports to be processed privately by whomever is charged with resolving UCoC violations
• Allow reports to be forwarded to relevant bodies; Link current cases to previous cases involving the same recipient of a complaint, including allowing reports made in-person or off-wiki to be linked to ongoing reports of UCoC violations
• Provide a way to integrate or document an in-person report into this same reporting system
• Allow those who are processing cases to filter out bad-faith reports

Transparent documentation

• Provide a way to publicly archive all cases in a searchable manner, while preserving privacy and security in non-public cases
• Assign each case a unique public identifier for the purpose of public visibility
• Allow limited data collection on basic statistics about the use of this tool, for the purposes of reporting out information about UCoC enforcement to the general public, in keeping with our principles of minimal data collection and respect for the privacy of our community members

Barkeep49 (talk) 16:31, 14 November 2023 (UTC)

To answer those questions I can answer as an engineer:

• The text that is shown around referring to administrators can be customised per-wiki (as you would do for a normal i18n message)
• The label used for the report link in the tools menu and DiscussionTools overflow menu link can be customised via the message key MediaWiki:reportincident-report-btn-label.

WBrown (WMF) (talk) 13:08, 14 November 2023 (UTC)

@WBrown (WMF) pardon my ignorance but what's a "normal i18n message"? Best, Barkeep49 (talk) 19:05, 14 November 2023 (UTC)

Apologies for the technical jargon. What I mean by this is a string of text that is present in the MediaWiki namespace, and so can be modified by administrators on a wiki. For example, MediaWiki:reportincident-report-btn-label can be edited to change the text from Report to Report harassment.

i18n in this context is a shortening for internationalisation. Hope thats clearer. Happy editing, WBrown (WMF) (talk) 19:16, 14 November 2023 (UTC)

One more thing: just wanted to clarify that the product that is now available on Beta is not a finished product that we're planning to just push to production. This is a first step of many and we want to use these conversations to identify what are some essential features for this to be a useful working tool. MAna (WMF) (talk) 14:21, 14 November 2023 (UTC)

I don't see the overflow button either on Firefox nor on Chrome. ChristianKl ❪✉❫ 18:19, 14 November 2023 (UTC)

@ChristianKl please try using link [1] or [2] instead.

[1] https://en.wikipedia.beta.wmflabs.org/wiki/Talk:African_wild_dog?dtenable=1

[2] https://en.wikipedia.beta.wmflabs.org/wiki/User_talk:Testing?dtenable=1

Let me know if this helps. Thank you for your time. –– STei (WMF) (talk) 09:31, 16 November 2023 (UTC)

Curious if that works for them given that I can't get the three dots to work (even while I can get the menu option to work). Best, Barkeep49 (talk) 17:21, 16 November 2023 (UTC)

@Barkeep49 and @Johannnes89, thank you both for your patience and comprehensive feedback. Happy holidays! –– STei (WMF) (talk) 12:45, 18 December 2023 (UTC)

It seems it was an issue of having to be logged in with an account. I got one step further and got to the button only to get a message that I need to verify my email address but somehow the email for that doesn't deliever to my gmail. ChristianKl ❪✉❫ 09:45, 19 November 2023 (UTC)

@ChristianKl I will forward this problem of not being able to get email verified. Please check your spam too, while I get back to you. –– STei (WMF) (talk) 15:50, 20 November 2023 (UTC)

@ChristianKl unfortunately there are issues with email delivery on Beta cluster. When the matter is resolved, I will let you know. –– STei (WMF) (talk) 12:03, 21 November 2023 (UTC)

@ChristianKl, I was able to get a confirmation email when using a gmail address (tried it just now). The email will come from wiki@wikimedia.beta.wmflabs.org WBrown (WMF) (talk) 13:25, 21 November 2023 (UTC)

Thank you @STei (WMF), for the opportunity to participate in this testing. I like that submitting an incident report is just two easy steps. However, the report icon is not conspicuous enough in my opinion.

For the reporting process, I expected to see a message or a notification to assure me that my report was filed after submitting it. Uzoma 20:21, 12 December 2023 (UTC)

@Uzoma Ozurumba, the two points have been well noted. Thank you for the feedback. –– STei (WMF) (talk) 12:42, 18 December 2023 (UTC)

Tested Incident Reporting System

• I love this reporting process and I like the fact that its anonymous as well.
• I will actually love an extension like this which will make is easy to make such a report.
• So far I don't know if people with only IP addresses can file such reports?

Owula kpakpo (talk) 14:21, 6 December 2023 (UTC)

Owula kpakpo, to clarify your question, you want to know if an IP (an unregistered user) can also file reports? –– STei (WMF) (talk) 15:43, 6 December 2023 (UTC)

Yes so that maybe if they want to keep their anonymity they can just use their IPs in cases were its a matter or abuse and harassment. Owula kpakpo (talk) 10:27, 9 December 2023 (UTC)

@Owula kpakpo, we have not made a decision yet about what’s going to happen with anonymous users. It will require discussions for multiple reasons. However for this MTP, our focus is around registered users. –– STei (WMF) (talk) 14:46, 11 December 2023 (UTC)

This is great to know then. Thanks for your time. Owula kpakpo (talk) 19:10, 18 December 2023 (UTC)

Not working

Attempted many times to submit it using both the [...] on @STei (WMF) lines and the Tools option, nothing worked. I always get the "Your request could not be submitted due to an error. Please try re-submitting." - Darwin ^Ahoy! 20:14, 11 December 2023 (UTC)

Hmm.. Nevermind, now I've seen the 3 hour limitation. It certainly was that. - Darwin ^Ahoy! 20:16, 11 December 2023 (UTC)

@DarwIn, thanks for letting me know! Even without submitting, if you have a general impression about the form itself or about the idea of reporting from a talkpage, please let me know. That is also good feedback we would like to have. –– STei (WMF) (talk) 20:28, 11 December 2023 (UTC)

@STei (WMF) Hmm.. I've finally tested it, and it worked fine, as expected. As a sysop and a tester/user of the mentorship feature, I wonder if this will not cause a flood of bad reports that would render the tool almost unusable for most. I believe that at least there should be clear that such a report may be or may be not acted upon by a sysop, and that for urgent, confidential, and more serious stuff, T&S should be contacted directly. The option to report it to T&S should either be present, or their emails provided. - Darwin ^Ahoy! 18:08, 17 December 2023 (UTC)

@DarwIn, thank you for taking the time to complete the testing and sharing the feedback. It will be taken into consideration. Happy holidays. –– STei (WMF) (talk) 12:39, 18 December 2023 (UTC)

@STei (WMF) Thank you, happy holydays! - Darwin ^Ahoy! 13:37, 18 December 2023 (UTC)

No verification email received

The error message is

---

Verified email address required We need your verified email address to provide updates on the report or follow up if necessary. Please add or verify your email address from your user account settings before filing a report.

---

I simply did not receive a verification email. I tried resending the email, but the system only allows users to change their email but not to re-send it. I was scared to change my registered email. Bluerasberry (talk) 22:36, 20 December 2023 (UTC)

Bluerasberry, I am sorry about your challenges, I will forward this report to my colleagues who can help and then I will get back to you. Thank you so much for making the time. –– STei (WMF) (talk) 08:43, 21 December 2023 (UTC)

Feedback on Nov 2024 update

Hello, thanks for your effort! Here are some feedback and concerns:

1. It currently seems that all users can report threats to the emergency team. I am worried that this system will be abused by newcomers and vandals. Perhaps the access to this feature should be limited to only confirmed users or patrollers, rather than all users?
2. Currently, the emergency report is only sent to the emergency team. Perhaps the report could also be sent to the local functionaries/sysops? They are more familiar with their local wikis and also have the ability to take action on local wikis (See also Responding to threats of harm), such as blocking users or hiding revisions in question. If there are any private and legal concerns, then only functionaries/sysops who hold the NDA can be granted access to these reports.

SCP-2000 03:30, 12 November 2024 (UTC)

Hi @SCP-2000, thanks for your feedback!

1. Only users with a confirmed email address are currently able to file reports. In order to prevent vandals from abusing the system there's other restrictions in place such as the number of edits and the age of the account (I can share the entire restrictions list if you're interested). We will monitor the emergency queue to see what kind of reports are being filed, how many etc. That will help us understand if we need to add more restrictions or eliminate some.
2. Right now we're focusing on reports that are handled by the emergency team as per the emergency policy. As we will expand the system further we will discuss other types of reports and how various groups will interact with them.

MAna (WMF) (talk) 15:46, 13 November 2024 (UTC)

@MAna (WMF): Hello, I am curious about what is the entire restrictions list. Thanks:) SCP-2000 05:20, 17 November 2024 (UTC)