Research:Cross-wiki notifications user research/Protocol

Round 1

Before the test. About the usual volume of cross-wiki notifications and current user processes.

Do you participate in multiple Wikimedia projects or different languages? Which ones and how often do you participate in them?
Do you usually check what's new across different wikis? Can you show us how you would do that in your usual wikis?
- When checking the notifications from different wikis do you try to find the new ones from any wiki or do you focus on one wiki after the other?

Prototype overview

This prototype simulates you are an editor from English Wikipedia named Cronopio.

Notification processing

One notification is about a new message in your talk page, what is the message the notifications mention ?
Can you find a thanks notification?
- Where would you expect to go if you click that notification?
- Who thanked you? How would you find out more details about that person?
- Imagine you don't need more info about it, can you mark it as read without actually opening the notification?
- Imagine you get too many thanks notifications, is there a way where you could avoid those from showing the next time?

Cross-wiki navigation

Currently, you are on English Wikipedia. Can you identify notifications from other projects like Wikimedia Commons?
- How recent are those notifications?
- Can you check them and describe what are they about?
- In this context, would you have preferred to find out about them earlier, even though you were at English Wikipedia?
Which are the remaining notifications?
- Where are they from?
How would you continue processing the notifications from English Wikipedia?

Discard all

There is a notification from Spanish Wikipedia, how would you discard it ?
How would you discard all the notifications?
- Do you expect that to affect notifications from all sites?

After the test.

Which is your general impression of the notification system you have been using?
- Anything that worked particularly well or not, based on the way you consume notifications?

Do you think the proposed system will help you to track what happens on all wikis easily?

Before the test. About the use of the notification panel.

How often do you get notifications? What are they about?
Do you check these notifications as you get them, or do they accumulate unread? Why?
Do you often get irrelevant notifications? Have you considered disabling those?
When you get a notification, which information are you interested about, and what do you normally do with it?
Can you show us some of your notifications so that we can check the recent ones? Can you tell us which is the interesting information on each one and which was the next steps you did about each?
Is there any notification that combines several messages? Did you miss any detail of the individual ones when presented in this way?
Do some of the notifications require follow-up actions you need to take later? If you already read several, how do you keep track of those with pending actions from your side?

During the test

Prototype overview:

This prototype simulates you are an editor from English Wikipedia named Cronopio. Many aspects are not supported in the prototype yet, if you get lost you can always refresh the page and start again.
Which notifications can you see there? What are they about?
- If you were this editor, which notification would you process first?
We'll go through the different notifications in the list…

Secondary actions:

What is the first notification (the one on top) about?
Who created the topic?
This is a topic you have not participated but looks interesting to you. Is there a way to get notified of the upcoming discussion updates? (the user is expected to use the "watch new activity" action)

What is the second notification about?
Who reviewed the page?
How would you show appreciation to that user for their work? (the user is expected to use the "thank Ludmilla" action)
What would you do to give the reviewer a more detailed thank you note (the user is expected to explore more options in the "..." menu or use the user link to access their talk page).

Volume control:

The next notifications are about Easter Island, a topic you were interested in the past before you focused your editing activity in space exploration. You got yet another topic notification about Moais, is there a way to avoid getting distracted every time a new topic is started about this? (the user is expected to access the "mute new topics on this page" inside the "...", volume options or general settings represent also valid paths).

Bundling:

What is the other notification about Moais about? Can you check what are the responses about at a glance? (the user is expected to either expand or just open the notification)
- (if the user didn't expand) Is there a way to quickly check what are the individual responses about without leaving the notification panel?

Mark as unread:

The last two notifications are about an interaction with Cesio. Can you tell what happened?
Imagine you have read already all the notifications. Can you mark all notifications as read? (the user is expected to use the "mark all as read" in the notification panel).
In case you want to further discuss the reverted edit later, is there a way to keep the notification as an action item to be used as a reminder? (the user is expected to use the "mark as unread" from the "..." menu)

Notifications page:

We've seen that Cesio complained about the lack of references in your edits, but you don't recall experiencing many reverted edits by Cesio. Is there a way you can find all past notifications where Cesio reverted one of your edits? (the user is expected to access the notifications page and set the appropriate filters about the user and the notification type).

After the test.

Which is your general impression of the notification system you have been using?
- Anything that worked particularly well or not, based on the way you consume notifications?
Do you think you could save time when processing your notifications with this system? Why?
- Would you be able to reduce irrelevant notifications?
Do you think you could track better the work you need to do based on the notifications? Why?
- Would you be able to access relevant info and actions for notifications?