Wikimedia Blog/Drafts/Understanding workflows for editors

Title ideas

Ideally three to ten words, the headline to your piece will show up in social media shares and on the blog's homepage. Try to capture the most interesting part of your piece.

• Understanding editors' workflows
• Understanding workflows of editors
• A contribution taxonomy for Wikipedias

Summary

A brief summary of the post's content, about 20-50 words. On the blog, the summary will appear in italicized text underneath the headline. You can use this space as a teaser, expansion of your headline, or as a summary of the post

• We researched with Wikipedia editors the workflows through which they work together. Our colleagues will use this understanding to improve editing for all. You can too.

Body

What do you want to tell the world? Put it here. The best imagery helps convey your most basic ideas without doing it overtly. Ideas on introductions and writing style can be found in our guidelines.

Workflows involve a lot of steps. Photograph by Frederick H. Evans, public domain.

Wikipedia exists because of a vast army of people who work alone and in concert. The amazing work of these volunteer editors in making Wikipedia is a complex world of interactions, processes, and activities. They range across the whole work of donating content, including editing, fixing things, adding media, and many other ways. Collectively, these contributions are the lifeblood of our wikis.

These volunteer efforts happen through many different workflows. Our work at the Wikimedia Foundation is to help our volunteers do their very best to share knowledge for everyone in every language. In this this project, we wanted to better understand which areas of contribution Foundation technical teams could work on, to improve the experience for all people who want to take part.

Even within one workflow, our users have quite varied steps based on the different steps and tools available to them, depending on their wiki, device, and interfaces. Here are four screenshots of the same action, tagging a page as spam that should be deleted, in different ways:

• 
  On the Chinese Wikipedia, on desktop, using the visual editor. Image by James Forrester, public domain.
• 
  On the Russian Wikipedia, on desktop, using the 2010 wikitext editor. Image by James Forrester, public domain.
• 
  On the Korean Wikipedia, on mobile Web, using the mobile wikitext editor. Image by James Forrester, public domain.
• 
  On the English Wikipedia, on desktop, using a custom community gadget to help. Image by James Forrester, public domain.

Contribution to Wikipedias is complex, and is hard to see as a whole. There are many workflows and many ways to accomplish them. We needed to think about the whole, all of the parts, and how the parts work together. We called this work the Contribution Taxonomy project.

A part of the Wikimedia movement strategy for 2030 is to enhance knowledge equity across the movement. We know that it isn't as easy and welcoming to participate in our community processes when on a mobile device. Key language communities, who we wish to support better, are mainly spoken by people who can only access Wikimedia through mobiles. We wanted to help the platform teams decide which workflows and steps are most important to improve for people editing on mobile devices.

What we studied

Our initial work was to consider the different things people meant when they talked about the varying forms of contribution. We went back and forth a fair bit about how to understand these, before settling on a taxonomy in which the crucial groupings are moderation, where the focus is on changes, and curation (and creation), where the focus is on content. There is a continuum between these two poles, as tasks may be started from one prompt and then migrate to another. For example, if your plan is to write about an artist, you may see something wrong in their article, and then investigate when it was introduced and follow-up on changes by that individual to other pages.

The taxonomy. Image by Abbey Ripstra, CC-BY-SA 4.0.

We also identified some major groups of work that we chose to exclude from our project. First, we chose to exclude items related to multimedia. Uploading educational videos, reviewing licences, re-touching files, and the dozens of other related tasks are key workflows for our communities. However, the current workstream of adding structured data to how Wikimedia Commons works will likely prompt them to alter these. Next, we decided to not try to map out the detailed workflows of community governance, as these generally only involve elite, very experienced editors and are significantly based on social status rather than tools we could help teams build better. Finally, we confirmed we were to look at donations of content and work, rather than financial ones, which were out of scope.

With our taxonomy in hand, we investigated the five communities we chose to study for this project: the Korean, Czech, Hindi, French, and English Wikipedias. Working with dedicated community experts from each wiki, we pieced together a list of all the different decisions users would come across, what tasks they would involve, and finally what workflows each wiki had in place to get those tasks done. Many workflows are common between wikis, whereas others are bespoke to one community circumstance.

Workflows are broken down into a series of steps, points where the user can "hand off" the work to someone else. Some of these steps have alternatives; some are optional; some can be repeated; and some trigger whole new workflows. Different wikis might have the same step done in slightly different ways (such as a different template used to flag a page for action), or have alternative steps that you would do there (such as listing a page on a clean-up list instead).

The initial master inventory. Image by James Forrester, CC0.

Here is the breakdown of one of the common workflows, getting a page deleted quickly because it makes no claims about notability as to why it should have an article in an encyclopædia. You can see a number of steps, with alternative steps 1a, 1b, and 1c of which you would do one; step 1b has four different ways to be done, depending on what tool you have available on your wiki; steps 2 and 3 are optional; and the process ends after one of steps 4a or 4b are chosen. We also added step codes for each common type of step to help our teams recognise common interaction patterns.

In total, we identified 88 distinct workflows across the five wikis we studied, with a total of 550 steps.

Overall view of the Contribution Taxonomy. Image by James Forrester, CC0.

The tasks are of the form "Improve current or add new illustrations to a page" or "Direct users to be better contributors". Each of these has multiple potential workflows you could follow to get them done in one instant, such as adding a caption to an existing media item or adding a new media item, or telling a user what to do or what policies they should follow. The cultures of different wikis are expressed through what workflows they have, and so the implicit focusses they uphold; if we studied other wikis, as well as much in common we would no doubt find additional workflows or alternative steps.

Once we had identified the workflows and steps, we needed to have a system by which to evaluate these workflows to help teams focus on where they could add the most value. Some of this was objective – "On what wikis does this workflow take place?", "How often does this happen?" – and we added six details for these directly.

Another set of criteria we created in collaboration with our technical teams was around user difficulty – "How visible is the process?", "How comfortable do I need to be with technical things?" – as that's a key part of the approachability of different workflows. Finally, as we mentioned above, we wanted to investigate how possible it would be to undertake these workflows on mobile devices – "Do I need to have many things open at once?", "Must I do lots of steps in succession?" – to speak to the potential for technical teams to support mobile-only users. We added ten criteria for these concerns, rated on a scale from one to five, and agreed them with our various stakeholders.

The addition of scores to the master inventory. Image by James Forrester, CC0.

With these criteria, we rated all workflows and all steps, with a set of around 5500 ratings, and cross-checked them between different wikis' processes. This helped us review whether two wikis really had the same workflow or a different one, ensure that all our community ambassadors had the shared understanding of how they worked, and reconsider the language and specificity of the steps and tweak where we documented the boundaries and sequencing of steps.

The addition of score weighting to the master inventory. Image by James Forrester, CC0.

Working with our technical teams, it became swiftly apparent that a one-size-fits-all approach to the criteria would be insufficient for their needs. The teams serve different groups of people, and intend for different use cases. For example, the Android app team want to support people in very quick editing interactions, whereas the iOS app team didn't consider this as important. To help teams prioritise the workflows for their needs, we built them a weighting matrix. Each teams' leads can set a relative weight for each criteria, and adjust them as they reconsider their areas of focus.

Show, don't tell

In addition to the taxonomy, master inventory, assessment criteria, and weighting system, we built a visualization system. Visualizing workflows makes it easier for multi-disciplinary teams to understand the details and variations of a workflow in the same way. This common understanding makes it easier for people to share their expertise and collaborate toward improving the experience with a team.

Carolyn LaMadeo, a designer at Wikimedia Foundation, designed the visualization system. It is a flexible, modular system, consisting of elements that are mixed and matched to represent any of the workflows:

The basic parts of workflows. Images by Carolyn Li-Madeo, CC0.

Visual workflows in practice

Visualisation of the tagging workflows. Image by Abbey Ripstra, CC-BY-SA 4.0.

Pulling these pieces together, this is an example of a visualized workflow. It represents the group of workflows to tag a page or section for improvement.  There are a handful of reasons a page or section could be tagged, which each trigger a slightly different workflow. Each are represented here, along with which wikis have which kind of tagging available, and the steps and tools needed to accomplish it.

Visualisation of the copyright violation deletion workflows. Image by Carolyn Li-Madeo, CC0.

A more complex workflow is this one, for deleting pages for copyright violations. Not all of these workflows or steps exist on every wiki, and some parts are limited to users with particular rights, so the complexity should not scare you away!

Initiation step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

You initiate one of the workflows by discovering an article where you think that it should be deleted because of a copyright violation. Getting to this point is out of scope of this particular workflow – it's often triggered by another workflow, such as patrolling changes or just from regular reading.

Selection step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

The first decision is which is the right workflow for your situation. Here, we choose the middle option, where you're on the Czech, English, or French Wikipedias and are moderately confident that the article has content that has been taken from somewhere else.

Tagging step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

Step one involves only one possible action – you need to tag the page with the "Likely copyright violation" template, by editing to add it to the top of the article's talk page and linking to which edit you think was wrong. (For simplicity, we don't list out all eight possible editors you might have to do this if using the desktop Web interface.) This can be the end of your part of the workflow.

Agreement step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

The second step involves another decision. An editor who comes to the page having been altered by the tag added in step one has to choose if they agree that it is likely a copyright violation, or is not. In this case, they decide you are right, find the original source of the violation, and so add it to the central listing.

Notify step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

The third step – where someone can warn the author that their edit is thought to be in error – is optional on some wikis.

Discuss step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

The fourth step is a very common workflow item: different community members are welcome to weigh in with their thoughts, to add observations or concerns, and so help make the right decision. Again, this is optional – the workflow can continue without comment.

Decide step of the copyright violation speedy deletion workflow. Image by Caroyn Li-Madeo, CC0.

The fifth step is the final one, where a user has to decide whether the listing is correct, in which case the article is deleted, or incorrect, in which case it is unlisted and untagged. The first of these choices would need the user to be sysop, as the deletion tool is only available to them; on most wikis, there is a cultural expectation that the second choice is also only made by a sysop, who has gained some trust or "social capital" that they spend on making this decision.

That's a brief walkthrough of one of the workflows; there is lots more to study on the project page. The top ten workflows that we identified across the three target teams are as follows:

• Work to expand an article directly (includes a series of sub-workflows)
• Tag a page/section as needing something done
• Add an image/video/audio/3D item to a page
• Add a gallery of media items to a page
• Add a caption to an existing media item on a page
• Move a page directly
• Copy-edit / re-write
• Manual welcome
• Review raw feed of edits, looking for vandalism (very frequent; hard to implement on mobile)
• Patrol new articles

From us, to you

We built the contribution taxonomy as a tool for everyone to use and contribute to. You can find the master inventory (including the weighting system), criteria, visualization system, and a  hand full of visualized workflows on the project’s page on MediaWiki.org.

There are several things you can do with the contribution taxonomy.

You can learn about a specific workflow that is of interest to you, either by finding it in the master inventory and learning about it there, or if it is one of the visualized workflows, learn about that workflow by checking out it’s visualization.

The master inventory contains workflows from Korean, Czech, Hindi, French, and English Wikipedias. You can check out the workflows and steps of a specific Wikipedia, and see how they work together, and how they are represented in the contribution taxonomy.

You can look at workflows across the different wikis within the taxonomy, and learn how they are similar and how they differ. For example, which wikis use HotCat, Twinkle, or other gadgets, and which don’t.

Along with using the contribution taxonomy how it is, anyone can contribute to it. For example,

you can make a copy of the master inventory, and add the workflows from a wiki that isn’t already represented in the current master inventory, and then upload it back to the project’s mediawiki page so others can learn more about that wiki too.  

If you would like to use the visualization system to build out a workflow, go for it! The elements of the visualization system are available here as SVG files. SVG files are editable using vector graphics editing software. The English Wikipedia has a list of such tools, including some open source ones that you can use to put the elements together to represent a workflow. Use the master inventory as your guide for how to put the elements together, and if it is helpful, use one of the already visualized workflows to as an example to start out.  Once you have completed a workflow visualization, upload it to the project’s media wiki page, so others can benefit from it too.

It would be great to see this system grow to represent more Wikis’ workflows and steps.

James and I are keeping an eye on the project’s talk page, so please reach out to us there if you have questions, or suggestions, or want to let us know how you have used or contributed to the contribution taxonomy.

[Acknowledgements – Carolyn, the community ambassadors, colleagues in product and design, …]

...

Abbey Ripstra and James Forrester, Wikimedia Foundation