Collaboration/Enquête de satisfaction sur Flow/Compte-rendu

En septembre 2016, la Fondation Wikimedia a organisé une enquête sur Flow, un système de discussion et de collaboration moderne. Flow permet des discussions structurées et prend en charge à la fois le mode de rédaction en wikicode et celui visuel. Un petit nombre de wikis utilisent Flow par défaut pour toutes les pages de discussion en lieu et place des pages de discussion non structurées basées sur le wikicode. Quelques autres wikis permettent à des utilisateurs de convertir individuellement leur propre page de discussion à Flow (Fonctionnalité bêta), ou utilisent Flow sur des pages communautaires (forums d’aide, bistros…).

Résumé

L’enquête, menée par l’équipe Collaboration technique, visait initiallement les utilisateurs et communautés qui avaient eu une expérience directe des deux systèmes de discussion. La participation était anonyme et ouverte à quiconque ; toutes les questions étaient facultatives. 583 réponses provenant de 20 projets Wikimedia identifiés ont été enregistrées.

Les principales raisons des contributeurs de préférer les discussions Flow sont :

• un système de discussion structuré par logiciel, lequel permet de suivre certains sujets au lieu de pages entières,
• des notifications spécifiques lorsqu’une réponse est reçue qu’un sujet est réellement apprécié,
• un design clair pour créer de nouveaux sujets et modifier ses propres messages, particulièrement apprécié par les personnes qui travaillent avec les nouveaux,
• une meilleure intégration et usage depuis des appareils mobiles,
• notifier des personnes, remercier et répondre aux autres utilisateurs est plus facile que dans les pages de discussion non structurées en wikicode.

Les principales raisons des contributeurs de préférer les discussions non structurées en wikicode :

• trouver une discussion spécifique d’une page de discussion non structurée en wikicode peut être réalisé avec la fonction de recherche de MediaWiki, alors qu’avec Flow, même les fonctions de recherche élémentaires manquent,

• visualiser les changements et patrouiller contre le vandalisme dans les pages de discussion non structurées et en wikicode peut être réalisé à l’aide des fonctions de MediaWiki qui permettent de suivre le contenu d’un article, alors que Flow a plusieurs lacunes qui rendent le suivi et l’annulation de changements plus compliqués,

• certains utilisateurs préfèrent la vue compacte des pages de discussions non structurées et en wikicode, ils trouvent que les discussions Flow laissent beaucoup d’espace blanc et obligent trop à faire défiler la page,

• certains utilisateurs préfèrent les flexibilité fournie par les pages de discussion non structurées et en wikicode, qui débutent comme des pages blanches, par contraste avec le comportement guidé des discussions structurées de Flow.

Selon les données de l’enquête, les contributeurs quotidiens et hebdomadaires tendent à préférer les pages de discussion non structurées et en wikicode, alors que Flow est préféré par les contributeurs qui utilisent moins fréquemment les pages de discussion. Ces données reflètent deux aspects de Flow dans son stade actuel :

• Flow est déjà un outil utile pour participer et répondre aux attentes des contributeurs nouveaux et occasionnels qui sont familiers des autres systèmes de discussion en ligne, mais qui ne sont pas familier avec les pages de discussion en wikicode et non structurées spécifiques à MediaWiki, ni avec les conventions sociales que les communautés Wikimedia ont construit autour des discussions sur les pages de discussions (utilisation de deux-points, de listes à puce, modèles de discussion, signature à ajouter…),

• cependant, les fonctionnalités manquantes de Flow (pour suivre une page, effectuer une recherche, modérer une discussion, séparer un sujet…) et le manque de maturité de l’expérience utilisateur devient plus évidente pour les utilisateurs réguliers.

L’enquête montre que les discussions non structurées et en wikicode ont une solide prise en charge à côté de Flow à ce stade. 52 % des participants ont exprimé leur préférence pour les pages de discussion en wikicode et non structurées, contre 38 % pour Flow et 10 % étaient indifférents. Dans les réponses libres, la plupart des contributeurs ayant une légère préférence pour les discussions non structurées et en wikicode ou n’ayant pas de préférence pointent du doigt les faiblesses de Flow qui, si résolues, les feraient utiliser davantage le nouveau système de discussion.

Les réponses libres montrent également que le développement de Flow continue d’être un sujet de discussion brulant. Un groupe de contributeurs expérimentés qui se fait entendre s’oppose fortement au développement de Flow par principe. D’autres contributeurs expérimentés demandent aussi la poursuite du développement de Flow pour corriger les fonctionnalités manquantes et le déployer sur leurs wikis.

Améliorations

Une meilleure assistance pour vérifier l’historique d’une conversation et avoir une façon de rechercher sur un panneau Flow sont les fonctionnalités les plus demandées. Ces améliorations ne devraient pas avoir un impact important sur l’aspect visuel de Flow, elles devraient donc être équilibrées par des changements plus visibles, comme fournir un paramètre pour rendre les conversations plus compactes, plier les indentations ou avoir des URLs intelligibles.

Le design pourrait être revu pour donner à tous les utilisateurs une possibilité d’avoir quelque chose de proche de la disposition actuelle des pages de discussion non structurées et en wikicode. Les notifications de Flow sont appréciées mais ont quelques critiques. Les entrées dans la liste de suivi sont également critiquées. Ces deux éléments pourraient être revus pour éviter les « mauvais bruits ».

Satisfaction with particular functions

The survey asked users to evaluate how well both talk page systems perform particular functions.

Comparison of Flow and unstructured wikitext discussion pages functions

Fig. 1 - Favorite tool for similar actions, in percent.

Users were asked to compare how well Flow and unstructured wikitext discussions pages perform across five typical talk page functions:

• reading talk page discussions,

• understanding talk page discussions,

• writing and editing in talk page discussions,

• engaging with other users on talk pages,

• summarizing discussions on talk pages.

In general, users preferences in this breakdown line up with their overall tool preferences: a majority of users prefer unstructured wikitext discussions pages and prefer it more strongly, but Flow makes a strong showing as well. That comparison is not always relevant: unstructured wikitext discussions pages are blank pages allowing combinations of features such as discussing, polling, drafting and more. Flow has been design to be a discussion system, which could be used to build other tools (at the moment, Flow is mostly used on user talk pages).

Flow feature satisfaction

Fig. 2 - Appreciation of Flow features, in percent.

In this section, people were asked to say how useful they find Flow for particular talk page activities, absent any comparison to unstructured wikitext discussions pages.

• adding a new topic or comment in talk page discussions,

• editing a message in talk page discussions,

• following the layout of Flow discussions,

• resolving conversations,

• summarizing discussions

• watching and receiving notifications about individual topics (rather than entire talk pages).

Evaluation of Flow tools

50% or more of respondents find Flow "completely" or "somewhat" useful for all of the tasks.

The three tasks where Flow gets the most support are:

• watching a section instead of a whole page (68% satisfaction).

• adding a new topic to a Flow page (61% satisfaction). Flow provides dedicated tools for adding new topics to a discussion which, feedback indicates, even experienced users recognize as making the task easier, especially for newcomers.

• editing a message in talk page discussions (57% satisfaction). Because users don’t need to open a whole page to fix something in one comment.

While strong integration of Flow and Notifications is clearly an area where Flow shines, it's worth noting feedback in the free-form fields concerning the high number of notifications received by people who follow Flow boards. That Flow expands the volume of Notifications on wikis where it's implemented is a known fact. Any project seeking to expand Flow usage will do well to consider this and to provide new means for users to manage what Notifications they receive.

Of respondents who said that Flow is “Not at all useful” for the tasks in question, a majority (60%) are users who overall “strongly prefer" unstructured wikitext discussion pages. More people who "strongly prefer wikitext" appreciate Flow's features than vice versa, which makes sense if Flow users are in general less experienced.

In free-text answers, many respondents complain about the design of Flow pages (“... leaves much more space that lengthens and drowns the discussion”; "looks like a blog"). Other commenters point to Flow's superiority for mobile use ("Mobile support!!! It's already much better in Flow than in the old talk pages.") and archiving ("Archiving systems are not required anymore").

Flow functions for engaging with other users

Fig. 3 - In regards to engaging other users, to what extent are you satisfied with the following features of Flow.

Helping users to engage with one another is another area where Flow finds strong support. Thanking, mentioning and replying to users all get +50% satisfaction. Respondents show the strongest satisfaction with Flow's Thank feature (32% "very satisfied").

Evaluation of Flow discussions functions

Fig. 4 - Possibly useful aspects of Flow

To gauge how well Flow achieves a number of its design goals, users were asked to agree or disagree with a series of eight propositions:

• writing and editing with Flow is easy and intuitive,

• the layout of Flow discussions is easy to read and understand,

• watching topics on Flow discussions is easy,

• following individual topics on Flow through notifications is more useful than following an entire page,

• Flow makes discussions easier to summarize,

• Flow makes it easier to mention other participants in the discussion,

• Flow makes it easier to mark the conversation as resolved,

• with Flow, the interactions with other users are easier than with wikitext (easier to mention, thank, respond, watch).

It's worth noting that users who "strongly agree" and "strongly disagree" are more or less evenly divided. But total approval for most of these eight key functions is near or over 50%. A couple of results stand out:

• there is an overall appreciation of Notifications related to Flow, as seen in the popularity of both "Watching topics on flow" and Following individual topics on Flow through notifications." These same 2 value propositions point to another thing users like about Flow, the "structured" part of "structured discussions" —they like following individual topics. This is one of Flow's core ideas,

• mentioning other users is considered the least easy feature and should be investigated.

What Flow improvements do users want most?

Respondents were asked to rank the importance of improvements to Flow that users have requested in the past. Here is the list, as ordered by survey respondents:

1. better support for checking the history of a conversation, display diffs, and clearing vandalism,
2. search and find specific topics,
3. being able to move topics to a different page would help,
4. filter or categorize topics to make it easy to focus,
5. view all my Flow activity at once on a single page.

Better support for checking the history of a conversation, its changes, and clearing vandalism would be helpful for most users. On free-form fields, users have shared examples, like the lack of similarities with the current history system (lack of comparisons for the whole activity on a page between two moments using diffs, no way to undo things easily), the difficulties of doing some particular actions (delete contents from your own page, difficulties for sysops or oversighters to perform maintenance actions), etc.

Search was a virtual tie with better history support. In free-text comments, users clarified that they want to search Flow comments overall and they want to be able to search a single Flow page for a specific comment.

Managing and organizing conversations by moving topics among pages was a close third. In the comments, users asked for ways to split, merge, move or relocate topics or parts of topics.

Conclusion concerning particular functions

When users are comparing unstructured wikitext discussion pages and Flow for the same activities, there is a preference for unstructured wikitext talk pages. But when users are judging specific features, Flow has more support: strong support from people who like the tool and meaningful support from people who prefer unstructured wikitext discussion pages. Users who don’t have a preferred system tend to appreciate Flow’s ease of use.

That feature support notwithstanding, Flow has a lot of room to improve. Creating a search function, providing better history support and enabling users to better manage and organize topics will be first steps towards winning over Flow skeptics and strengthening the approval of existing fans.

Satisfaction générale

The two discussion systems are quite different; one purpose of the survey was to know which system people prefer. With some limitations, the data can provide useful information about attitudes.

Overall satisfaction based on frequency of contributions

Fig. 5 - Favorite tool by editing experience, grouped, in percents. "Wikitext" refers to unstructured wikitext discussions pages.

Looking at tool preference through the lens of frequency of contribution is revealing: it's clear that Flow appeals strongly to users who edit less frequently. Among users who edit once per month, 58% prefer Flow, compared to 24% for unstructured wikitext discussion pages. By contrast, among those who edit every day, almost 40% prefer unstructured wikitext discussion pages compared to 26% for Flow.

The same pattern can be seen when the axis of the analysis is reversed: among editors who prefer Flow, 40% are infrequent users (editing once a month or less). By contrast, among users who favor unstructured wikitext discussion pages, only 17% are infrequent users.

That said, it’s worth noting that Flow has many adherents among frequent users: of those who edit once a week, 32% favor Flow.

The fact that new users favor flow would seem to be consistent with other research showing that beginners are able to do more tasks using Flow than unstructured wikitext discussion pages.

Overall satisfaction based on project where Flow is mainly used

Fig. 6 - Preferred tool by wiki with more than 40 answers recorded, in percent. Wikis with less than 40 answers have been grouped in the "all other wikis" bar.

Unstructured wikitext discussion pages is favored by a similar margin on mediawiki.org, the biggest wiki where Flow is the default discussion system. Wikis that appreciate Flow and had more than 40 responses are Wikidata (over 70% support), French Wikipedia and Chinese Wikipedia (~40% support). As a comparison, users of English Wikipedia show the strongest support for unstructured wikitext talk pages, at about 70%, based on 24 answers.

Overall satisfaction based on first year of contribution

The largest cohort of respondents started to contribute in 2005 and 2006. 209 respondents started to contribute before 2009, 198 after 2009 (included). However data are not conclusive, because of the bias related to first year of contribution, which is not meaningful compared to experience. Users who contribute frequently gain experience and comfort with unstructured wikitext discussion pages quickly enough so that a survey question based on starting year doesn't show a pattern based on the first year of contribution.

Conclusion concerning satisfaction

Users who contribute frequently tend to prefer the unstructured wikitext discussion page editor. Users who contribute less regularly tend to prefer Flow.

However, these preferences are not absolute. Among people who contribute at least once a day, 34% say they value both systems, and 27% prefer Flow. The difference between daily users who prefer unstructured wikitext discussion pages and daily users who prefer Flow is only 12%. That difference decreases to 6% among users who contribute weekly (31% for Flow, 37% for unstructured wikitext discussions).

In free form comments, the majority of users without a strong opinion for unstructured wikitext talk pages (from strongly or somewhat supporting Flow to indifferent or somewhat preferring unstructured wikitext discussion pages) tend to point to Flow weaknesses and suggestions for improvement. However, a majority of editors expressing a strong preference for unstructured wikitext talk pages just ask to remove Flow.

Recommandations

Based on the result of this survey, the Technical Collaboration team recommends:

• to continue the development of Flow, focusing on major missing features and design issues, as a natural complement to the editing experience provided by the visual editor and as part of our effort to onboard and retain new editors successfully,

• to focus the development on the needs reported by the communities that are already using Flow or that are willing to adopt Flow,

• to focus first on the use cases where Flow is a viable alternative already today, notably user talk pages, which are the main channel through which new and casual users engage with other editors, and where other editors reach out to them,

• to offer Flow to communities that are looking forward to adopting it.

À propos de l'étude

Distribution

The survey was open to anyone. The survey was distributed to all public wikis using Flow in order to reach users who use Flow and can compare it knowledgeably to unstructured wikitext talk pages. A message was sent to users that use or used Flow (including people who disabled it) on wikis where Flow is enabled as a Beta feature, manually activated on user talk pages or used as the default talk page system. A message also was sent to main community hubs on wikis where Flow is used on a few pages as a trial. The invitation was also sent to some public Wikimedia mailing-lists or other public hubs, and shared by users to others. Any user who has seen the survey’s link could participate, even if they never used Flow. One answer was allowed by IP address.

Distribution has been done by messages batches, with an invitation posted in the wiki language, when possible. A few days after the beginning of the distribution, while no message has been sent to Mediawiki.org users, 17.47% of recorded answers were from that wiki. Due to the fact that Flow is the default discussion system on that wiki, it was not possible to identify which talk pages were used by really active users. Hence, a decision had been made not to deliver messages on Mediawiki.org user talk pages, just on community pages.

Limitations

These are the main limitations registered in this survey:

• this survey has been promoted to existing users. However, Flow is being developed equally for new users and those who are not here yet,

• this survey doesn’t claim to take into account a representative sample of Flow users or Wikimedia users, since participation was open to anyone:

• • regular users and active users at that time are more likely to be over-represented versus casual users.

• • a self-selection based on high interest for/against Flow is also expected.

• • the survey was promoted by some users seeking like-minded participation.

• • people that stopped using or never used Wikimedia wikis due to the barrier to entry of unstructured talk pages are not surveyed.

Voir aussi

• Raw data from the 2016 Flow satisfaction survey