VisualEditor/Why/User Test Data
See Research:VisualEditor's effect on newly registered editors/May 2015 study for more recent research
As part of our preparations for the launch we ran a group of users from UserTesting.com through a set of tasks, first using the VisualEditor and then using Wikimarkup. Some of them had tried to edit in the past; many were total newcomers. The script for the tests can be seen below. This is not the only data we are relying on; we are also (and more usefully) gathering quantitative data from a test on the English-language Wikipedia, in which 50 percent of new users were given the VisualEditor by default.
Using the source editor edit
"There was tons of foreign jargon amidst the content and it looked nothing like how the real page looked."
"I don't want to have to learn a new language just to edit Wikipedia."
When presented with the source editor, users tended to have the same set of problems. Many of these centred around identifying what they were expected to change; with so much markup, they found it difficult to identify things in the markup view that matched what they'd seen when reading the rendered page. Users were also worried by the clutter of the editing interface, particularly the mass of buttons at the bottom of the "save page" window.
Users struggled to understand the wikimarkup found in a moderately-sized article; when they managed to identify bits, it was almost entirely from comparing their memory of the rendered page to individual words, and looking at the formatting around those words (for example, noting that all of the headers had equals signs, and thus determining that equals signs made headers work). With one exception, every user found the source editor intimidating and would opt not to use it.
Using the VisualEditor edit
"[The VisualEditor] feels more like editing a word document and isn't as intimidating as the [source editor] which feels more like editing code"
Several problems were raised with the VisualEditor. Many users found adding links to be confusing, something we have noted and are evaluating, and (as known) the VisualEditor was slow to load for some testers. Several other (now fixed) bugs, such as problems with saving the page, also frustrated users. However, all-but-one of the testers concluded that they preferred using the VisualEditor to using the source editor, one of them noting that "[with the VisualEditor] I would be more likely to make edits. That interface was a lot easier to understand and I had more confidence that the changes I was making were the changes that I wanted to make. I also like that I had an opportunity to review the changes and note them."