Wikimedia monthly activities meetings/Quarterly reviews/VisualEditor-Parsoid, 2013-03-27

Slides from the meeting

QUARTERLY REVIEW MEETING: Visual Editor / Parsoid, March 27, 2013

The following are notes from the Quarterly Review meeting with the VisualEditor and Parsoid team on March 27, 2013 (1pm-4pm)

Agenda

1. Overview of Objectives
2. Progress to Date
3. Expected Deliverables in 2012/13
4. 2013/14 Planning
5. Questions/Thoughts

Present

Roan Kattouw, Mark Holmquist, James Forrester, Trevor Parscal, Erik Moeller, Sue Gardner, Tilman Bayer (taking minutes), Howie Fung, Terry Chay, Gabriel Wicke, Rob Moen

Remotely

Subramanya Sastry, C. Scott Ananian, Ed Sanders

Please keep in mind that these notes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Introductions

James:
presents agenda

Objectives

"VisualEditor enabled by default for (almost) every Wikipedia / MediaWiki instance" (1 July 2013)

• for all ~300 languages
  • deep level support: Unicode, IME (input method editor)
• every MediaWiki instance: graceful way for extensions/special workflows
• default means: replace edit button, for everyone

Needs to:

• "the target audience is males and females aged zero and up"
• work on all content pages (not just special namespace)
• have consistent, comprehensible and coherent UX. indicate parts that are not yet editable
• be able to be used by all users
• not screw up stuff
• work reliably on major browsers/platforms

Progress to Date

two major deliverables:

• 2011/12 Q4: prototype content editor in test deployment in VE namespace on mediawiki.org. MET
• 2013/13 Q2: alpha of basic content editor in production. MET

What currently works:

• VE can do edits of most basic content (text, link, headings, bold/italic, lists)
• Parsoid avoids dirty diffs
• > 1500 users have enabled it, >1700 edits have been saved (and lots more exploring using it)
• positive feedback, bug reports

Expected Deliverables in 2012/13

What we expect will work
Q4 beta as default for all - '"on track"'

• references (regular: simple)
  • clarify multi-use cites to the same block
• templates (at least, core templates used in references)
• images (set/adjust options and captions, delete, insert)
• categories
  • (add/remove, sort keys)
  • needed to keep people from deleting articles
• deploy to all users on main + user namespaces on all Wikipedias
  • needs testing before actual release

Things that won't work:

• tables
• very complex templates/compound templates
• galleries
• ParserFunctions
• media galleries
• non-content pages (e.g. talk pages)
• actual HTML in document (e.g. <div>s)
• some extensions not supported, e.g. math
  • MapSources (Wikivoyage)
  • ProofreadPage (Wikisource)
• Gadgets will break

First look at 4 new features (Trevor):

Template dialog

• A way to edit individual parameters of a template, suggests values
• will use TemplateData extension to hint what a template's parameters can be
• fall back to providing the parameters as used on the page and the ability to add free-form ones for un-hinted templates
• very complex/multi-part templates will not work, at least at first

• Hope it will motivate community to document parameters of template
• Gabriel: often not representable in DOM (ex.: template params are just strings, can be start tag only and unrestricted)
• James: Can't cover some edge cases - user can put arbitrary values in template
• James: Hope it will make community write saner templates
• Erik: e.g. help for formatting journal refs?
• Trevor: different situations, e.g. no information about values vs. some parameter names present
  • e.g. cite template is an advanced template, will need UX (on a one-off basis) to allow template editing in a non-basic manner.
  • have a path from worst case (wikitext) to more easily usable ones
• James: will go in manually and spec important templates like Cite (overall there are millions, can't scale to that), Trevor will then provide dialog UI for those.
• Gabriel: hard to support e.g. style attributes input as template params; easier for new content
• Trevor: can't enforce specific formatting (e.g. dates: ISO format vs. "in the 1950s")
• Trevor: Templates were invented on-wiki, there should also be the possibilty to improve them on-wiki (configurations, gadget integration, etc.)

Reference Dialog

• Similar to templates (it's actually just a template), but with extra guidance
• Lets editors add new references, duplicate references, and edit existing ones' contents and citation name
• depends on the template dialog & hinting to usefully edit the majority of references
• not yet decided UX-wise how <ref> and <references/> blocks will be edited.
• grouped references may not work at first

Sue: e.g. discovering reftoolbar [by Kaldari] was really important for my productivity as editor (input ISBN...), also ProveIt (by MattF?)

• Trevor: have those features but make them more discoverable as well
  

James: similar for DOIs (bot fills them in)

Category Dialog

(actual screenshot of it in development on Rob's box)

• page-level metadata setting
• also language links (James: not for July because of Wikidata issues and complexities with making users understand Wikidata license vs. enwiki license)
• more generally (post-July), dialog for e.g. NOTOC, #REDIRECT
  

Trevor: This is important, e.g. missing categories are a major reason why pages get deleted

Media Dialog

• lets user add new media items (photos, videos)
• able to set and edit captions as wikitext
  • Gabriel: unlikely to be wikitext, will be DOM
• add new items through wiki's libraries. is extensible, e.g.:
  • YouTube (for Wikia)
  • Flickr|CC for Commons
• uploading new items (through drag and drop?) post July
  • James: Mobile has shown Legal is okay with this
  • James: naming categorization issues to avoid flooding Commons

Trevor: this is to be led by Wikia developers

Team Resourcing

James on team resourcing:
Since start of 2012/13, we have added Ed, but availability of Roan is reduced

• Wikia
  • IK = Inez (in slide 18)
  • CW = Christian
  • also designer (Earl) just started

Overall, resourcing on VE is about 2x that of Parsoid

Projected Deliverables

for July 1

• editor that can edit the majority of content that new users will be expected to create
• parser that copes with actual edit rate: up to ~50Hz
• replace "edit" button. wikitext still available through "Edit Source" (hidden under the menu tab at top level along with move, protect, etc). Deployed on all WPs, possibly also Meta, Commons
  • This was the original design, but had the current system due to alpha release
• deploy to language Wikipedias + Meta + Commons (maybe: to get galleries)
  

Trevor: longer term plan for "outline mode" to rearrange larger blocks. Rearranging floating things is a problem in all visual editors. Some technical problems to solve first. Alongside that would be source-editing of the overall page inside VE

• templates: way for editors to mark up templates for VE

Discussion of VE major tasks / stretch tasks

• Definition Lists called out
• µ-VE surface for subset of wikitext used in log entries, for image captions and possibly other areas.

Discussion of Parsoid major tasks

Gabriel:

• start converting to HTML as soon as we can, and store that (instead of doing conversion later)
• might have to have wiggle rooms to use short-term optimization (high edit rates in estimate today)
  • wiggle room comes from bot edits - less timely for them, can postpone until next edit, VE only needs to keep up with human edits (human editors want to see saved result immediately)
• make it sane to edit parts of of a page (DOM subtrees), etc. but needs dependency tracking, invalidation logic etc to leverage it for performance
• Trevor: DOM based saving can make it instantanous for editing in VE, but puts strain on conversion
• Gabriel: incremental reparsing (originally stretch, but needed for new performance targets)
• Roan: might need more solid storage mechanism instead of Varnish
• in April, want to determine with Ops how many machines needed
• Gabriel: The rest of the list is just features for VE, but not a big issue
• Gabriel: pushed back some research tasks, see https://www.mediawiki.org/wiki/Parsoid/Roadmap#Q3_2013

Product risks and mitigations

Oliver has a product risk analysis:

• VE's impact on triage workflows (NPP, AfD, etc)
• concerns about community to cope with additional edits due to VE
• worry that to cope they might do anti-wiki defensive measures (ex. mass apply FlaggedRevs)

Oliver recommendations:

• further resourcing of AdminTools
  • What other tools can we give them short of "Special:Nuke" :-)?
• more outreach work to communities: "this is coming, better get ready "
• slower, graduated VE rollout

Sue: What would this (worst case) actually look like?

• James: Expect edit rates will go up, but less repeated edits to same page. edits per editor might go down (increase success due to preview/save workflow, time in edit decrease)
• James: we don't know for sure
• James: will need to do A/B testing
• Trevor: before save, show diff. Need way to preview diff

Sue: will it be easier to vandalize?

• James, Terry: Easier to do "sneaky" vandalism, just changing a number or word
• Trevor: But will be counterbalanced. Limit the "piano key" variety of toolbar buttons (keep vandalism away from complex stuff)
• Roan: But will make it easier to e.g. edit infoboxes
• Trevor: vandals will always edit, but the point is to give great people the ability to edit. Hope to get them involved in review too.
• Howie: automated tools, e.g. AbuseFilter
• James: will have global AbuseFilter soon (community could write an AbuseFilter filter to deal with VE-related abuses)
• Erik: UX when triggering AbuseFilter in VE?
• Trevor: need to pay attention to that
• Trevor: We have an opportunity to detect vandalism the moment it is happening (in the VE on the UI)
  • Not sure how to surface that
  • but we could do this in software for common vandalism issues
  • should look into ways to provide immediate feedback (e.g. "this link is blacklisted")
  • this could AGF so AbuseFilter doesn't trigger when people are well-meaning
• Erik: philosophically: Complexity of Wikitext as barrier to bad edits is stupid idea ;)
  • If barriers are needed, they should be intentional/planned. If system can't sustain open-editing anymore, then we should attack that orthogonally.
  • Trevor: Wikitext is a crappy IQ test.
• Gabriel: longer term, make HTML diffs available
  • This explains things in the terms people understand visually

Sue: A/B testing discussion (and deploy on limited set of articles)?

• Will be touched later in the meeting

Technical Risks & Mitigations

1. corruption from Parsoid/VisualEditor saves
   • could re-enable diff before every save (not really helpful to the new user, but helps current user to surface problems)
   • Roan: could make it mandatory if things go really wrong
2. Missed features
   • there is a list here ordered by significance?
3. High demand/overload of parsoid service or cluster
   • they're testing scalability resources
   • work with Ops
   • Trevor : Doubled machines, CPUs at 1-2% currently
   • Gabriel: currrent peak rates : need between 25-100 machines (assume all edits are to the Barack Obama page)
   • Erik: Would nice to have that order in this fiscal year :-)

Discussion of media impact:

• James: could mitigate by graduated rollout (e.g. by class of users). Problem: awkward to explain to users why we don't care to give it to them, media coverage
• Sue: rollback after/during big media coverage is costly because most people are low info and they won't come back
• Erik: opposite path is very doable (slow rollout starting with experienced users first)
• Trevor: Rollout over few weeks in July, no singularity
• Sue: Realistically, tech press will pick it up as soon as live on enWP.
• Erik: Perhaps launch to logged in users would be picked up too.

Metrics to consider

A/B testing:

• cohorts based on (newly registered) users (not articles)
  • ex. all new user signups (on enWP) for one or two weeks, half gets VE
• Sue: Learn about vandalism, easier UX. Erik: use user metrics API

metrics

• number of pages created
• number of edits made
• number of edits to same article in a session (failure rate)
• time to edit
• revert rate (hopefully will be down)
• time to reach x edits.

Sue: non logged in users?

• James: no good way to track them, in particular to distinguish new/experienced users in anon edits
• Roan: (no consistent way) to give consistent experience to logged-out users, not switch between VE/wikitext arbitratry
• Sue: this A/B testing looks like a way to explore the upside of VE. What would be a way to directly explore worst case scenario that people are talking about?
• Erik: Pick two schools in the US as target groups ;) Agrees that we should look at ways to research this.

Potential roll-out plan (illustrative, not definitive)

James:

• (pre): Team in Amsterdam
• 27 May: Read-to-go code test deploy
• 3 June: A/B metrics test & review
• 24 June: go/no go decision
• 1 July: VE default for enwiki
• 15 July: VE for top 10
• 29 July: VE default for all Wikipedias (depends on Parsoid)
• +1.5 weeks: Wikimania

• default editor means the edit tab goes to VE
• pullback after rollout would be awkward
• Trevor (re Wikimania): best case: all goes well, can focus on conference / worst case: will have whole team in one place (Hong Kong) to troubleshoot
• Erik: Right now, almost no-one is using VE (even less than mobile editing), so we are getting very little stress testing. Are there other steps before ?
• Trevor: Use CentralNotice, worked well for Vector rollout (in 2010)
• James: testing in other languages important
• Roan e.g. have standing bug from hewiki
• Erik: Parsoid hasn't been updated since December, when next update to move to 2-week cycle?
• Gabriel: had to rewrite serializer, now stable in roundtrip testing
  

Roan: needed rollback in January because of serializer. Also want to do performance testing
Erik: why low usage rate?
James, Sue: because it still lacks needed features, in particular citations
Gabriel: template parameters worked on. roundtrip success back up to 98% after serializer dip
Erik: worry most about pre-May period
James: will change integration in April, so that people with VE in preference will get it as default

Questions/Thoughts

Sue: some questions before moving on:
1. hearing not much emphasis on media? e.g. because of shifting dates
Trevor: we are not Apple ;)
James: partly don't want media attention, brings bugs etc ;)
Sue: opposite argument would be: brings new editors
but probably enough new editors try out WP every day to grow this organically, users brought in by media attention might be too far up the funnel, compared to more savvy/motivated users who try out themselves. Howie?
Howie: Media stuff will get us one-time bump, but cannot replace sustained marketing effort.
Sue: Okay so we are saying: not media for sake of media. Communications team will be prepared, but not make it priority of communications/sink lots of resources into it. This makes sense because the upside of media coverage is not that big (natural growth is what we're aiming for) and there is a small risk of a big media splash followed by a rollback which would be frustrating for new editors. The no-media focus might be surprising to the communications team because VE is our top priority this fiscal: it needs to be conveyed to Jay.
Trevor: media coverage (at launch) means drama. much better story to get out would be looking back at successful rollout sometime afterwards

Sue: 2. Community outreach:
James: half of me has been doing this ;) sent out updates to wikitech-ambassadors
Wiki page with info on VE
Erik: for fortnightly updates, can use global message delivery
Roan: Parsoid had been multilingual for a while
Sue: need campaign, also to reach non-interested areas of community. not just two weeks before launch. E.g. Oliver or CA should get on this
Erik: recall Usability Initiative (that was when ambassador group was formed), this was the last time we had big multilingual UX changes
assume every language gets biweekly updates, invite ambassadors for this particular rollout, leave out village pumps for those wikis/languages where ambassador exists
Erik: might have some more contractors like Oliver soon
Sue: e.g. Fundraising had helpers like that
Scott: minor languages might be in most need of messaging
Gabriel: RTL/LTR variance within a single article is an issue
Trevor: ask i18n team?
Roan: e.g. they don't have speaker of Serbian (2 variants), or kk (3)
Erik: I can help connecting

2013/14 Planning

(slide 29 ff)
James:

• early Q2: stability, performance, scalability
  • pay tech debt, internationalization, establish as platform and attract non-Wikimedia or even non-MediaWiki contributors
    

Erik: interested parties?
Trevor: Wordpress might be a candidate (their VE is pretty bad), has similarities with us, has big community. or Etherpad
Maybe also other smaller projects, even if less intensive relationship whtn with Wikia
Trevor: In Bay area, some ex Google Wave people, Etherpad, etc
vision of VE as "editor for the web"
Relicensed to more permissive license (MIT) early on (when only ca. 15 peope were involved)
saves money, broadens horizon

• early Q4: alpha real-time collaboration (in production, session opt-in)
  

James: need to figure out revision history, diffs, attribution (need to do better than Google Docs)
Trevor: always envisaged this, even though it had to be second as priority
looked at Google Wave etc
Sue: also realtime chat?
Trevor: Yes, every realtime collaborative tool has chat. Docs has this because otherwise people chat in document itself
James: could do comment on blocks of text
Trevor: value to ephemeral stream of messages
Sue: just to flag issues: vandalism/harassment/define ephemeral nature
Trevor: might be for established connection/invite only

Parsoid

• support HTML-only wikis, integrate HTML processing in core and remove dependency on Parsoid. See https://www.mediawiki.org/wiki/Parsoid/Roadmap
• performance and efficiency

2013/14 additional objectives

VE

• table editing
  

Gabriel: priority of this vs. realtime collaboration?
James, Trevor: might be orthogonal. platform vs. feature
Trevor: problem: there are very few pure tables in existing wikitext, most are generated from templates
want to get on to the more exciting community building stuff
James: also, over time, corpus will shift automatically to content produced using VE

• micro-VE for log entries
• visual diffs, history playback
• editing galleries (crucial for Commons)
  

Erik: (diffs/history playback) this seems crucial, any prototyping done?
Trevor: not as much as hoped. might be great GSoC project
Trevor: important: respect privacy expectations
Erik: That would only be important for detailed session playback
Gabriel: blame function right in HTML with annotations. develop architecture for storing HTML

• Integration with other priority development areas, e.g. Flow, mobile
• non-Wikipedia wikis (e.g. ProofreadPage on WIkisource)
  

Wikia will do very limited rollout in July, default everywhere in September

Parsoid

• DOM-based templating
• Fragment caching
  

See https://www.mediawiki.org/wiki/Parsoid/Roadmap for details.

Questions/Thoughts

Sue: This was really useful.
Usually in these meetings, team presents asks/needs/blockers.
Here, I see mostly community liaison need?
Trevor: could use more emphasis/resources for CE (ContentEditable), have someone working
Gabriel: We were blocked on Parsoid side by lack of manpower, now improving with Scott joining as contractor temporarily, hope for longer term solution
James: I myself would like to be able to spend more time on community outreach
Erik: is working on delineating work of CA and Engineering liaisons more clearly
Sue: would be fine adding some resources for contracting some more community members
Roan: liaises with Ops and other technical teams. Wants to get estimate to Ops in April for number of machines needed. Relationship with Ops is good at this point.
Gabriel: will need to work with Aaron on jobqueue, storage infrastructure (but might not need add for HTML parsing / storage things)
Sue: getting the message that we are on track. A lot of complexity, things that could go off track, bridges to cross. but reasonabe hope to hit targets 100%
Team is now bigger, feels more confident, has accomplished more.
Gabriel: not a lot of wiggle room in Parsoid because of manpower
Erik: wants to highlight that this is a drama-free team, very productive in last 12 months and before, is very proud of their work.
Everything on track to hit July milestone, even though we can't control every eventuality
Terry: Have we done a good job managing expectations?
Sue: Board understands complexity (now)
With community, we haven't done a lot of work yet informing the uninterested editors.