Wikimedia monthly activities meetings/Quarterly reviews/Editing, July 2016

Notes from the Quarterly Review meeting with the Wikimedia Foundation's Editing team, 11 July, 09:30 - 10:30 PDT.

Please keep in mind that these minutes are mostly a rough paraphrase of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Present (in the office or remote): Trevor Parscal (head), James Forrester (product lead), Neil Quinn, Rob Lanphier, Katherine Maher, Roan Kattouw, Benoît Evellin, Maggie Dennis Amir Aharoni, Elena Tonkovidova, Erica Litrenta, Geoff Brigham, Gregory Varnum, Guillaume Paumier, Heather Walls, Joady Lohr, Joe Matazzoni, Joel Aufrecht, Kartik Mistry, Lindsey Anne Frankenfield, Michelle Paulson, Pau Giner, Runa Bhattacharjee, Sherry Snyder, Subramanya Sastry, Toby Negrin, Volker Eckl, Wes Moran

Open

Metrics

New active editors

Neil: Infrastructure changes. In the past, using numbers from Wikistats. But slow: can take up to a month to get numbers, and hard to break down numbers for investigation. Analystics eng. working on better solution

New metrics infrastructure

Neil: Building on solution by Aaron H.

Metric: Active editors

Neil: Using Editing data warehouse for all those metrics
Slight decline over the past 5 years in active editors

Neil: New definition and old wikistats numbers

Metric: New (registered) active editors

Neil: New active editors: more volatile than active editors

Neil: New active editors: one of the main drivers in changes we're seeing in active editors. Non-new active editors (red line) is stable.

Neil: Break down new active editors by platform (desktop & mobile)
Desktop new active editors ~ flat over the past 2 years
Mobile new active editors: decreased ~50% in 2 years

Neil: concerning mobile edits, there is a plateau since July 2015
intake of new registered editors on mobile has declined

Neil: Mobile edits broken down by registration status. Early growth: by registered users, because logging in was required (?) to edit. After we enabled anonymous editing on mobile web, a lot of edits started being done unregistered
Large portion of mobile edits by unregistered users: not a problem per se
Plateauing in mobile editing is an issue that requires further investigation
Katherine: do we have any metrics on the quality of anonymous mobile edits?
Neil: No. We have previous metrics on anonymous desktop edits. Will look at this.
Joe: Is there really a decline? This graph shows an upsurge
Neil: There is a decline of new editors on mobile; stability is not victory;
JamesF: Also remember that the Internet population is going up lots still. Merely having "stability" is not necessarily a success, it depends on whether readers are being well served, sustainably.
Joe: Research suggests anonymous editors on desktop have high quality; reason to expect a difference on mobile?
JamesF: Anecdotally, [missed some of this], but quality of anonymous on mobile seems to be higher than quality of anonymous on desktop, perhaps because of the difficulty of editing on mobile.
Neil: Rise in anonymous editing accompanied by a plateau in registered editing (?)
Katherine: Wondering how much this is related to the barrier to registering
James: Call to register was more prominent in the past
Heather: There could be other reasons why people choose not to register, e.g. they don't want to receive messages etc.
JamesF: there are advantages to create an account [...], but also disadvantages from some POVs, yes.
Toby: Do these numbers change anything about how you look at the number of editors about where people are editing from?
James: No, this gives a bit more insight into the numbers but we are looking at other numbers too, like the overall levels of editing and new editor retention.
James: This number originally calculated in Summer 2011 ("oh shit" graph). Flat-ish for a long time since 2011/12, with a slow downward component. This has been very high profile KPI, though imperfect, and we should be careful if we are to switch from it to something else.
Katherine: reformulating Toby's question: Is this going to influence product development?
James: Flat is not (necessarily) a success. The level of editing "inclusion" is what keeps me up at night; driver behind a lot of Editing's product work. Continuing to improve how people can edit, we still think the main area of concern is making editing simpler and a more positive experience on all classes of device.
Toby: Solid thought process; great to see more work on metrics; would be useful to see the bright line between the metrics process and the end goal. Maybe have another/new KPI that tracks what Editing see as the strategic issues here.

Metric: Edits

Neil: New calculation attempting to exclude bots, and now including all content wikis (previously we were just measuring Wikipedias). Uptake over the past year or so is mostly on Wikidata.
Katherine: [question about session] - in a nutshell, is it possible to track more than # of edits so we get a picture of how much time people are spending as opposed to simply how many edits they make?
Neil: Yes, and worth doing, since some people make many edits within a two hour window while others may do only a couple, but it is same amount of time spent....
Maggie: I'd love to see that
Neil: Aaron H. has done some work on sessions; we'd need to productize this.

Collaboration

Objective: Finish Notifications Work

Joe: finished up the Notifications MVP, including bundling notifications and Special:Notifications page. Thanks to Neil and J-Mo for metrics and inputs

Objective: Finish Notifications Work

Joe: Demo of Special:Notifications
Joe: Lot of new functionality designed to help people focus on the work they're most interested in at the time.
Michelle: Any bump in people getting more relevant notifications on their work (?)
Joe: No. We just launched this week. Because the page didn't do anything in the past, people aren't accustomed to using it, so we're working on getting the word out (blog post, etc.)
Katherine: Is that a metric you're going to start measuring? Measure of success?
Joe: We'll look at whether people are using that page. We have a number of notification metrics that we're working on—didn't quite get to it this quarter—and we'll be instrumenting them this quarter.

Objective: Design Research for Q1 Products

Joe: Not a published goal, but possibly even more relevant in light of Neil's discussion of new active editors.
Joe: partial credit—we didn't come up with a new shovel-ready design, but we identified a project that we've been doing a lot of investigation on.
Joe: Using ORES; aimed at retention issue
Michelle: Who are the edit reviewers? Are we doing any recruiting?
Joe: one of the big questions we have. Just establishing that platform won't be enough to get this started. We're thinking about it
Katherine: Have you presented this in any showcase etc.?
Joe: Not yet; we will.
Roan: (via chat) reviewing other people's edits isn't a new activity, it's just not a process/activity that we've supported much

- Other successes and misses

Joe: Thanks to Matt and Matthias.

Language

Runa - multiple focus areas this quarter. Team maintains multiple products.

Runa: didn't add any new features to Content Translation this quarter. Instead, focused on stabilizing the tool and preparing a plan for moving out of beta.
Runa: Wanted to add an estimated timeline, but when we looked at the scope of work called for by the criteria, it became too speculative.

Objective: Increased reach of Content Translation

Runa: Ongoing goal: addition of machine translation services. No new services this quarter, but new languages. We've observed that the sometimes low quality of machine translation services doesn't impact the low deletion rates of articles created using ContentTranslation.

Objective: Deploy Compact Language Links out of beta

Runa: Beta feature since 2014. Replaces long list with easier to use list. As a team, we were always talking about moving it out of beta, but we didn't have time. Invested in that this quarter: had to do a significant amount of development work, rewriting of old code this quarter.
Co-ordination with the Reading department
Toby: I think the way our teams collaborated was awesome; just try to give more advanced notice in future.
Runa: We reached out early in our development cycle but I agree we should think about it earlier and be planning our development earlier.

Objective: Address technical debt in Translate

Runa: No new features, but major maintenance work
Runa: Led to better performance for users as well when using the translatable page editing view.

Other successes and misses

Runa: a lot of enthusiasm from volunteers
Runa: encouraging fact: Norwegian volunteer developments added a highly-demanded language pair to Content Translation.
Runa: will continue similar community consulations in the future. Will discuss with other teams that have done consultations like this in the future, try to tweak the process.
Katherine: Did you do the consultation with the Technical Collaboration team or independently?
Runa: We consulted them but we probably need more help
James: pretty common in VE to run a consultation, and get no/little feedback until just before/after deployment change.

Appendix - articles published via Content Translation

Runa: 2 000+ articles created each week with ContentTranslation; upcoming milestone
Katherine: Have quarter-over-quarter measure? [Added to slides subsequently.]
Runa: we have monthly reports

Multimedia

Objective: Content interactivity

James: Some wikis (e.g. es.wikipedia) have started doing some community work on interactive content (like https://es.wikipedia.org/wiki/Juego_de_la_vida ). Most wikis: everything is static.
James: trying to dip a toe into interactivity with this last quarter's work.
Extended the gallery system very slightly: now: single image and you can paginate through the other images. Will roll out to production everywhere this week as a new option, ask for feedback. Editors need to change a gallery manually (gallery mode), no change for anyone unless it's manually enabled on each usage.
Katherine: What's the plan after that?
James: working with Commons community to encourage them to use it and see what further changes they would like.

Other successes and misses

James: We track upload failures. Worked on reducing them this quarter—now 20x fewer. We reduced a lot of technical debt.
James: Large upload support is very useful for videos, e.g. for Wikimania talks
James: Commons community very interested in ImageTweaks
James: Annotations: Structured data is a good fit for this problem
Katherine: (about file annotations, structured data) Are you also in conversations with the GLAM community?
James: Yes, the structured data bit more than the annotation bits. Newly have a monthly meeting with Alex Stinson to keep abreast of demands from the GLAM community.

Parsing

Objective: Improved media support in Parsoid

Subbu: Main reason for these objectives was to support VisualEditor team with galleries, and Reading dept. with audio and video
Subbu: most of our parsoid development time went into fixing some of the tech debt for goals from previous and current quarter

Subbu: moving to Node v4 and scap3 is blocked by the migration of Parsoid to the service-runner framework. Hence a higher priority task.
Subbu: replacing Tidy ongoing project over last 9 months
Subbu: Kunal has been working on Shadow namespaces (mainly addressing technical debt).
Subbu: PDF generation is primarily in maintenance mode. Scott is the main person here. Operations requested that we work on this during the past quarter to address some longstanding issues.

Product

Objective: Consolidate data sources

James: Goal was an intentional miss. Rather than putting undue burden on Analytics, we worked on other things
James: take dashboards on in the coming quarter instead.

Contributor rôles:

Guillame's work: https://meta.wikimedia.org/wiki/Research:Codex/Roles_of_contributors
James: really important part of our long-term product decision making.
Katherine: is this being complemented by the persona development? feels as though it's pulled mainly from academic sources; are we looking at combining it with more traditional user research?
Guillaume: goal of project was really to review the literature (there's a lot we're not aware of or using in the WMF). Synthesize something that would be broadly useful.
James: will be an input to persona work. Not currently a part, but that's the intention.

VisualEditor

Objective: Increase use of the visual editor

James: rolled out on WikiVoyage in 15 of 17 languages; not yet to Chinese (design issues) or Russian (community queries).
Michelle: is it being used?
James: VE has certainly become more used at the Wikivoyages (thousands of edits in the few weeks it's been on), no push-back.

Other successes and misses

James: lot of work on what happens in VE when you press letters on keyboard—sounds simple but isn't. Now on by default at 220 wikis; going to be working with Arabic next. Spanish and English are complex; Dutch community isn't ready.

General

Katherine: comment about presentation structure: makes sense to rotate order so VE doesn't always get smushed at the end.