Wikimedia monthly activities meetings/Quarterly reviews/Mobile contributions/September 2014
The following are notes from the Quarterly Review meeting with the Wikimedia Foundation's Mobile Contributions team, September 25, 2014, 1PM - 2:55PM PDT.
Present (in the office): Leila Zia, Dario Taraborelli, Kevin Leduc, Ellery Wulczyn, James Forrester, Toby Negrin, Rob Lanphier, Max Semenik, Lila Tretikov, Monte Hurd, Moiz Syed, Ryan Kaldari, Kaity Hammerstein, Dan Garry, Howie Fung, Erik Möller, Tomasz Finc, Tilman Bayer (taking minutes), Vibha Bamba, Maryana Pinchuk, Jared Zimmerman, Jon Robson, Carolynne Schloeder, Terry Chay (joining later); Participating remotely: Bahodir Mansurov, Oliver Keyes, Yuvaraj Pandian, Kristen Lans, Dmitry Brant, Arthur Richards, Brion Vibber, Bernd Sitzmann, Jeff Hobson
Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material
This is probably the largest crowd ever at a Mobile quarterly review ;)
this is for Mobile contributions (both app and web)
catch up until current time, i.e. including present quarter
Bahodir joined mobile web team last week
Sam and RobM moving over
right now, mobile is a third of all our traffic
Lila: how does this compare to other sites?
Maryana: we'll get to that
had an inflection point (general US data from comScore): now more than 50% UVs are mobile
Dario: caution - their mobile reports are very recent
Maryana: industry-wide, even higher mobile ratio, e.g. Facebook (68%), Twitter
Mobile users are shifting to apps
Toby: but keep in mind that this is US-only data
except for news sites: mobile web > mobile apps
Lila: where would syndicated content (Flipboard etc.) appear here?
Maryana: mobile web portion
Howie: this is all third-party data, we don't know details about methodogy
major takeaway: trend towards apps, with exceptions: news sites, and us
example Pinterest: mobile rise after native app launch, now 75% app traffic
example Facebook: >80% active user mobile, 30% mobile only
apps are a big driving force
Our mobile growth though is organic
- browser -> search -> WP
A lot of trends we are seeing - also editor numbers - are probably influenced by this
Lila: in those top ten examples, how different is the app experience from the web experience? do they have different use cases and interaction paradigms?
Jared: example Pinterest; they first made the mistake to emulate desktop in app, huge backlash
now: some interactions very different
even though use cases are the same
our mobile environment today: ...
User engagement spectrum: Who are we supporting?
Lila: this is about web / app?
This is about everything
Dan: some app uses come via web - in Google search results, it asks you if you want to open via app. People told us they do actually use that, set it as default
Lila: this overview is very helpful
scale is sort of about amount of knowledge users have about WP (none / a lot)
Dan: a lot of these people e.g. don't have user pages, might not regard themselves as Wikipedians if you asked them
Lila: it would be really important to quantify this
Howie: we have data about that
3 major initiatives
tablet switchover to mobile
it was a bit reactive - we knew the UX was really bad
we designed a mobile site that worked especially for tablets
added things that would not work on phones, e.g. uncollapsed sections
clean UI, not cluttered
some people had actually already opted into mobile site - which was really hard to do (needed to find a small link somewhere)
Lila: can we target these 5% for survey, ask why they went back?
and, why the decline afterwards?
Oliver: this is because of weekends
Lila: OK, where are we now?
Lila: so how did we get this data then?
(generated by Oliver specifically for this occasion)
Erik: new infrastructure that Analytics works on will provide that
will be able to get that sort of data on a regular basis
Oliver: caution: additional work needed for identifying tablets
Erik: let's just re-run the previous
Lila: you guys need to know that data to make decisions
mobile site currently has higher activation rate (new editors making first edit)
Lila: are these productive edits, do we subtract reverted ones? no
This is great data
Howie: keep in mind that one needs account to edit on mobile web
so let's not read too much into the difference
still, a lot of people editing on tablets and phones
in previous world, had "born Wikipedians" with high motivation to edit
these people exist on mobile too
mobile edits are smaller
one would think that iPad users are making larger edits, but the difference to phones is not big
Dan: mobile UI (section edit links) encourages smaller edits
VisualEditor on mobile
not full VE, but some elements of it
still opt-in, because:
- client-side performance issues - mobile devices still don't have the same computational power as desktop. e.g. half a minute for w:Barack Obama
Lila: only article, or talk pages too?
(Maryana:) only articles
- are features mobile appropriate?
e.g. don't need bolding a lot (often only for bolded title in first sentence, when starting a new article - but that's not possible on mobile anyway currently)
and filling out forms (for citations) is hard on mobile
acquisition is higher on apps - not a surprise (one gets asked to sign up when installing app)
activation rate is lower, for the same reason (so not a reason to panic ;)
Howie: note - anon editing is possible on the app
Dan: caution: small sample size on apps
built this for:
- casual reader - quick way to look up information
- power reader: saved pages for offline reading, e.g. commute, developing countries[?]
Oliver: how do we know that?
found no strong correlation between usage and country[?]
Lila: can't we just do that via caching?
Dan: might run into memory limits pretty fast, better to have explicit option
62K accounts created - recall, there was a prominent CTA
500K pages saved
Oliver: where did that number come from? Eventlogging
engagement might be about the same on iOS and Android
it's not "build it and they will come", need to promote app specifically
session lengths longer on apps
Oliver: did not confirm that these are the same users
Dan: yes, see footnote on the slide
Lila: concerned about the low number of sessions/month
now that we know more about mobile readership behavior, no excuse not to improve their experience
we do see highly motivated contributors coming to mobile
need to cater to them
Dan: 10% of app sessions are 18min or more - that's a long time
Monte: also, they care - give us feedback
Toby: about definitions - do we differentiate between engagement and retetion?
Maryana: not yet
how grow readership?
why do we grow these numbers?
it's in our mission: sharing the sum of human knowledge
in contrast, Google and Facebook care about eyeballs because of ad revenue etc.
we are low on the list by average time spent on site (Facebook etc. have been focusing on this for years)
We have not worked on this
need to work on reader engagement
for that, need to make the apps awesome
1. better search and surfacing content - e.g. notifications, widget, integrate with other content
typo tolerant search - not implemented on apps yet
Dario: prefix fetch vs. fulltext search
Lila: we will fix this
Tomasz: we should never show no search results
trending articles - what people are editing (or reading) right now
Lila: great idea, but you should have several of these ideas and test them
also, think about localization (not English only)
Dario: some of these depend on services that we don't have yet
Vibha: (separate slides)
slide: the more you read, the more you are likely to edit
0 edits person wants consistent experience with previous
comfort level is function of several things, e.g "prevent me from failing"
saved articles as vector to get people spend more time with app
app is a great place to try thing like this
tagging, providing Wikidata description might be more accessible task
we are going to fail with a lot of these ideas, but have big wins with 2 or 3
this will help existing community too, e.g. adding Wikidata descriptions improves searchability
Lila: really happy you guys do that kind of thinking
get together for a few days and develop some blue sky ideas
people who e.g. come online in Global South might learn quite differently
should have a group that develops such ideas, and open it to community, let them hack on this - it doesn't all need to come from us
Growing mobile contributions
App and mobile web: editors
we do have a new class of mobile editors coming in, retained at high level
they are mobile only
editors stay where they start, don't see a lot of dots in the middle (people doing lots of edits both on mobile and desktop)
Lila: perhaps because of people who don't have desktop computer?
Maryana: recall this is ENWP
Oliver: found that mobile is better at encouraging people to edit at hours they wouldn't have before. but not a significant difference between countries
this is right after tablet switchover, so probably some test edits
not huge changes, like refactoring of whole articles
e.g. see a mistake, fix it
power editors (e.g. create Featured article) - on desktop , not apps
what this means for our work:
serving most editor use cases right now (with our limited means)
no wiki rabbit holes on mobile
porting desktop features will not always be a good idea
Lila: think about (blank slate approach)
Kaldari: experimental mobile web feature (on Beta)
at the end of section, message "improve Wikipedia by tagging information on this page"
suggests guessed Wikidata property (profession, guess from links in articcle),
user provides yes/no confirmation
Lila: how do you know what is accurate?
Kaldari: right now collect these into temporary table, will check them before submitting to Wikidata
later can start submitting directly
Lila: the idea and direction is great
we're likely to end up with a lot of garbage
Maryana: not necessarily: when testing this with people on the street, they coped well with this task
Lila: how do we know this succeeds or fails?
Maryana: ratio of total answers to quality answers
Toby: this aims to drive engagement, or retention?
Maryana, Moiz: engagement
but other features retention, too
Jared: right now it's purely opportunistic (need article with missing Wikidata item)
but could become more purposeful, lead into a space where one does that taks repeatedly
Toby: tradeoff with other things one could do, be mindful of opportunity costs
focus for Q2
compare to annual goals
this is not the time for power editor features (like recent changes, reverting, Flow for user talk pages)
Lila: I'm sold on this ;)
Dario: risk of creating a population of atoms - e.g. no talk page
Maryana: most people can't use talk pages anyway
Oliver: OK, talk pages have horrible UX, but that does not mean we can ignore that sort of communication
if my edit is bad, how will I learn about it?
Lila: fair question, but don't need to solve it in this meeting
Maryana: might seem scary, but could be incredibly rewarding
could revisit it later
Lila: I need to leave - please work on this, document success and failure criteria
Analytics - need these desperately
Toby: you have two people
Kaldari: should have a frontend person on architecture cabal - so far only backend people
Brion: sounds good, can talk about that
Jon: mobile goals for non-mobile teams
Tomasz: we talked about "mobile first", but we're not there yet
Lila: it's a process, need to be clear about mobile first
Maryana: also for new feature
not everything can be measured
but strive for better relationship with qualitative evaluation
more design and product focus on apps - e.g. Moiz has to work on tons of different projects
Lila: Thanks! (leaves) (2:40pm)
(back to presentation)
Jared: convergence mobile/desktop
e.g. for Yelp, the bridging mechanism was draft review
(take notes on mobile in restaurant, edit them for publication on desktop at home)
organization of UI
does not mean things look the same, but that they can be understood in the same way
design patterns: branding
look the same --> mean the same
should still "feel" like the Wikipedia space
lessons learned from Flow - was built for desktop from the beginning
on mobile, can just about read the first topic only :(
Erik: right now pushing for it from architectural standpoint: make everything available via API, at least
need to see in next few months to see if there is a use case for mobile
Toby: what about consistency of mobile experience if every team develops for mobile themselves?
also, worry it might slow down experimentation
Jared: even small things like colors can make similar things look very different
also, standardizing placement of e.g. notifications will help
but can also be comfortable with some differences until productization
Maryana: e.g. VE was conceived before we even had a real mobile site
Tomasz: team who develops product is in best position to decide about consistency, not team who is handles it afterwards
Maryana: limited resources means we need to pick focus
Erik: agree with everything said here
but: we still all have laptops in front of us in this meeting ;)
shift to mobile just means more casual users
so it's still worth focusing on improving the crappy desktop experience
the two come together in a positive way