Wikimedia monthly activities meetings/Quarterly reviews/Reading, July 2015

Notes from the Quarterly Review meeting with the Wikimedia Foundation's Reading team, July 8, 2015, 8:00 am - 9:30 am PDT.

Please keep in mind that these minutes are mostly a rough paraphrase of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Present (in the office): Toby Negrin, Terry Gilbey, Kevin LeDuc, James Forrester, Luis Villa, Geoff Brigham, Garfield Byrd, Zhou Zhou, Anne Gomez, Vibha Bamba, Stephen LaPorte, Kim Gilbey, Katherine Maher, Tilman Bayer (taking minutes), Lila Tretikov; participating remotely: Dmitry Brant, Arthur Richards, Adam Baso, Corey Floyd, Joaquin Oltra Hernandez, Bryan Davis, Kristen Lans

Toby, Terry: welcome

[slide 2]

Toby:
inserted some front matter to explain where team is coming from
focused on continuity
WMF hadn't thought strategically about readers in 15 years

[slide 3]

mobile apps goal
link preview feature (only on beta so far)
fundamentally different from how mobile web version works
success would be that ppl would click more - this was actually the case
need to discuss what this means
are pageviews the metrics we want to drive?
because this clearly (reduces them)
Lila: app context is different
so the 30% increase in links clicked means people get more information?
Toby: yes
but PVs decrease, e.g. a concern for fundraising

[slide 5]

(link preview screenshot)
also tested Hovercards, which is basically link preview for web
props to Vibha who designed this
Vibha: have good user feedback...
often people get first sentence, which is all they need
Toby: also related to stopping to think about article as core unit to display
Lila: which would be a big change...
Terry: objective was Android and iOS, but only have data for Android
Cory: did qualitative on iOS
Toby: on beta release
about 100k users
Lila: Beta users are a self-selecting group, does this skew numbers?
Can we automatically push updates for app?
Corey: yes for stable (app store), beta app has to be downloaded anew
Terry: this is success or miss goal?
Toby: two measures, two goals
Corey: ...

[slide 4]

Toby:
Gather
this is a bit challenging regarding the team's new focus, because it's as much about contribution (than reading)
not deployed to production
Lila: what were main concerns?
Toby: only 1% of users actually contributing
wanted team to focus on getting users to build collections, rather than sharing
Arthur: criteria for success - beta is production too?
Toby: was meant to be enabled in stable

[slide 6]

(Gather screenshot)
Lila: OK with putting this on ice for a bit, but seems like a feature we want
us to not forget about it, revisit next q
Toby: and last 2 experiments we made on microcontributions were successful
when we decided to stop WikiGrok, we said value wasn't there. but here it might be
Lila: agree
when we come back to this, want conversation with community what contributions would be most valuable
e.g. curation, tagging, categorizations
e.g. this thing could be valuable for categorization tasks
Toby: some of these features don't have the same tight feedback loop (press save, affect #1 google result)
Terry: cost of getting feature in production =?
Toby: don't have dollar value, but would need to cover e.g. processing community feedback, owning product going forward
Lila: so for next q, Gather is suspended?
Toby: yes
also ties into identity concepts (I like your edits, so I follow your Gather feed)
Vibha: could explore Collections [Gather] on apps
development is cheaper there
interactions more stable
Lila: could see someone following an editor's edits on mobile
Toby, Tilman: RC curation on mobile
Lila: it hurts to suspend Gather right now
Toby: there are learnings here, and we're building muscle
also earlier used WikiGrok to get people sign into beta
know how to onboard them now
Lila: awesome work, Vibha

[slide 7]

(Toby:)
successes
Design team raises valuable points about collaboration, Engineering stepped up, integrated Design into core workflows
Terry: did we have a goal for driving downloads? no
Lila: do we have connections with the Google and Apple app stores?
Toby, Dmitry: have some, especially technical
Toby: would like to work with Partnerships team on that
Vibha: they feature products that have new features, can't get featured without adding significant new things
Corey: for Apple store, adding support for e.g. Siri or Apple Watch (i.e. becoming a good platform citizen) important for getting featured
Toby: getting featured is really valuable

[slide 8]

[slide 9]

misses
Terry: question on how we measure pageviews, uniques
Toby: advantages: available, also for long time into past
on apps, have access to a lot of other metrics
Lila: iOS is concerning
Toby: had crashes in iOS, saw that in ratings
we use a 3rd party service to monitor ratings
quality matters
Lila: why crashes?
Adam: codebase had a big database change in April
Corey championed getting better crash reporting
Terry: how quick between identifying crashes and pushing fixes?
Corey: within a month, we pushed 2 releases
we learned a lot from that, now do smaller releases
Toby: crash reporting is 3rd party product, so didn't have to spend time on it ourselves

[slide 10]

Misses
Bryan had to interim manage new Community Tech team, distracted him from other tasks
Lila: perspective of CE team?
Toby: need to sit down with Luis and Rachel to explore
had some pushback
Lila: what did CT team do so far?
Toby: just forming, preparing
Kaldari coming up with charter
Luis: budget, junior engineers who need to familiarize themselves with things
--> Lila: as we look at MPL (master project list), explore adding other senior engineers to Community Tech team
Vibha: want to add that new app versions improved significantly over previous app versions. E.g. earlier, 50th percentile never opened app again after downloading
Terry: do apps increase our overall readership? or do they just move readers from one platform to another?
Toby: mobile continues to inch up
Terry: my challenge: mobile rising, but does it contribute to overall goal [or cannibalize]?
Lila: these things are hard to attribute
good benchmark: compare to industry trends
Terry: are we investing in the right places?
Toby:

[slide 12]

no deliberate marketing plan
will have app preload experiment in Brazil in q1
my personal feeling is that big marketing efforts would be premature, need to improve our stickiness first
Lila: should try to drive (app adoption) up from mobile web
Toby: Appendix: Q4 traffic stats

[slide 14]

[slide 15]

about a year ago, extrapolated that mobile would rise above desktop (around 2015), didn't happen
Adam: my hypothesis: business people still using work computers, GS not rising as rapidly as anticipated
Toby: also, mobile may see less pvs per user [?]
--> Lila: would like to have some people from Android team join Adele for this [Brazil/Latin America] discovery mission, write questionnaires, etc.
Toby: we will actually do some user testing in Mexico
Anne: timeline for preload not definite yet
Lila: good, that gives us some time to prepare
Toby: GS still has huge potential, core strategic question

[slide 16]

Lila: figure out how to drive engagement on mobile
what can make it take off exponentially
Toby: agree
Toby: right now can fundraise on mobile pretty well
app PV share like 5% would be the right threshold to start thinking about FR on apps

[slide 17]

James: drop at end of June might match editing data we have
Lila: not convinced PVs are right metrics, UVs might be more salient
Toby: FR too feels donations are tied to UVs, not PVs
Corey (on hangout): For mobile apps, we are working with Analytics to field test a better analytics platform that will help us better visualize metrics

[slide 18]

[slide 19]

Appendix Apps

[slide 20]

[slide 22]

Toby:...
Lila: need to get KPIs cleared up
might not be pageviews
Kevin: have last access (cookie for uniques counting)
e.g. HTTPS switchover messed up things
Garfield: do we have segmentation of readership, e.g. which readers can impact FR, editing
Toby: can segment by geolocation, user agent
start producing consistent dashboards
did a personas project with Abbey
this was first step for also thinking about this quantitatively
Vibha: apps is 1% now, have a long way to go, have work to do on mobile web
can't hold apps hostage because of FR on apps
Lila: when is app experience good enough to start incentivizing people to use it, and add FR [then]
Vibha: look at time spent/user in apps
Lila: agree

Further appendix slides

(See also Q4 Feature Results appendix)