Wikimedia monthly activities meetings/Quarterly reviews/Wikipedia Zero/March 2014

The following are notes from the Quarterly Review meeting with the Wikimedia Foundation's Wikipedia Zero team, March 5, 2014, 1pm-4pm

Present: Carolynne Schloeder, Toby Negrin, Jove Oliver, Tomasz Finc, Yana Welinder, Sue Gardner, Tilman Bayer (taking minutes), Adele Vrana, Dan Foy, Erik Möller

Participating remotely: Adam Baso, Ingrid Flores, Yuri Astrakhan

Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases and other official material

Agenda

I. Results YTD

Launches
Carrier marketing
Page view metrics
Technology
SMS pilot

II. FY Q4 Priorities

Target metrics
Framework
Regional strategies
Product roadmap
Other priorities

Welcome/Intro

Carolynne:
This is a great timing for a review
last 6 months: transition, me ramping up, hiring team, refining processes
Will go through launches in the next slides
last few months more reactive than proactive, but it's great to get lots of requests
seeing huge CSR awareness with partners, PR potential

Results YTD

2013-14 Goals: pageviews, two major new partners, 20 countries, portal, explore USSD/SMS

currently: 23 countries, 27 operators
we might face a tradeoff in goals: # countries vs pageviews
good pace in Q2 - half from initial group deals, half from independent operators and inbound requests
Erik: all these are m.wikipedia.org launches?
Carolynne: yes, except Grameenphone grandfathered in as zero.wikipedia.org
Pipeline: 3 signed contracts, implementing now; around 25 operators in the pipeline, including three possible group deals and 10 very active.
Operators go quiet on us, but with increased team capacity, should be able to ramp up more quickly by staying more in touch with operators

Outreach by partners

examples:

Aircel: retail outlets,
social media (#IWikiWhen Twitter contest) - not a lot of effort needed,
blog event - quite a lot of impact
...Facebook campaign (teasers with questions where answers can be found on WP)

Airtel Kenya - SMS pilot - "Free-kipedia"
WP text nominated for GSMA Awards at Mobile World Congress (didn't win, but still nice)
Telenor operators Grameenphone Bangladesh and dtac Thailand are great examples of corporate social responsibility, e.g. in Bangladesh: press event and GP hosted "wiki contributors day" with chapter (WMBD) - to recruit and train volunteers
Carriers want to support development of small language Wikipedias, see it as building a national resource to support culture and development
With Heather, produced draft template poster that local communities will be able to adapt
Great press - best example: MTN video reaction :)

Metrics

clarify definitions of Reach:

Signed group level agreements cover 750M subscribers
vs. Launched - 417M subscribers (but they may not all have suitable devices)
vs. 300M (estimate) with internet capable devices

Erik: how easy is to convert to WP0 as a customer?
Carolynne: if on a basic phone, need to buy a new phone and data SIM (prepaid plan shouldn't be necessary, but might be required in some markets).
Erik: I.e. could WP function as a driver to buy internet phone? How reach people who don't have one yet?
Carolynne: Our SMS service is a way to reach people without an internet-enabled phone. Some operators may not let users open the browser without purchased data, we need to investigate. In reality people without a data plan don't even try to use the browser.
Sue: so we have internet incapable phones vs. capable but not yet turned on (enabled) phones vs. disabled phones
To ask Erik's question in a different way, can users in the second category (with capable but not enabled devices) use WP0?
Carolynne: Our policy is to zero rate without a data plan - anyone with an intenet-capable handset could access WP0, but we have to verify that operators don't require the user to have a data plan.
Sue: I think that (driver of internet phone purchases) is one of the motivators on the carrier side
Carolynne: yes, but need data showing this actually works
CSR argument works better for now
Sue: concern about CSR: incentive could be just to get press and a mention in their annual reports. With business case, incentive would be more about actual usage
Carolynne: agrees; views differ by carrier - some have strong CSR focus (like Telenor Internet for All)

35 million pageviews goal:
Limn chart (still some zero. traffic, but phasing out)
growth has come from big operators launches
Top 5 carriers 90% of PVs, top10 97%
Insights:

marketing drives PVs
decline in 16 out of 25 operators - IP mismatches?

Erik: correlation of this with operator / country?
Carolynne: looked at that, also competition within one country could be useful (tension with # countries goal here)
examples:
Two operators put WP on Opera mini speed dial: powerful driver
total mobile PVs vs. WP0 PVs (in respective countries) - growing slower than market, need find out why

Technical accomplishments

Dan:

Worked with Ops on IP whitelisting
Opera/proxy support, control banners for that
App support
SMS pilot

IP whitelisting

benefits:

key to getting HTTPS to work
reduces "URL creep"
does not distinguish by language. explain to partners that non-main languages only amount to like 2-3% of traffic, this alleviates concerns

Status:

want all to upgrade

Opera Mini support

Opera very important in (part of) developing world, as opposed to US and Europe
Carolynne: if an operator has an agreement with Opera, zero-rating is very easy, and they can use the speed dial. If the operator does not have an existing agreement with Opera, that's a huge hole for us, we cannot zero-rate through Opera.
(Dan:)
Opera's data compression is important to carriers
Local language support critical in some markets (e.g. Asia - Bangla, Burmese), Opera renders fonts as images, removes need for language support in phone's browser
Carolynne: working with Alolita on that

Apps

(Dan:)
worked with Tomasz to include WP0 notifications in new apps

SMS

Why SMS:
90% of subscribers in Africa don't use data
reach people we otherwise can't reach - pretty much every single phone out there
Carolynne: easy on cellphone side, hard to manage carrier integrations
Dan: grow WP awareness - even in higher end African markets, WP is not well known
Paying per MB and slow network speed deter discovery browsing. Users accustomed to Facebook as default, instead of doing random searches on Google
Sue: and, no mental model of encyclopedia
Carolynne: overall optimistic about mobile data adoption, growing broadband traffic, although it takes time
Dan: cost problem
Adele: lot of operators ask us specifically about USSD
in Africa, mobile internet penetration rose from 2% to 11% in recent years
Dan: SMS pilot goals:
improve technology, integration pain points, UX
package for operators to install themselves, or run by 3rd party like Praekelt
First pilot with Airtel in Kenya from Nov 2013 (small market - never done outside lab before)
Second pilot: include hyperlink to WP0 at end of article snippet, in addition to "next snippet" option
aim at 4x the usage
do user research (first part was just about getting technology to work, so we didn't want to do UX research there)
Sue: is it straining to use?
Dan: it's actually surprisingly usable
e.g. numbered menu for disambiguation, section headers
concatenate SMS for longer text, ~400 characters, press for next
Erik: last time, talked about SMS being cash cows, seeing it replaced currently (e.g. Whatsapp)
Dan: service is completely free in these pilots
callback messages about account balance, include a bit of space at the end for marketing
get a place in those marketing messages, in some portions of market, drives huge spikes
Carolynne: visible in graph, starting to see organic growth too
didn't show up in the beginning because of tech issues
Erik: what does number of users mean in the graph?
Dan, Carolynne, Yuri: cumulative # of numbers who have appeared in logs up to that point
Dan: pilot results:

had peak usage / overcapacity on carrier side, sorted out in the last few month
Praekelt did load testing
load problem for WP search engine

pilot still ongoing, a lot of uniques so far
High average number of article sections read each month
Yuri: that number indicates it's usable enough
Dan: shows uniques chart (Sept-March)
Sue: So roughly a third of users succeed in getting content?
Dan: or didn't enter a search term
Yuri: counted beginnings of a session, not users who got content
Erik, Sue: so earlier chart counts all users who expressed interest
Yuri, Carolynne: yellow line = got first SMS, etc.
Dan: this is where user testing comes in, will get answer to these question - 1st pilot was about getting tech to work
Yuri: March usage (3 days only) already saw a lot of users coming back, so not actually a drop (just incomplete month)
Dan: start of organic growth, but there is no discovery on SMS
Scaling WP via text: 3 models

WMF sanctioned: WP zero partnership, press releases, Praekelt integrates and hosts

Carolynne: as Kul said earlier, no carrier will want to offer this completely for free, have their own costs for this and want to cover them, maybe as part of subscription bundle without incremental costs
Erik: talked about this before, WP0 is always completely free
Carolynne: Everything has to be free concept will hold us back (distinguish Zero from other initiatives)
Erik: growth model for this? How would this register on pageviews? sessions can't be counted as usage
Carolynne: it's a bit like opening WP main page on browser
Erik: but at least that serves content
Suppose we say that reaching a person without any internet access in Kenya is worth 10x as much (which we don't suppose), but even with that kind of factor, will it scale?
Sue: carriers' costs are operating + opportunity costs?
Erik: small amount of data
Carolynne: might be possible to have USSD menu up to hyperlink/content free, but not on basic phones
Erik: can't be under WP0 brand
Carolynne: yes, not necessarily under WP0, but there is demand for this product
Dan, Adele: yes, operators clamoring for this
Carolynne: maybe we just need to get out of the way, putting it out as open source
Sue: so are the 3 models on this slide recommendations, options?
Just thinking aloud: WP0 known as free, other products which charge would dilute that
We are not necessarily opposed to products for which people need to pay, but not as WP0
But they would come out of this team, whose mission is to grow mobile usage in Global South
we don't have a formula "1 Kenyan worth 50 Germans" ;)
Carolynne: it's an issue on how we use it to promote WP0
Erik: knowledge as human right, everyone free access to WP
once we leave that, leave CSR/goodwill area
SMS is a huge money printing machine
2 worries:

unfree text product will dilute our messaging, internal mission

Carolynne: agrees should not be under WP0 label
let me jump ahead a bit
Preface by saying we think a lot about our values, having private sector experience a team, bridging the gap to private sector, getting it to support our mission. Yana presented at last Metrics on partner mission alignment

Slide on promotion opportunities, 3 models:
Short term promotions through Opera Mini only, 2-3 months, speed dial - low risk of cannibalizing WP0
This is more of a data trial model, not making WP avaialable for free forever
Sue: yes, teaser/loss leader
(Carolynne:)
carriers use zero rating more (just) for promo purposes now, cf. Facebook Zero
but could open funnel for us
would get data from Opera, carrier get comfortable with WP traffic.
other promotions (just link, no WP0 brand): hard to say no - no work for us, raises awareness a lot, low cannibalizing
Erik: potential indefinite cooperation with Opera?
Carolynne: yes, good relationship with people at Opera, they like us, give data, help with tech
Erik: # of Opera users ?
Carolynne: a lot - currently at 250M wikipedia page views monthly
Erik: also had good earlier contacts with Opera, can follow up on higher level
Carolynne: that's good to know
Erik: fine to have non-WP0 branded WP link on speed dial
Carolynne: (second scenario) Zero on Opera mini only (i.e., not via other browsers) is a slippery slope
Adele: only went for this as alternative to paid bundles, which we won't do
current momentum on these exceptions, goal is to expand to all browsers later
Yana: also, not in any countries - this was just ZA, in response to the kids' video
Sue: one of the challenges in pickup was when it was complicated for people, e.g. the warning about leaving and incurring costs scared off people
how will I know I'm not in free-land?
Carolynne: just won't see "free" message on other browsers
Sue: beauty of WP0 is simplicity - always free
we worked to simplify it (e.g. remove zero. domain)
Adele: MTN won't market it without Opera mini
Erik: other examples where we did it with only certain browsers?
Carolynne: some
Erik: why did MTN insist on it?
Carolynne: they want to encourage Opera mini use - control of speed dial, Opera mini compression is also a motive -- we are still working on reducing page weight
Erik: not ideal
Carolynne: yesterday, learned of example of operator zero-rating WP by themselves
Erik: some have actually been doing it for a long time
Erik: worst case scenario: high school student with crippling Wikipedia bill...
Dan: users monitor their data usage, and it is prepaid (making overage charges impossible), so it's unlikely
Sue: yes, but Erik's scenario still stands
Erik: haven't yet seen a compelling growth model for text, significant expenses

--break--

Dan: on scaling models for Wikipedia via text
First pilot is about tech and users, not scale
1st model: WMF sanctioned...
other 2 models: step aside, let others use our open-sourced code, no endorsement
3rd model: Praekelt steps aside too
our involvement (financial) would be about developing technology only
Adele: this would open up markets in Nigeria, Uganda, Ghana and Burkina Faso
Carolynne: 3rd model already happening where some created text gateway on their own
Erik: compared to that, what are the advantages of our code? e.g. for disambig?
Dan: yes, all the stuff that uses our API to prepare content
Carolynne: difference between 2nd and 3rd: with 2nd, we can feel comfortable about UX
Praekelt committed to project, wants to expand it, confident they can raise money themselves
Erik: how much tech work left to do?
Dan: 2nd pilot (April - June) will add hyperlink - not trivial, need to get our own URL shortener together, needs to be part of WP0
3rd pilot (Aug-Oct) SMS only - benefit: less reliant on carrier, don't need to integrate two different services (USSD+SMS), everyone has SMS, some markets do not have USSD - tied to GSM
Erik: so USSD (useful for menus) entirely carrier-operated?
Dan: yes, easier for carrier to offer free
Carolynne: with pilot 2, still making sure it makes sense as product
Erik: also, I understand it can serve different functions:
lead into WP0, lead into data usage, grow on its own
but not yet seeing that it is something we have to do, and if there isn't a compelling case for it, we shouldn't do it - team is small, this will be a distraction for it, and also there's the dilution of the core job of making WP available for free
right now, not comfortable on signing off on additional budget and extra time spent for the team
Carolynne: OK, good to have clear marching orders
Erik: if team has better ways to spend their time, OK to walk away from it
Carolynne: it's indeed taking some time from other work
Erik: also, what about lower speed internet (2G)?
Carolynne: most people in Africa still on 2G
Erik: also asking because SMS is a very limiting technology, will it stay large entry point for next 3-4 years? It would take time to get this program up and running.

Q4 priorities

Carolynne: target still 32 countries
Latin America still empty - long lead times
Erik: still want to use that number as target?
Carolynne: not necessarily
PV projection for Q4 - assumptions: possible launches, partners' subscribers, applied 0.2 pageviews per user, to project page view growth.

Regional strategies

separating by team members is important because of diversity
concern about Asia (to be hired)
Segment partners in three groups
1) Ingrid and Adele prioritizing by countries for reach
2) support all group operators
3) inbound requests: they are very motivated, mission-aligned
Need portal to handle the smaller markets
country segmentation by big/small WP, 2G/3G mobile market - translates into very different strategies
e.g. Thailand - small WP, 3G
Philippines - large WP, 2G
Erik: role of Apps in overall strategy?
Carolynne: no numbers yet
but strategically important. opportunity in Africa for app preloads
Tomasz: app will have support for (old) Android 2.3, for wikitext editing and browsing
Carolynne: and 2.3 percentage might actually grow in some markets, with cheap devices being dumped on them
Erik: are our apps preloaded on any devices currently?
Carolynne: yes, have Firefox OS, and we're working on Android preload with one of our partners, plus Qualcomm Reference Design for "Africa phone"
Erik: personally I would guess that this (2.3) is an area where user research would be valuable
understand that preloads might be a much bigger win in the long run that WP by text
team should drive the (software) requirements for this
Sue, Erik: how do Zero and Mobile teams work together?
Tomasz: on engineering level, designers too
Erik: on app, we have a lot of room to experiment, higher level of control (basically what operators have in the SMS case...)
Dan: but no engineering resources spent on SMS right now - basically just me meeting 1h/week with Praekelt
Erik: not saying we should not do WP via text, but...
Carolynne: Tomasz acts as connector between teams
Sue: this team knows stuff about needs in dev countries that the rest of the organization needs to know, glad that Tomasz is in this meeting...
Carolynne: that's why it's good to work under Erik ;)

Wikipedia Zero for SE Asia & LATAM

Ingrid:
match "who needs it most" (economy, education) with "whom can we help" (3G penetration...)
e.g. high need/low capacity: Thailand, Myanmar - needs more time, support
issue with LATAM (Latin America):
reluctance to zero-rate
Toby: do we know why?
Carolynne: not willing to give up the revenue
Yana: large monopolies [or duopolies], not a lot of competition
Carolynne: Ingrid also did analysis for Asia
Erik: Why Pakistan on needs support side?
Carolynne: she was looking at local languages
Erik: that approach is OK, but incomplete - absolute number of English speakers might still be huge, e.g. Pakistan
Carolynne: yes in the middle class, but the people we are trying to reach might not speak English

Africa and Middle East

Adele:
70 countries, >200 operators
25 million pageviews right now
How to move forward: look at top 5 group carriers in countries with 10M+ literate people
Add 7 criteria: e.g. income, Wp monthly views, brand awareness per Alexa as a ranking tool
Used priority index to define where to go next.
moving forward: active pursuit and developing existing partners (e.g. cleaning up IPs, Android rollout) and raising awareness
Erik: the listed Wikipedia text rollouts, would they be blocked on us?
Adele: rollouts would be pilots and/or open source code used by partners. Facing demand.

Product Roadmap

Dan: goal is to support 200+ partners with small team - need to automate!
WP0 portal: outside part for partners, internal for WMF partner managers
internal: track communications, testing etc.
external: MVP in 2-3 months, won't wait until perfect

Asks

external dependencies in next few partners:

work with Ops on portal architecture, ESI. recently they have been busy with datacenter migration...

Erik: need to discuss these two items
Dan: we already worked ahead on some code
Erik: working relationship? Brandon assigned to WP0 work?
Dan: yes
Adam: might not deprecate zero namespace immediately, working on it
Yuri: Brandon has been very helpful on e.g. SSL
Yuri: thinking about moving stuff to private wiki, blocked since about 2 weeks
Erik: I'm on that thread, some complicated issues still need to be solved. should continue conversations on expectations, collaboration
Sue: unfortunately I need to leave now (4pm), this was a lot of interesting information
Carolynne: real quick, we also have priorities to work on Comms. Excited to work with the new CCO and get a WP0 Communications manager (under new CCO)
Erik: on scrummaster support: current timeline is for hiring by around July/August. when do you need it?
Dan: Currently I'm filling out on this role, as well as PM, tension between these roles
Carolynne: two contractor request: 1. UI
Erik: Jared will support this
Carolynne: 2. CRM, in our budget. Plus Mobile analytics - analyst role (and third Zero engineer)
Toby: Dario will share JD, aim for hire end of July(?)
Carolynne: net neutrality
Yana: in the past, there hasn't been a conversation about this
now, new proposed laws, e.g. countries adopting Dutch law
some groups already started forming assessment of WP0 in this respect, with incomplete information (e.g. assuming WMF pays carriers -- which it does not ever do)
Erik: good to have our position formulated, would like to be able to refer to it.