Wikimedia Foundation Report, October-December 2015

See also the original PDF version of this report

2008

2009
Davos report (January) January February March April May June July August September October November December

2010
January February March April May (to come) June (to come) July August September October November December

2011
January February March April May June July August September October November December

2012
January February March April May June July August September October November December

2013
January February March April May June July August September October November December

2014
January February March April May June July August September Q2 (Oct-Dec)

2015
Q3 (Jan-Mar) Q4 (Apr-Jun) Q1 (Jul-Sep) Q2 (Oct-Dec)

+ Search

Wikimedia Foundation

Quarterly Report

October — December 2015
Q2 of the fiscal year

Foreword edit

We are pleased to bring you the Wikimedia Foundation’s Quarterly Report for Q2 of the 2015/16 fiscal year, a comprehensive summary of how we did on the objectives defined earlier in our quarterly goal setting process. We have continued to optimize the report’s format and the organization’s quarterly review process based on the feedback that we have received.

Teams have highlighted key performance indicator (KPI) - with ongoing efforts to identify the best possible metrics - and to estimate how much time fell into each of the three categories from the 2015 Call to Action (strengthen, focus and experiment). We have organized the content of team reports to present all the information that is related to a particular objective in one place (description of the goal, measures of success, how we did on achieving the objective, and what we learned from working on it).

As before, we are including an overview slide summarizing successes and misses across all teams. In a mature 90-day goal setting process, the “sweet spot” is for about 75% of goals to be a success. Organizations that are meeting 100% of their goals are not typically setting aggressive goals.

Wikimedia Foundation Leadership Team

2

Quarterly metrics scorecard (beta) edit

Participation
Active editors (5+ edits/month)	75.5k/mo	+0.6% from Q1 (July-Sept) +0% y-o-y
New editor activation (not directly comparable to active editors)	22,531/mo	-3.7% from Q1 (July-Sept) -10.2% y-o-y
Edits via mobile (on WP, percentage of total)	3.9%	Q1: 3.0% Q2 2014-15: 2.2%

Site reliability
Read uptime	99.979%	+0.037% from Q1 (July-Sept) y-o-y: N/A
Write uptime	99.974%	+0.039% from Q1 (July-Sept) y-o-y: N/A
Read latency Median first paint time	865ms	Q1: 642ms
Write latency Median page save time	1056ms	Q1: 1042ms

Readership
Page Views Crawlers excluded	15.6B/mo	+1.3% from Q1 (July-Sept) approx. -8.8% y-o-y

Content
New articles	16.7k/day	+119% from Q1 (July-Sept) +126% y-o-y
Edits (in Wikipedia articles)	10.52M/mo	-4.2% from Q1 (July-Sept) +9.1% y-o-y

Fundraising
Amount raised	$46.9M (exceeded $34M target)	+$2.9M y-o-y
On mobile (Dec. 2015 only)	20%	Q1: 12%Dec. 2014: 20.8%

3

Q2 2015/16: Successes/misses by team edit

4

Q2 2015/16 objectives overall edit

142 objectives
34 teams

92 successes (65%)

50 misses (35%)

Misses include anything not delivered as planned, including “yellow” goals and those that were intentionally revised or abandoned due to changing priorities.

5

Community Engagement edit

Quarterly review

Q2 - 2015/16

6

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Community Advocacy edit

Q2 - 2015/16

Approximate team size during this quarter: 7

Time spent: strengthen 20%, focus 50%, experiment 30%

Key performance indicators

SLA for Trust & Safety correspondence: Resolving 95% of emergency@ within three hours	100%	+/- 0 change from Q1 (100%)	+/- 0 change YTD (100%)
SLA for public correspondence: Resolving 95% of answers@ and business@ within two business days	100%	+3.5% change from Q1 (97%)	+3.5% change YTD (97%)

7

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Community Advocacy

Objective: Harassment strategy

Objective	Measure of success	Status
Innovate improved community health through creation of a community-informed harassment strategy. Team members involved: 6	Launch user experience survey Launch first round community consultations (30 days), engage with community throughout	Survey released on schedule, with 3,845 responses from 14 projects. Consultation launched on schedule and generated many suggestions for next steps on addressing harassment issues.

Notes: Survey went smoothly given scale with some wrinkles. Many volunteers helped make it and the consultation happen, with pre-release feedback and translations. The consultation was kept open beyond 30 days to permit more time for feedback.

Learnings: Launching surveys in the modern movement is complex, and understanding the requirements and reaching out to stakeholders was more time-consuming than expected. Future surveys will build in more time for procedural complexity. We also learned that we need better advanced coordination with other teams, like Community Resources, on how to pipeline community-promoted initiatives generated by such consultations.

8

Q2 - Community Advocacy

Objective: Refining T&S processes

Objective	Measure of success	Status
Improve execution of trust & safety by refining and clarifying processes. Team members involved: 2	New child safety (images) policy created and submitted for manager approval Implementation of new child safety (images) policy	The updated child safety (images) policy is underway and much research was put into it, including with the assistance of legal, but the policy update is not complete.

Notes: Other focuses at the Foundation particularly around strategy limited staff capacity, and the refinement of this process - given that we do have a functional policy - was deprioritized to allow us to complete other projects.

Learnings: The complexities of laws regarding processing inappropriate images of minors internationally makes establishing an international team for this workflow unlikely.

9

Q2 - Community Advocacy

Objective: Functionary liaison

Objective	Measure of success	Status
Focus on community health by supporting and facilitating improved collaboration of key functionary groups. Team members involved: 4	Successfully host conference of international stewards in San Francisco Collect, analyze, and reasonably implement feedback on how we can improve services to this group	The conference took place in October, with 15 attendees out of 37 invited. All stewards - not just those in attendance - were polled about how we can better assist, and a report completed on 25 November. CA is coordinating with other departments and teams to meet steward needs.

Notes: The Stewards in attendance voiced strong opinions that the meetings were useful for them to connect on better ways to conduct their work and better ways to coordinate with the Wikimedia Foundation on supporting the movement through their role. As a cost-saving mechanism, it would be worth exploring whether we could in future add special functionary days to broader movement meetings, like Wikimania, with funding provided for the extra day of attendance or as part of the scholarship work.

10

Q2 - Community Advocacy

Objective: Community communication.

Objective	Measure of success	Status
Devise and implement an experimental plan for a central community hub that will allow community predictable and reliable access to WMF staff. Team members involved: 6	Present proposed community hub plan for DCE, C-level and ED review Refined per feedback, launch experimental community hub	The design of the experimental hub was completed and approved. However, implementation was put on hold in order to maintain capacity for CA to more fully engage with the WMF strategy process.

Notes: The design was completed on schedule and cleared the review period, but put on hold to keep capacity for strategy. If capacity permits, it will be launched in Q3, but is not on Q3 Quarterly Goals due to need to retain availability for strategy consultation. It may be combined with hub pages serving communications and programs/affiliates into one hub.

Learnings: Remain flexible on release dates, but try not to release heavy engagement projects around major holidays.

11

Q2 - Community Advocacy

Objective: Maintain the core.

Objective	Measure of success	Status
Maintain the core. Team members involved: 7	Execute and maintain core workflows with 95% of inquiries responded to within 2 business days.	CA met KPIs related to defined inquiry paths and maintained core workflows in spite of staffing transitions.

Learnings: While CA grew in new ways this quarter with the addition to our team of senior strategist Haitham Shammaa to help guide us in new directions, core work continues to consume the bulk of our time and attention. Through consultation with Terry, Luis and Lila, CA learned that we need to protect the focus on our core even though that will mean committing to fewer additional projects. Our focus now is on using our added capacities to ensure that additional projects are properly focused to nurture global community health.

12

Community Liaisons edit

Q2 - 2015/16

Approximate team size during this quarter: 7 FTE (6 FT, 2 PT)

Time spent: Strengthen 30%, focus 50%, experiment 20%

Key performance indicators:

Individual contributors with 2+ edits in product pages	Q2/O - 915 Q2/N - 920 Q2/D - 929	+3.17% from Q1	N/A YoY
Page Views	Q2/O - 24,812 Q2/N - 22,891 Q2/D - 20,775	-7.31% from Q1	N/A ToY

13

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Community Liaisons

Objective: Improve understanding and input

Objective	Measure of success	Status
Goal 1 - STRENGTHEN Improve community understanding of and input to product development Team members involved: 2	Creation of 5-10 Principles for Design and Development, submitted and ready for presentation in January Finalize definitions of Community Liaison toolkit within product development	Principles are drafted but have not been submitted for signoff Community Liaison toolkit incomplete

The schedule for Design and Development Principles was built with delays for survey response and peer review, but end of year scheduling conflicts, holidays and vacation time created additional delays on obtaining signoff.

Discussions around the toolkit continue into Q3

14

Q2 - Community Liaisons

Objective: VisualEditor’s progress at enwiki

Objective	Measure of success	Status
Goal 2 - FOCUS Focus on VisualEditor’s progress at enwiki Team members involved: 4	VisualEditor on more accounts and logged-out users on the English Wikipedia; opt-in visible to all.	Goal changed after quarter start Single edit tab development work still on-going, expected early Q3.

Goals changed after quarter start for quality reasons: brought forward work from Q4 as new blockers.

Erica was instrumental in supporting conversations around VisualEditor at es.wp, resulting in VE being turned on per request, as well as reactivating discussion about launching VisualEditor at nl.wp

15

Q2 - Community Liaisons

Objective: Future Planning

Objective	Measure of success	Status
Goal 3 - STRENGTHEN - Future planning Team members involved: 3	Annual Planning process and finalize team narrative for upcoming year Make an offer to a community liaison for the Discovery team	Annual planning process beginning Q3 Offer to CL candidate made after goal deadline of 11/15

Our goal was to extend an offer to a Community Liaison before Thanksgiving. After adjusting the job description in mid-November, we extended an offer which was accepted by the end of the quarter.

16

Q2 - Community Liaisons

Objective: Maintain current workflows

Objective	Measure of success	Status
Goal 4 - CORE Maintain current workflows Team members involved: 7	Providing designers, PMs, and engineers with community feedback for products Participating in Reading strategy Maintaining Tech/News	Teams assigned into: Editing, Collaboration, Community Tech, Reading, Analytics

17

Developer Relations edit

Q2 - 2015/16

Approximate team size during this quarter: 3 FTE

Time spent: strengthen 60%, focus 20%, experiment 20%

Key performance indicator (NOTE: we are going to change KPIs)

Users of Wikimedia Web APIs	N/A (T102079)	N/A from Q4	N/A YoY

18

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q1 - Developer Relations

Objective: Wikimedia Developer Summit

Objective	Measure of success	Status
Goal 1: STRENGTHEN Successful organization of the Wikimedia Developer Summit 2016 Team members involved: 2	Logistics of a 3 day event in 2 locations are all solved. The schedule reflects WMF Engineering priorities. Scheduled sessions are preceded by online discussions involving their stakeholders. 25% of registration are volunteers or third party developers.	Successful logistics 2+1 days. Successful program organized in 5 areas, with pre-scheduled sessions and unconference. Successful prior discussions and note-taking. BUT only 15% volunteers / 3rd parties (25 total, last year was 15 - 10%).

Rob Lanphier (program) and Valerie Aurora (unconference) were key contributors to the event.

Attracting more volunteers and 3rd party developers in the Bay Area requires a different type of event and promotion.

19

Q1 - Developer Relations

Objective: Code Review performance

Objective	Measure of success	Status
Goal 2: STRENGTHEN Define potential actions to reduce code review queues and waiting times. Team members involved: 1	Proposal to address the code review queue of changesets submitted by volunteers, presented to WMF Product, Technology and Team Practices, and to our developer community.	Extensive research was made, and a draft identifying problems exists. However, we still haven't presented a proposal yet. This was a prominent topic at the Developer Summit, and we are integrating the outcome.

This goal is a stepping stone toward the definition of a Service Level Agreement by the WMF.

Partly thanks to DevRel's work in this area, today WMF developer teams agree that there is a problem with code review that needs solving.

20

Q1 - Developer Relations

Objective: Third Party Developers

Objective	Measure of success	Status
Goal 3: FOCUS Map current use of Wikimedia APIs Team members involved: 1	List of known users of Wikimedia web APIs specifying main uses. Metrics of quantitative use of these APIs	We ended up without resources for this goal. Strategic Partnerships worked on a survey, but that is not enough to have the list of known users. Quantitative metrics are still missing. We depend on Reading Infrastructure.

The root of the problem still is the lack of a clear WMF strategy around third party developers.

DevRel has no resources to research and reach out to 3rd party developers; our few hands are full with open source contributors to our projects.

21

Q1 - Developer Relations

Objective: Code of Conduct

Objective	Measure of success	Status
Goal 4: FOCUS Binding code of conduct for all Wikimedia technical spaces Team members involved: 1	Code of Conduct for technical spaces approved Committee created	The Code of Conduct is still being drafted. The main part has been reviewed. We are defining processes with support from specialists. The approval process is not defined yet. The creation of a committee has not started.

Matthew Flaschen, Moriel Schottlender, and Frances Hocutt are driving the drafting in a volunteer capacity.

WMF Community Advocacy, Community Resources, and Legal are contributing as well.

We agreed to contract services from Valerie Aurora and Ashe Dryden to provide specialist advice.

22

Q1 - Developer Relations

Objective: Wiki Loves Open Data

Objective	Measure of success	Status
Goal 5: EXPERIMENT Organize the start of Outreachy round 11 Team members involved: 2	Organization driven by volunteers. 5 featured project ideas. Selection process completed and documented.	Tony Thomas is the main org admin. He is a volunteer, former mentor and intern. We had 9 featured project ideas. The selection was completed, refreshing the related documentation in the process.

The Winter round of Outreachy is a good occasion to onboard volunteer org admins because the activity is lower than in the Summer round with Google Summer of Code.

There were more candidates selected by our mentors, but a new rule by Outreachy discarded all candidates with full-time studies during the internship.

23

Program Capacity & Learning edit

Q2 - 2015/16

Approximate team size during this quarter: 11.3 FTE

Time spent: strengthen 82%, focus 11%, experiment 7%

Key performance indicators:

TWL Unique recipients	2598	+ 8.98% from Q1	+ 23% YoY
Education program leaders served	229 (85 countries)	+ 22% (+ 39%) from Q1	YoY n/a
Community leaders engaged - learning and evaluation	96	- 49% from Q1	YoY n/a

24

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Program Capacity & Learning

Objective: Learning Infrastructure

(3 staff involved)

Objective	Measure of success (direct project hours)	Status
Focus	CR Quarterly Metrics roll-up & tracking support ✓	Complete.
Strengthen	Complete remaining program evaluation reports for 2015 ✓ Proposal Analysis for Round 1 APG ✓ Program design toolkits UX design research design ✓ Review of TWL branch outcomes: Review and revise setup guide ✓
Experiment	Global Metrics API tool successfully piloted in a partnership with analytics simplifying the pulling of global metrics by staff and volunteers by over 99% (estimated saving 800 hours of labor time per year for staff and volunteers reporting on grant funded programs) ✓

The Global Metrics API (appreciatively referred to as the Magic Button) takes the pain out of data collection for staff and volunteers identified in the most recent grants consultation. A collaborative partnership effort Amanda Bittaker (PC & L) Dan Andreescu Analytics) and Madhu Viswanathan (coding).

25

Q2 - Program Capacity & Learning

Objective: Program Infrastructure

(6.3 staff involved)

Objective	Measure of success	Status
Focus	Sustain and expand TWL branch activity & sharing of best practices ✓ Monthly TWL branch check-ins, document new community models ✓	Complete
Strengthen	Support pilot of a university course in Oman & two in Palestine ✓ Education Data Collection Campaign - 20 leaders ✓ Enhance editor access to research with new major partners and by extending the reach of those accounts ✓ Continue scaling TWL including building library partnerships. #1Lib1Ref Social Media Campaign ✓	Complete
Experiment	Build for a more efficient, robust, inter-connected TWL platform for the future X	Incomplete

Mature Wikimedia programs demonstrating scalability and impact require infrastructure and support for

continued growth and impact. This includes documenting best practices, community leadership development

global program advocacy and tools to support community volunteers. The road map work is

helping this integrated team prioritize supports across programs and communities.

26

Q2 - Program Capacity & Learning

Objective: Community Leadership Development

(5 staff involved)

Objective	Measure of success	Status
Strengthen	Community Knowledge Sharing: Consultation with WMDE on program design for Wikimedia Conference 2016 and WMIT for Wikimania 2016 ✓ Transition AffCom to staff supported model under PC&L ✓ 10 Community leaders recruited & onboarded as conference faculty ✓ At least 2 established Central & Eastern Europe education program leaders mentor 2 less experienced leaders. ✓ Identify and scope 2-3 projects with affiliates for Q3 and/or Q4 (e.g. Survey Bank) (10) X	Incomplete

Successes and Miss

We were able to launch the staff supported model for AffCom including 92 affiliates reviewed for Wikimedia Conference eligibility status, 28 affiliates renewed through reporting updates. Special appreciation to Jaime Anstee who has taken on a leadership role collaborating with Greg Varnum and Community Resources in this transition.
Full Affiliate project scoping delayed due to roadmap work and transition to AffCom staff model.

27

Community Resources edit

Q2 - 2015/16

Approximate team size during this quarter: 8.75 FTE (+2 part-time interns)

Time spent: strengthen+core 80%, focus 10%, experiment 10%

Key performance indicators

People supported Global metrics from reports by resourced initiatives this Q	59,428 total individuals involved	2,324 new editors (4% of total)	945 active editors (2% of total)

Grants to Global South approved this Q	27 grants	$527,291	48% of total #	12% of total $	-72% # from Q1	+61% $ from Q1	+93% # YoY	+17% $ YoY

28

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Community Resources

Objective 1: Pilot Simple APG

Objective	Measure of success	Status
Pilot simpler annual plan grants Team members involved: 1	Pilot launched with 5 participating organizations; applications submitted and approved for first round V1 best practices/guidelines for organizations Advisory committee onboard	4 grants approved for WMEE, Shared Knowledge, WMCZ, and WMES, totaling $175,979 of $234,229 requested. 1 application withdrawn (due to local regulations, in resolution) V1 good practices + budget template available 8-person committee with expertise from PEG, IEG, FDC, and AffCom delivered quality recommendations

Learning: Applicants need hands on support from staff and committee members to improve their applications throughout the review period; but with that support, results are great!

People: Winifred Olliff, 8 volunteers on the Simple APG Committee, the people behind our 5 amazing applicant organizations, and our volunteer and staff advisors, are making this pilot happen :)

29

Q2 - Community Resources

Objective 2: Community Capacity

Objective	Measure of success	Status
Pilot Community Capacity Development support with 2 emerging communities Team members involved: 1	Q2: 2 pilot projects launched, each focused on developing 1 capacity with 1 emerging community.	Behind schedule: 3 communities on board, projects are still being designed until end of Jan, haven't launched yet.
Q3/4: Pilot communities meet planned measures of success, demonstrate a move from NO/LOW capacity to SOME capacity in the target area		On track

Success: three emerging communities (Ukrainian [Ukraine], Portuguese [Brazil], Tamil [India]) engaged, toward building three different capacities, will allow us to maximize learning from first round of pilots

Miss: Planning phase took longer than expected; WMF turmoil and natural disaster (floods in Chennai) contributed to delay. Compressing community conversations to catch-up isn’t an option for building successful pilots, however.

30

Q2 - Community Resources

Objective 3: Wikimania planning

Objective	Measure of success	Status
Build a sustainable strategy for supporting Wikimania 2017-2021 Team members involved: 4	Decision finalized and announced for 2017 team	Montreal confirmed as host for Wikimania 2017
Input gathered for 2018-2021 strategy from Wikimania steering committee, WMF staff and at least 50 community participants 2018-2021 strategy draft begun on meta-wiki		Consultation launch was delayed due to other WMF priorities in Q2. Expected to close in January of Q3.

Miss: Q1 communications regarding selection process for Montreal were not clear. Revamped Wikimania planning process to come from consultation is aimed at avoiding repeat of this scenario for future years.

Learning: A dedicated Program Officer is needed to own events strategy overall, including integration between global and regional events. Much director (and other staff) time is spent filling in gaps.

31

Q2 - Community Resources

Objective 4: Maintain Grants Core

Objective	Measure of success	Status
Maintain full coverage for Annual Plan (APG), Project and Event (PEG), Individual Engagement (IEG) grant program workflows Team members involved: 7 (CR)	All PEG, IEG and APG workflows are fully staffed for H1 2014/15 Meeting commitments for 10 of 10 grantmaking workflows on-time APG core includes WMDE’s proposal for a restricted grant for Wikidata	Kacie Harold maintaining PEG, with 17 funded requests this quarter. 2 new applicants from Inspire! IEG round 1 funds 14 new projects (of 29 proposals), 19 grantees for a total of $83,113. ½ are tools-focused (most ever!) FDC recommended ~$3.77 million in Annual Plan Grants, including restricted funds for WMDE’s work on Wikidata.

Success: ½ of new IEGs are focused on tool-building, with partnership with CE-Software team and easing of IEG eligibility restrictions. Thanks to Marti Johnson, Quim Gil, and the IEG committee!

Learning: More engineering support is needed for reviewing APG and IEG grant proposals, given increasing technical focus of some grant requests.

32

Finance edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: … 5.5

Time spent: strengthen 35%, focus 60%, experiment 5%

33

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Finance

Objective: Implement Purchase Order

Objective	Measure of success	Status
Implement purchase order system to be utilized by departments and teams to make purchases. Team members involved: 3	100% of all purchases are processed via the purchase order system.	Project was not started due to the departure of both the CFA and COO. Also, this is put on hold until we hire a Purchasing Specialist and finalize a purchase order workflow/process.

Project will be discussed and reviewed with the new CFO to determine when it’s feasible to commence and complete. Commencement date is TBD.

34

Q2 - Finance

Objective: Annual Planning Process

Objective	Measure of success	Status
Create a master calendar for the Annual Plan process. Team members involved: 2	A standard and sustainable calendar that can be used for the current annual plan as well as future years.	For the FY 16-17 Annual Plan, a calendar was created to accommodate the FDC and community review and was approved by the C-Level team. Since this is our first year following the FDC timeline, we don’t know whether this can used as a master calendar going forward. However, we will incorporate all lessons learned from this year’s process and iterate as necessary.

35

Q2 - Finance

Objective: Optimizing Concur For Credit Card

Objective	Measure of success	Status
To optimize Concur for our monthly credit card reconciliations Team members involved: 4	All credit card reconciliations are being completed by the 8th of each month.	A majority of the reconciliations are actually submitted by the 4th business day of the month which is earlier that the 8th. At times, there are late submissions due to unforeseen events (e.g. lost supporting documents or difficulty in obtaining supporting documents from vendors).

36

Objective: Baseline Integration Process with HRIS and Payroll

Q2 - Finance

Objective	Measure of success	Status
To create a baseline integration process between HRIS/Payroll and Intacct. Team members involved: 2	Completion of integration process.	We currently have a manual integration process between ADP Payroll and intacct. Since we have not implemented a new HRIS/Payroll platform, we did not commence the work on creating a baseline integration process.

37

Talent & Culture edit

Q2 - 2015/16

Quarterly review

Approximate team size during this quarter: 7

Time spent: strengthen 50%, focus 30%, experiment 20%

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Talent & Culture - Boryana

Objective: T&C

Objective	Measure of success	Status
Boryana onboarding - Culture discovery process Team members involved: 3 Staff involved: numerous	Document, share & seek calibration & final confirmation of cultural pillars	Completed and posted summary on personal page for feedback and comments
T&C team strategy Team members involved: 2 Staff involved: numerous	Defined & documented strategy for T&C team, gain leader consensus	Strategy completed and shared with stakeholders. Strategy will get refined and project priority order may change depending on org needs.
Board Recruiting Team members involved: 2 Other involved: staff, community, board, BGC, advisory board, etc.	2 trustees appointed by board	2 trustees appointed by board

39

Q2 - Talent & Culture - Boryana

Objective: T&C

Objective	Measure of success	Status
Boryana onboarding - Culture discovery process Team members involved: 3 Staff involved: numerous	Document, share & seek calibration & final confirmation of cultural pillars	Completed and posted summary on personal page for feedback and comments
T&C team strategy Team members involved: 2 Staff involved: numerous	Defined & documented strategy for T&C team, gain leader consensus	Strategy completed and shared with stakeholders. Strategy will get refined and project priority order may change depending on org needs.
Board Recruiting Team members involved: 2 Other involved: staff, community, board, BGC, advisory board, etc.	2 trustees appointed by board	2 trustees appointed by board

40

Q2 - OD & Recruiting - Amy & Boryana

Objective: Recruiting

Objective	Measure of success	Status
OD Team Hiring Team members involved: 5 Staff involved: numerous	Hire Learning & Development Manager	PENDING - Finalists selected. Working through final logistics
Engineering Hiring Event - phase 1 Team members involved: 3 Staff involved: 3	Number of attendees	PENDING - Wikimedia Tech Talk scheduled for February 24 2016: 2 speakers (Moriel, Aaron H)
Agency vendor streamline Team members involved: 3	Bring all existing agency relationships into Recruiting Create process for Recruiting to be the only point of entry for agencies Create a preferred and approved agency list that will have standard terms and conditions	POSTPONED - This goal became less of a priority, thus postponed, as all recruiting needs were able to be handled in-house.

41

Q2 - Recruiting - Amy & Joady

Objective: Job Descriptions

Objective	Measure of success	Status
JD project Team members involved: 6	JDs reviewed, streamlined & leveled (for Engineering - other depts to follow)	Recruiting created template level JDs with HR for Product for each band. Draft Product level chart created in FQ1 by HR, to coordinate with Radford data.

Additional changes in Product/Engineering teams & roles made additional work on this goal challenging. Draft work has been finished but additional work with managers is needed before it can be rolled out to staff.

Learning: Blanket JDs for all Product will not work considering the variations required for each team, but general levels can be detailed. Closer coordination with Radford formats will provide the best matching to compensation and experience.

42

Q2 - HR Ops - Joady

Objective: Open Enrollment

Objective	Measure of success	Status
Successful Open Enrollment Team members involved: 3	Benefit budget increase kept under 13.6% (the budgeted amount for FY15-16) Open enrollment completed on time by December 12/11/16	Increase reduced by 5.7%, to 7.9% (95k annual savings) Open enrollment completed on schedule, with only 2 staff members requesting extensions due to their schedules.

It has been HR’s mission to create a quick and simple open enrollment, that minimizes work for staff, and maintains strong, stable & cost effective benefits.

Learning: Staff continue to value high contact with HR during open enrollment. Our brokers at Johnson & Dugan continue to represent WMF well to reduce carrier rates. PlanSource had some back end glitches due to a recent upgrade on their part, but staff did not experience it.

Elena successfully managed open enrollment, including communications to staff, several open office hours for staff, updates of all changes into staff plan materials, and coordination of the entire process with the brokers & carriers. Dan assisted with data coordination.

43

Q2 - HR Ops - Joady

Objective: Engagement Survey

Objective	Measure of success	Status
Run an updated engagement survey for req number staff, with high participation. Team members involved: 3	Run 2015 engagement survey for req# staff with 75%+ participation	Exceeded participation by 18% with 93% (2013 was 58%) Results shared on time Expanded goal: Added contractor survey completed with 84% participation

This was a last minute goal that was added for HR, and was able to be run on short notice. The project was expected to be a challenge both in managing expectations.

Learning: Next time conduct a simultaneous contractor survey, with questions limited to those that pertain to them. Include input from a broader group (than HR, Comms & Legal) in the planning process for the survey timing and sharing of results.

Anna successfully led the survey logistics, including: vendor selection process, question coordination with Culture Amp/HR/Legal/Comms, encouraging staff participation, running input sprints, and the added survey for contractors. Dan assisted with demographics data & reporting.

44

Q2 - HR Ops - Joady

Objective: Onboarding/Offboarding

Objective	Measure of success	Status
Req# onboarding & offboarding streamline - phase 1 Team members involved: 4 (plus 8+ non T&C staff)	HR, OIT & Administration has a fully documented process that integrates into a master process list.	Process maps for on & off boarding for HR, OIT, Admin, Finance & Tech Ops created on time and approved by respective teams.

With the process map created, this will help with the next phase of creating new systems to automate and streamline much of the processes, with the new HR Programs & Systems Manager.

Learning: Cross team collaboration can be challenging, with each team having different agendas and a different level of preferred detail.

Anna Lantz successfully led the project coordination, following up with all non T&C team members, making sure they stayed on track and informed.

45

Q2 - OD - Joady

Objective: Online Training

Objective	Measure of success	Status
Implement Training Resources and System Team members involved: 2 (and Legal)	Roll-out Mindflash for training access and tracking	Mindflash rolled out on time, with 5 T&C modules and 1 Legal module

HR and Legal needed a learning system that allowed for interactive training, that could be tracked and completion confirmed for risk management. Legal selected Mindflash in FQ1, with the system rolled out in FQ2.

Learning: More coordination will be needed for linking what staff want to have on Office Wiki and what is in Mindflash.

Legal led the project, with Joady & Anna advising. Anna led the addition and coordination of the T&C modules.

46

Team Practices Group edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 7 FTE

Time spent: strengthen 50%, focus 30%, experiment 20%

Key performance indicator

Positively impacting value delivered by supported teams	4.13 out of 5 (Likert scale)	baseline metric
Positively impacting sustainability of supported teams	4.47 out of 5 (Likert scale)	baseline metric

47

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Team Practices Group

Objective: Project Forecasting

Objective	Measure of success	Status
Improved metrics for product project forecasting Team members involved: 3	Burn-up charts actively used by three teams Five teams sample their maintenance fraction and two teams track it throughout the quarter.	Three teams were using burn-ups by end of quarter; six teams are currently using or configuring burn-ups. Eight teams planned to sample and five teams sampled. Three teams are continuing to track beyond Q2.

48

Q2 - Team Practices Group

Objective: Team Strategy

Objective	Measure of success	Status
Team Practices Group achieves consensus on the team’s strategy. Team members involved: 7	Defined and documented strategy for the Team Practices Group	Miss this quarter due to change in strategic focus in light of a number of organizational changes and divided focus across TPG slowing the strategy process.

49

Q2 - Team Practices Group

Objective: Tool Assessment

Objective	Measure of success	Status
Determine whether or not we can drive changes to Phabricator to enhance its usability as a project management tool in an environment with diverse workflows and data-driven needs. (multi-quarter) Team members involved: 3	Five high-priority project management-related Phabricator feature requests upstreamed	3 feature requests have been successfully upstreamed; 2 became obsolete. Intensive coordination is required to upstream, and we should continue to assess whether the impact is worth the effort.

50

Administration edit

Quarterly reviewQ2 - 2015/16

Team size during this quarter: 11

Time spent: strengthen 50%, focus 30%, experiment 20%

Key metrics and highlights of our core services can be found on final two slides.

51

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Administration

Objective: Refine and revise onboarding and offboarding with HR.

Objective	Measure of success	Status
Refine and revise onboarding and offboarding with HR. Team members directly involved: 3	Develop and produce collaborative workflow and documentation of all service teams supporting the initial onboarding of req# numbers.	Detailed work flow has been developed by OIT, HR, TechOps, Front office and Facilities to show all tasks and dependencies of onboarding and offboarding a req#

Facilities, Front office, and department admin representatives worked with HR, OIT, Tech Ops, to develop a comprehensive understanding and workflow of the steps involved in onboarding, and the various types of offboarding of req#. HR owns the workflow document.

Learning included understanding of risk dependencies and timelines, need for process on all types of onboarding, conversions, and planned and unplanned offboardings.

52

Q2 - Administration

Objective: Monthly Travel back-up submitted before 4th of month

Objective	Measure of success	Status
Create unique method of reporting transactional volume of centralized travel expenses (range of 200 - 500 transactions)	Travel expenses meet deadline of fifth business day each month during the quarter enabling earlier monthly close deadline needed in accounting.	Travel developed successful workflow within the current s/w and manual process to provide reporting necessary for early monthly close.

Will continue to work with data and in Concur and JPMorgan to see how to make process less time consuming.

53

Q2 - Administration

Objective: Develop Disaster Recovery plan

Objective	Measure of success	Status
Outline high level Disaster Recovery plan that complements Safety team work and documentation.	Team assigned, outline of process and office wiki page outlined, perform critical services inventory with each department.	Office wiki page outlined, team assigned, completed department interviews documenting critical services inventory done.

Key learning was to format recovery methods into levels or tiers of emergency. Team is working toward creating three basic business interruption scenarios to develop and practice various disaster communication and recovery tasks. Cross org team with Facilities and OIT and Safety.

54

Q2 - Administration

Objective: Evolution of Office space

Objective	Measure of success	Status
Outline process and launch team to determine critical needs for WMF work space; current lease terms September 2017	Real Estate broker and design team together with WMF will have outlined process, stakeholders and data needed to learn the critical priorities of our next working space.	Six Leadership interviews complete, space utilization data analysed, staff engagement exercise set up on site, Office wiki page set up to inform and gather additional information.

Key Learnings:

Staff input data largely reflects what is best for individual needs, and challenging for folks to think about what is best for organization. Need to push on this, and solicit more with this vantage.

Developing a scorecard to help with weighted needs next quarter. CBRE and VITAL work well together and bring different strengths to this important process.

55

Editing edit

Quarterly reviewQ2 — 2015/16

Approximate team size during this quarter: 31 FTEs

Time spent: strengthen 40%, focus 40%, experiment 20%

Key performance indicator; * – N.B. data for Q2 is for the first two months only

Monthly active editors on all wikis	76.3k average in Q2*	+1.5% from Q1 (75.1k)	+1.0% YoY (75.5k)

56

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Collaboration Team edit

Q2 - Collaboration Team

Objective: Cross-wiki notifications

Objective	Measure of success	Status
Focus: Improve awareness of activity by providing a cross-wiki notifications feature Team members involved: 7	Echo can deliver notifications between many wikis, allowing users to access their notifications from any content wiki on any other content wiki	Done. Roll-out is a Q3 goal.

● Cross-wiki notifications now available as a Beta Feature on Beta Cluster and on test.wikipedia.org.

● Further roll-out to all wikis over the coming quarter.

● Most of this awesome work done by Kunal, Moriel and Pau; thank you.

Image is a screenshot of code in MediaWiki/OOUI/Echo licensed under MIT/GPL.

58

Q2 - Collaboration Team

Objective: Flow opt-in beta feature

Objective	Measure of success	Status
Strengthen: Increase access to Flow by deploying and supporting an opt-in system for it Team members involved: 6	Users have the ability enable Flow on their talk page	Beta Feature was deployed to Wikidata and to Chinese, Urdu, Bosnian and Catalan Wikipedias.

● 462 users (~22%) opted in on the Chinese Wikipedia, 159 (~3%) on Wikidata, 31 (~6%) on Catalan, 10 (~23%) on Urdu, 6 (~22%) on Bosnian.

● Rough percentages for scale only, expressed as a proportion of ‘active editors’ on each wiki.

● The Czech Wikipedia community asked for this, then changed their minds.

● The Chinese Wikipedia deployment exposed technical issues due to IP block exemptions.

● Spam attack early in the quarter required us to bring forward improved anti-spam measures.

59

Q2 - Collaboration Team

Objective: Notifications prototyping

Objective	Measure of success	Status
Experiment: Increase engagement with notifications by prototyping and researching improvements to the notifications page Team members involved: 2	We have an informed plan for the next steps for improving the notifications page	Delayed.

● Completed one round of user testing, now refining design and planning second round in February.

● Research was delayed due to focus on cross-wiki notification research and delays on Design Research team’s side.

60

Language Team edit

Q2 - Language Team

Objective: User Engagement

Objective	Measure of success	Status
Focus: Improve user retention by adding additional 'suggestion' features such as customised lists and adding additional relevant notifications	Increase in the number of translations per user	Personalized suggestions integrated. Many enhancements for suggestions in Content Translation dashboard

New users have increased; percentage of returning users has also increased
Translators can create ‘Favorite list’ and discard suggestions
A campaign can be created using scripts
Suggestions are based on the previous translations
Teams involved: Language, Research
Tool is still a beta feature.
Blog post

Image is a screenshot of code in Content Translation licensed under GPL.

62

Q2 - Language Team

Objective: Infrastructure improvements

Objective	Measure of success	Status
Strengthen: Improve reliability by complying with WMF infrastructure requirements as defined by Services, Tech Ops and Security teams	Content Translation complies with WMF infrastructure requirements	cxserver successfully migrated to service-runner. Uses uniform service architecture to make the maintenance, logging, monitoring, analytics easy.

● API documented at https://cxserver.wikimedia.org/v1?doc

● Teams involved: Language, Services, Ops

Image is a screenshot of code in RESTbase/cxserver licensed under MIT.

63

Q2 - Language Team

Objective: Reliability

Objective	Measure of success	Status
Strengthen: Improve reliability by fixing high-priority bugs that affect basic functionality	Users can use Content Translation without interruption	High priority bugs fixed for saving and publishing errors. AbuseFilter related errors are displayed nicely so that users can fix and retry publishing.

AbuseFilter-related errors are displayed nicely so that users can fix and retry publishing.
Comprehensive logging of errors and daily analysis
The reliability of auto save and translation restore improved - compressed data and restore algorithm improvements

Image is licensed under CC-Zero and available on Wikimedia Commons.

64

Q2 - Language Team

Objective: Collaboration

Objective	Measure of success	Status
Experiment: Increase visibility for third parties by collecting and sharing parallel corpora of translation modifications	API will be able to access the parallel corpora	Development completed, pending deployment

Published translations API improved. Now lists revision ids, published date and unique id.

Parallel corpora (source-translation pair) API developed.
Deployment set to happen in late January 2016 on account of DBA scheduling.
Infrastructure also makes saving translations more reliable in long run.
Team involved: Language, Ops

65

Multimedia Team edit

Q2 - Multimedia Team

Objective: Drag-and-drop uploading

Objective	Measure of success	Status
Focus: Increase media contribution by integrating media upload into VisualEditor's media dialog and drag-and-drop system Team members involved: 4	Users can upload media files directly within VisualEditor by browsing or dragging-and-dropping	Done.

● Building on the work from the previous quarter and with support from the VisualEditor team, this additional feature was completed relatively early in the quarter.

● The cross-wiki upload tool to Commons is now used for approximately 1000 files by 527 users each day; this includes roughly doubling the number of uploads by first-time uploaders to Commons.

● The tool has roughly the same deletion rate for new users as other upload tools (see later slide).

● Thanks to the whole team – Mark, Prateek, and Bartosz – and to Ed from the VisualEditor team and Stephen from Legal for making this possible.

67

Q2 - Multimedia Team

Objective: Improve upload funnel drop-offs

Objective	Measure of success	Status
Strengthen: Improve reliability by resolving UploadWizard bugs that cause users to start over Team members involved: 2	Users can use UploadWizard to mass-upload without interruption.	Done. Further improvements to come.

● Over the quarter we substantially reduced technical debt, refreshing the UI and fixing outdated code there, improving the error/warning system, and modernising archaïc server code.

● The new user error/warning system means fewer unrecoverable errors, and provides us with much better contextual information to fix rare cases where they occur.

● These changes will help us continue work towards our longer-term goal of allowing users to resume old uploads and go back and forth between steps.

68

Q2 - Multimedia Team

Objective: Image editing prototype

Objective	Measure of success	Status
Experiment: Improve multimedia editing workflow by prototyping non-destructive image editing Team members involved: 2	Users can make basic changes to images such as crop, rotation, etc. directly on the wiki.	Done. Additional work forthcoming.

● We showed off a demonstration version of the prototype image editor to a lot of positive feedback.

● Our next step will be to make it available on Commons. We will assess real-world usage and needs, get feedback, and decide what next should be done.

● Many thanks to Prateek and Mark for their work on this.

69

Q2 - Multimedia Team

Objective: Future content type research

Objective	Measure of success	Status
Experiment: Improve value of content by researching the learnability of content to inform future prioritisation of feature development Team members involved: 0	We have an informed plan for next steps for multimedia contribution features.	Delayed.

● We continued our lightweight engagement with Design Research to collaborate with academia to consider possible options, but no substantive work was completed in this quarter.

70

Parsing Team edit

Q2 - Parsing Team

Objective: Parsoid extension registration

Objective	Measure of success	Status
Focus: Improve editing experience and draw closer to Parsoid HTML for read views by implementing support for native Parsoid extensions Team members involved: 2	Parsoid has an extension registration mechanism and native support for galleries	Native extension registration mechanism in place. Support for <translate>, <tvar>, and other tags used in testing. <gallery> deferred to next quarter (but have a prototype implementation from Wikia).

Required for supporting extensions that:

● … use wikitext internally (T110909)

● … need custom editing support in VE

Arlo did most of the work on this goal.

72

Q2 - Parsing Team

Objective: Replace Tidy with a HTML5 parser

Objective	Measure of success	Status
Strengthen: Modernise parsing infrastructure to use modern Web standards by replacing Tidy with a HTML5 parser in MW core Team members involved: 2	MediaWiki has an HTML5-compliant parser for fixing PHP parser output	HTML5 parser in place. Reliance on Tidy bugs/features means incremental fixes. Last step will replace Tidy. Working on visual diff testing to enable this rollout.

Parsing changes (Tim has been working on a lot of this with input from rest of the team):●

Done: Empty

items no longer stripped; rendering diffs hidden using CSS styles.● TODO: More CSS fixes needed and will be rolled out incrementally after testing.● TODO: Potentially some changes to parsing of wikitext - needs testing to identify impact on pages.● TODO: Enable editors to fix pages and templates that might break when Tidy is replaced. Testing infrastructure (Tim and Subbu have been working on different pieces of this):● DONE: Generalizations to visual diffing test framework to support mass visual diff tests.● DONE: uprightdiff: image diffs based on (video) motion detection to get actionable diff metrics.● IN PROGRESS: Puppetization and test mediawiki install

73

Q2 - Parsing Team

Objective: Prototype for balanced templates

Objective	Measure of success	Status
Experiment: Improve editing experience and take advantage of perf. opt. opportunities by prototyping some form of opt-in / opt-out system for balanced template output Team members involved: 3	MediaWiki has a mechanism for templates to specify their output as balanced or not.	Implementation sketch and RFC in place (T114445) Prototyping not yet begun. Proposal needs to go through the RFC process.

● Prototyping required in both the PHP parser as well as Parsoid.

● Will benefit from the visual diff testing framework.

● Replacing Tidy with a HTML5 parser will help with the PHP parser prototype.

C.Scott has been driving this work with input from rest of the parsing team.

74

Q2 - Parsing Team

Objective: Multimedia support in Parsoid

Objective	Measure of success	Status
Strengthen: Improve mobile reading experience and draw closer to Parsoid HTML for read views by improving multimedia support in Parsoid Team members involved: 0	Parsoid's HTML5 DOM spec and HTML generation have been updated for audio and video	No progress beyond some discussions.

● Got de-prioritized over all the other work since it turned out that mobile content services didn’t need it right away (compared to other requests)

● More progress expected this quarter

75

VisualEditor Team edit

Q2 - VisualEditor Team

Objective: Wider deployment

Objective	Measure of success	Status
Focus: Increase use of the visual editor by releasing it to more accounts and anonymous users on English Wikipedia Team members involved: 6	More editors use the visual editor.	Goal changed after quarter start. In-edit switching work complete. Single edit tab development work still on-going, expected early Q3.

● Goal changed after quarter start for quality reasons: brought forward work from Q4 as new blockers.

● Overall uptake is roughly the same as the previous quarter at around 16% (now ~10.5k edits/day).

● Gradually increasing on the English Wikipedia; now at ~4.5%, up from ~3.9% last quarter.

Proportion numbers reflect edits using the visual editor out of all article edits made by accounts & IPs, excluding registered bots, in the last week of December. Other tools (e.g. rollback, AWB) not excluded.

● In November we switched the Spanish Wikipedia back to default-on for accounts per their request.

● Particular thanks to Alex for his work on edit switching and single edit tab integration features.

77

Q2 - VisualEditor Team

Objective: Formula editing

Objective	Measure of success	Status
Strengthen: Improve support for math editing by improving the UX of the Math extension and researching its usability Team members involved: 3	It's easier to create and edit formulæ from within the visual editor.	Done. Deployed in pieces through over the quarter.

● We brought on Thalia Chan to simplify formula editing for users.

● We now syntax highlight the LaTeX, show a rendering preview, and provide a clickable browsable tray of all fragments to insert.

● Particular thanks to the volunteer extension maintainer, Moritz, whose advice has been invaluable, and of course also to Thalia.

Image is a screenshot of code in VisualEditor/OOUI/Math licensed under MIT.

78

Q2 - VisualEditor Team

Objective: Chart editing

Objective	Measure of success	Status
Strengthen: Improve support for chart editing by improving the UX of the Graph extension and researching its usability Team members involved: 2	It's possible to create and easier to edit charts from within the visual editor.	Done. Deployed in pieces through over the quarter.

● We brought on Frédéric Bolduc from GSoC to continue their work improving chart editing for users.

● We now let users create new charts, syntax highlight the JSON, make it possible to resize charts, and support a new version of the ‘Vega’ back-end for additional forthcoming features.

● Particular thanks to the extension maintainer, Yuri Astrakhan of Discovery, for their helpful support and vision, and to Frédéric for their drive.

79

Q2 - VisualEditor Team

Objective: Link suggestion experiment

Objective	Measure of success	Status
Experiment: Increase value of contributions by prototyping and researching integrating edit recommendations Team members involved: 1	We have an informed plan for next steps for integrating edit recommendations.	Development work done but late; deployment not yet undertaken, and so results not yet supplied.

● We worked with Research and Data to prototype a user tool to prompt adding cross-article links that were identified from reader behaviour but missing from the article copy. However, the work stalled and was not completed before the end of the quarter, which meant results to inform next steps are waiting.

● Lesson learnt: It was unclear who was leading this initiative, and so the drive to undertake the work was confused. Clarity on cross-team leadership should be established at the start of the work.

● Particular thanks for their design leadership to Nirzar, & to Ed for their rapid responsive prototyping of engineering solutions.

80

Q2 - VisualEditor Team

Objective: Mobile macro-design prototyping

Objective	Measure of success	Status
Experiment: Improve the visual editor on mobile devices by prototyping and researching using visual editor on mobile devices Team members involved: 0	We have an informed plan for next steps for using the visual editor on mobile devices.	Some product market research done, but the bulk of product and design research work was delayed due to more pressing other issues.

● We worked with Design Research to consider mobile editing editor concepts and how they match with the user archetypes.

● A wider market analysis for design research and prototyping of possible designs for user testing based on this was delayed due to competing priorities during the quarter.

● Particular thanks to Nirzar for their design leadership, and to the Design Research team.

81

Reading edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 28 FTE

(22 Reading, 4 Community Tech, 2 UX Standardization)

Key performance indicator

Global Pageviews (provisional*)	15.6 B / mo	+1.3% from Q1	-8.8% YOY (estimate, ±2pp)

82

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Reading

Objective: Engagement

Objective	Measure of success	Status
Strengthen: Drive engagement by launching Link Preview on mobile web beta	Engagement (5% increase in links clicked) in beta or on non-EN Wikis	Put on hold

Android link engagement stats promising
Not strong enough engagement to push to mobile web Q2
However, we need to reconsider metric

83

Q2 - Reading

Objective: Engagement

Objective	Measure of success	Status
Strengthen: Drive engagement by launching Read More on web beta	Engagement (5% increase in links clicked) in beta or on non-EN Wikis	Mobile web beta: January 1-14: 19% engagement when panel seen. Internally referred pageviews as a percentage of total pageviews in mobile web beta climbed from around 52% (Th, Nov 19) to around 67% (Th, Jan 14), whereas stable channel internally referred pageview rates were roughly unchanged (approx. 25%).

Scalability
Co-design
Difficulties of bolt-on designing in desktop

84

Q2 - Reading

Objective: Apps Engagement

Objective	Measure of success	Status
Experiment: Prototype services based architecture for web and apps	Working web app with feature parity of current mobile web stable reading experience (not scalability, quality).	Done. The specific single page app approach used for the prototype will not be used, based on discussion at the summit (full preso). An alternative, iterative approach will be used.

Image loading should be optimized
Reference sections should be loaded smarter
Hidden navbox HTML should likely be disabled completely
A lot of mobile users don’t actually expand sections, although this begs additional questions

85

Q2 - Reading

Objective: Apps Engagement

Objective	Measure of success	Status
Experiment: Drive app retention via feeds and modern UX (notifications Q3)	iOS App overhaul released 7 day 15% of users retained (up from 10% in Sept ‘15) (Feeds T104415, Modernization T104510)	Missed. However, the app was released to beta during the quarter. Currently in public beta and gathering user feedback.

Over 1,100 beta testers, with >30 responses on our form or via email
Very positive feedback on some changes (visual design, feed concept) as well as thoughtful concerns about some elements (navigation flow).
After project definition became clearer, pushed timelines by a few weeks to prioritize quality over schedule.

86

Q2 - Reading

Objective: Apps Engagement

Objective	Measure of success	Status
Drive engagement by integrating more types of content and media into the app (see appendix).	General elevation of UX and usability to strengthen user engagement and retention.	Done: Integrated Maps in Nearby, integrated with RESTBase, lookup of highlighted words from Wiktionary, playback of audio pronunciations, and full support for Android 6.0 (Marshmallow) design guidelines.

Led to the app being featured in the “Best of 2015” collection on Google Play, and resulted in a surge of new users.
Caused an appreciable increase in Maps traffic (see appendix), and improved engagement with the Nearby feature.

87

Q2 - Reading

Objective: API Engagement

Objective	Measure of success	Status
Focus: Measure MediaWiki API usage, and data dump loads.	Availability of usage metrics on Hadoop warehouse for analysis. (T102079)	Missed. Progress has been made towards building the metric pipeline but tooling challenges blocked final implementation in November when there was time to implement. Work will continue in Q3.

Data pipeline from MediaWiki to Hadoop via Kafka is brand new tech at WMF
Discovery and Analytics have been working out best practices
Reading thought Discovery pilot project was done in Q1 but technical challenges extended through most of Q2
Still some discussion about "best" encoding to use for data being pushed through the pipeline

88

Q2 - Reading

Objective: API Engagement

Objective	Measure of success	Status
Strengthen: Investigate migrating API traffic to OAuth.	Decision on if migration is desirable; then measurable impact based on existing usage (currently unknown)	Done. Decision was that encouraging mass migration is not advisable at this time.

OAuth cannot be used by all API consumers as it needs "shared secret" protections. This means that consumers such as native applications cannot securely process an OAuth handshake.
OAuth is currently recommended for API consumers who need special permissions when interacting with the Action API (e.g. making edits, reading protected content)
Requiring OAuth for all Action API access is undesirable
Cacheable API requests will necessitate more anonymous API usage, not less and will be important for scaling the Action API just as it is important for scaling web request processing

89

Q2 - Reading

Objective: MediaWiki Security

Objective	Measure of success	Status
Strengthen: Release Auth Manager: MediaWiki plug-in security architecture	Usage (# of services using new architecture) (T89459)	Missed. Lengthy fundraising code freeze and security review backlog contributed, but ultimately the work was more than 1.5 FTEs could accomplish in the quarter.

Single user OAuth (useful for bots) in 1.27.0-wmf.10
SessionManager component in 1.27.0-wmf.11
"bot passwords" feature for bots that cannot use OAuth in 1.27.0-wmf.11
Created php-session-serializer library for use by MediaWiki and other FLOSS projects
AuthManager progressing. Current estimate for full deployment is late February with uncertainty narrowing.
Security interested in collaborating on follow on 2factor auth project in Q3

90

Q2 - Community Tech

Objective: Establish credibility

with stakeholders

Objective	Measure of success	Status
Strengthen: Establish baseline for measuring community satisfaction with WMF tech support	Complete Tech Support satisfaction poll across multiple Wikimedia projects and languages	Poll completed Oct 22, with 467 participants from 10 wikis. Results posted on Meta Nov 4. [1]
Focus: Include Wikimedia communities in goal development and prioritization	Complete cross-project Community Wishlist survey, with community members voting on priority	Community Wishlist survey completed Dec 14, with 634 community members participating. Results announced Dec 16. [2]
Focus: Address current high-priority community technical requests	Complete 3 mid-to-large sized requests from existing community wishlists	3 requests: Fix HotCat on 100+ wikis (Nov), Fix and improve Citation bot (Dec), Compile gadget usage statistics per-wiki and cross-wiki (Dec).

1: https://meta.wikimedia.org/wiki/Research:Tech_support_satisfaction_poll 2: https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey

91

Discovery edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 12.5 FTE

Time spent: strengthen 40%, focus 30%, experiment 30%

Key performance indicators

User satisfaction	Start Q2: 15%	End Q2: 28% (implementation change)	-- YoY
Zero Results Rate	Start Q2: 33%	End Q2: 26% (normal variance)	-- YoY

92

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Discovery

Objective: Improve search language support

Objective	Measure of success	Status
Improve language support for search. Team members involved: 6	Run A/B test for a feature that detects language of user’s search query and adjusts results to match that language Determine from A/B test whether the feature is fit to push to production with aim to improve search satisfaction by 10% and reduce zero results rate for human searches by 10%	A/B test was run An A/B test report was produced The test did not show that the feature had the desired impact

After the A/B test showed disappointing results, the team re-focussed and implemented the completion suggester as a beta feature on all wikis (except Wikidata), which reduces zero results rate by ~10%. The beta feature was then rolled out. Thousands of users are now using the beta feature. User feedback has been overwhelmingly positive. The feature is in early stages; more work remains to be done before a production rollout. We’re targeting Q3 for a more full rollout.

93

Q2 - Discovery

Objective: Improve www.wikipedia.org

Objective	Measure of success	Status
Make www.wikipedia.org a portal for exploring open content on Wikimedia sites. Team members involved: 4	Measure usage of existing portal Perform A/B tests to improve portal Decrease time each user spends searching by Y ms (exact number pending logging implementation)	Portal was migrated to more standard setup (git/gerrit); this took longer than expected Logging added to measure use Portal dashboard created First A/B test launched, but abandoned due to logging issues

Migration of portal to git/gerrit setup is complete but took significantly longer than expected; see “Other successes and misses” section for more details.

First A/B test was launched in December. Unfortunately, problems with implementation of data collection meant that the data from the test could not be used. The test was successfully relaunched in early January, and the initial analysis shows the test had a positive, statistically significant impact.

94

Q2 - Discovery

Objective: Improve understanding of satisfaction

Objective	Measure of success	Status
Improve understanding of user satisfaction for search by iterating and improving on the search satisfaction metric. Team members involved: 3	Design QuickSurvey to qualitatively assess user satisfaction with search Tie qualitative QuickSurvey results back to quantitative search satisfaction data Validate/invalidate current approach for measuring satisfaction	Our QuickSurvey was designed Deployment of QuickSurvey extension (owned by Reading) was delayed due to technical obstacles with EventLogging and the extension Deployment freeze and annual fundraiser reduced time available to run survey Our survey was deferred to Q3

In response to a request from the Head of Research and Vice President of Product, the Discovery Analysis Team dedicated one of its analysts at 33% capacity to the annual fundraising campaign to ensure its success. This reduction in anticipated capacity affected our ability to achieve this goal.

95

Q2 - Discovery

Objective: Analyse WDQS and Maps service

Objective	Measure of success	Status
Support ongoing evaluation of usage of Wikidata Query Service and Maps services to decide on what’s next for these services. Team members involved: 2	Continue maintenance of Wikidata Query Service and Maps dashboards Review user feedback on services Make decision on continued level of support for WDQS and maps service	Maintained WDQS and Maps dashboards User feedback broadly positive Decided on modest continued investment in both services Q3 goals for WDQS and Maps posted

Gradual adoption of maps service as standard continues. Now standard on many Wikivoyages (English, Russian, etc.), and on the Android Wikipedia app.

Wikidata Query Service usage also continues steadily. Prominent third-party Wikidata consumers are planning switches over to the Wikidata Query Service. We continue to support the maintenance of the service.

96

Q2 - Discovery

Objective: Create performance indicator for referrers

Objective	Measure of success	Status
Create recurring performance indicator on referrer traffic from the primary search engines and determine what features may have largest impact on referrer metric. Team members involved: 2	Create dashboard which displays traffic to our sites broken down by major referrers. Based on the dashboard data, make a list of recommendations for features which could be implemented to increase traffic from prominent referrers.	Referrer data analysed Dashboard created List of feature recommendations not completed due to analyst support being required for annual fundraiser, and lack of clarity around intended output

In response to a request from the Head of Research and Vice President of Product, the Discovery Analysis Team dedicated one of its analysts at 33% capacity to the annual fundraising campaign to ensure its success. This reduction in anticipated capacity affected our ability to achieve this goal.

97

Advancement and Fundraising Tech edit

Q2 - 2015/16

Quarterly review

Approximate team size during this quarter: 27 FTE

Partnerships: 7 FTE; Online Fundraising and Operations: 8 FTE; Foundations, Major Gifts, and Endowment: 5 FTE;

FR-Tech: 7 FTE

KPI

Funds raised	$46.9m raised in Q2	$54.9m raised Fiscal Year-to-Date	$13.4m needed to reach annual goal

98

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Advancement

Objective: $34 million

Objective	Measure of success	Status
FOCUS- Raise $32 million in Online and Email campaigns in the US, Canada, U.K. Australia, New Zealand, Ireland, Italy, and France	Amount raised	$43 million raised (preliminary number). We exceeded the quarterly goal to put us on target to reach the annual $68.2 million budget goal.
FOCUS - Raise $2 million through Foundations and Major Gifts	Amount raised	$4 million raised (preliminary number).
STRENGTHEN - Complete the Endowment Plan and messaging; Prepare to launch Endowment Campaign in January	Buy in from the board and key donors	Board approved endowment plan. External announcement in conjunction with Wikipedia 15. Conversations with key donors to begin end of January.

99

Q2 - Strategic Partnerships

Objective	Measure of success	Status
Goal 1 - FOCUS Create pipeline of strategic partnerships	Identify and tap potential partners with substantial impact around: reach, revenue, community, content, and/or other benefits	List of key potential partners identified and in various stages of outreach
Goal 2 - FOCUS Secure 10 new Zero deals	Successful launches	Launched 14 partners - 10 in Caribbean, and other countries included Serbia, Angola, Tunisia and Algeria. Total reach added in Q2 estimated at 28M subs
Goal 3 - STRENGTHEN Identify app and mobile web needs for Global South and coordinate with Reading team on feature roadmap	Deliver multiple feature descriptions for mobile web and app planning for Global South	Roadmap features with Reading team established. Features for Q3 include: better preload tracking, App support for new zero features, improved web page performance and more.
Goal 4 - EXPERIMENT Refine syndication strategy, potential customer assessment, and business case	Research	Work still in progress.# Launched discussions with two largest syndicators - Google and Apple Developed insights from survey Started analysis of API usage

100

Q2 - Major Gifts, Foundations and Endowment

Objective: $2 million

Objective	Measure of success	Status
FOCUS:Raise $2 million	Amount raised	$3.94 million raised (preliminary number as of Jan 12).
STRENGTHEN:Hold successful fundraising event	Host 200 people, Raise $200k, Strengthen relationships	Goals achieved, except revenue target - event raised $120k
STRENGTHEN:CRM discovery	Decision on best CRM options	Plan to make improvement to Civi in FY16-17.
FOCUS: Finalize endowment terms and prepare for launch.	Public launch of endowment. Ready to begin conversations with key donors.	Announcement at Wikipedia 15.

101

Q2 - FR Tech

Objective: $32 million

Objective	Measure of success	Status
FOCUS: Support the Big English fundraiser	Fundraising creative reaches their donation goals in December	Done - 3.46 Million separate online donations in Q2 2015
STRENGTHEN: Get France campaign tech-ready	Fundraising creative reaches donation goals in October (in France)	Done
EXPERIMENT: Support Banner History rollout	Fundraising creative is able to see patterns and problems with banner consumption, before December	Done
EXPERIMENT: Start investigating SmashPig as its own code project	Rough estimate of the work involved to create the open source project -OR- a decision not to move forward with independent payment libraries	Didn’t Start. This evaluation was only going to proceed in Q2 if it didn’t interfere with mission-critical goals, and the preliminary conversations turned out to be extremely disruptive for all parties. Decision to halt progress on this goal until a less complicated time, was made early in the quarter (mid-October).

102

Q2 - Online Fundraising

Objective: $32 million

Objective	Measure of success	Status
FOCUS: Run and publicly post a reader survey on best banners of 2015	Survey posted in November with launch announcement for English campaign	Survey is posted: https://upload.wikimedia.org/wikipedia/commons/2/25/Wikimedia_Reader_Survey_November_2015.pdf
STRENGTHEN: Test & run campaigns with new World Pay and Amazon integrations	France campaign completed Have Amazon tested and included as payment option for US during Big EN	Done
STRENGTHEN: Add a backup credit card processor	Have backup tested during Big EN	Done
EXPERIMENT: Increase sales at the Wikistore	Increase sales by adding the store link to the Thank You page	Done. More info in the slides to follow

103

Communications edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 7 FTE

Key performance indicator: See slides 6, 7, Scorecard appendix of department report

104

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Communications

Objective: Message platform

Objective	Measure of success	Status
Message Platform Produce clear message and proof points in support of organizational strategy.	Clear concept and phrasing for top-level strategy message Supporting proof points and messages	Postponed to Q3 based on strategy process timelines

105

Q2 - Communications

Objective: Brand strategy

Objective	Measure of success	Status
Brand Strategy Complete brand discovery process. Present synthesis of findings and recommended course of action.	Recommend course of action to develop effective, powerful brand strategy	Brand discovery summary for Board Further work ongoing for Annual Plan Complete with all measures of success met.

Reviewed stakeholder statements, known challenges, comparable case studies, and external vendor services for problems

Brand brief including: Organizational philosophy; Vision & mission; Being effective; A reason to unite, Measures of success; How much change; Options of approach; Moving forward; Timeline draft
Finalized deck for Board presentation

106

Q2 - Communications

Objective: Wikipedia’s 15th birthday

Objective	Measure of success	Status
Wikipedia’s 15th birthday Create strategic messaging opportunity to engage new Wikimedia stakeholders and audiences.	Clearly defined WP15 campaign WP15 microsite/AR staged by 12/30 for 1/15 launch 15 AAA Wikipedia media pieces secured 5+ global community events planned	‘Knowledge is joy’ concept 15.wikipedia.org launched ~7.5M site impressions 400+ press stories, trending positive 149 listed community events Complete with all measures of success met.

107

Q2 - Communications

Objective: Core

Objective	Measure of success	Status
Timely, professional, first-rate support on core and reactive communications workflows.	Evaluative chart of core workflows and SLAs	See Core workflows and metrics (slide 20 of department report). Complete with all measures of success met.

108

Legal Team edit

Quarterly review

Q2 - 2015/16

Approximate team size during this quarter: 11 FTE

Time spent: strengthen 70%, focus 25%, experiment 5%

*Temporary staff this quarter: 7 legal fellows/interns.

109

Q2 - Legal

Objective: Public Policy Strategy

Objective	Measure of success	Status
Public policy strategy: address and support as appropriate key issues that could threaten or compromise mission and projects Team members involved: 4	1 persuasive opposition to intermediary reform in EU; quality comments on mass digitization; develop baselines through censorship study for future measurement. All to ED/GC/community satisfaction.	Intermediary Reform Opposition submitted to European Commission. Comments submitted to the US Copyright Office on mass digitization in early October. Entered into contract with Berkman Center to begin censorship study and develop baselines for future measurement (ongoing).

110

Q2 - Legal

Objective: Training

Objective	Measure of success	Status
Training: address and support as appropriate key issues that could threaten or compromise mission and projects Team members involved: 2	Selection and implementation of training software that permits flexible presentations on multiple legal topics. Q2 all staff legal presentation to 275+ employees.	Implementation of software training.

111

Q2 - Legal

Objective: Innovation

Objective	Measure of success	Status
Experiment: Facilitate and provide bandwidth for unplanned innovative projects with no or minimal legal hurdles (5%). Ensure proper delivery of endowment analysis in support of Advancement team. Team members involved: 3	Satisfaction of ED and C-level peers. Deliver plan on new endowment, including a cost/benefit analysis of various structures to satisfaction of Revenue Officer, ED, and Board.	Provided support and detailed analysis of endowment to allow the Board’s approval of an endowment plan.

112

Q2 - Legal

Objective: Core

Objective	Measure of success	Status
Core: Top notch, quick legal advice and support on wide host of issues constituting our 34 legal workflows. Categories include: Transactional Litigation Privacy Public Policy Trademarks Governance Technology Fundraising Training Team members involved: 11	KPIs	Turn-around rate for contracts exceeded KPI of 95% w/i 7 days (100%). Turn-around rate for legal@ exceeded KPI of 95% w/i 7 days (100%). Litigation win: Evelyn Schels v. WMF in Germany. Core legal advice and daily operations to the satisfaction of ED and C-levels per KPI.

113

Research and Data edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 4.5 FTE, 2 research fellows, 10 collaborators

Time spent: strengthen 40%, focus 20%, experiment 40%

114

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Research and Data

Objective: Data ownership map

Objective	Measure of success	Status
FOCUS Streamline ownership of data and research Team members involved: 1	Publish a map for internal use of who is responsible for specific requests around research and analytics. (T112317)	completed

Worked with individual teams on mentoring / backfilling / hiring of data analysts
Published research/data ownership draft
Socialized division of labor with Audience and Technology teams, Legal, Comms, C-team

https://office.wikimedia.org/wiki/Research_fingerpost

115

Q2 - Research and Data

Objective: Revscoring integration

Objective	Measure of success	Status
STRENGTHEN Revscoring integration Team members involved: 1 Collaborators: 9	Bring revscoring to fruition to our users as a Beta Feature (score integration into RC feed) (T112856)	missed Reason blocked on code review for months volunteer time went away during FR

we weren’t able to complete the design and deployment of this beta feature, due to limited resourcing and dependency on volunteers
revscores integration into edit histories / RecentChanges feed pushed to Q3
focus shifted on other priorities (massive deployments, impact analysis) see other accomplishments below

https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service

116

Q2 - Research and Data

Objective: Recsys productization

Objective	Measure of success	Status
STRENGTHEN Productize article and link recommender systems Team members involved: 2 Collaborators: 3	Article creation (T112321) and link (T112322) recommendations productized as services so they can be integrated with products and community tools.	completed

APIs available on Labs
architecture and service implementation blessed by Ops
First 3rd party tools adopting the service (example)

https://meta.wikimedia.org/wiki/Research:Increasing_article_coverage

117

Q2 - Research and Data

Objective: Value added research

Objective	Measure of success	Status
EXPERIMENT Value added research Team members involved: 1	Deliver research on historical data to quantify who adds value to English Wikipedia	missed Reason completed part of the project (historical measurement of productivity on English WP)

Finished historical analysis of productive edits by anons vs registered users(presented at January 2016 Research Showcase)
Partnership with external Hadoop as a service provider (Altiscale) ended in Q3 due to limited funding (currently exploring potential pro bono extension of services)

https://meta.wikimedia.org/wiki/Research:Measuring_value-added

118

Q2 - Research and Data

Objective: Reader segmentation

Objective	Measure of success	Status
EXPERIMENT Reader segmentation research STRETCH Team members involved: 3 Collaborators: 1	Deliver research on reader segments and behavioral patterns (T112326)	completed

completed 3 contextual surveys and identified robust categorization scheme for reader motivation and information needs
presented results to Reading team in preparation for Part 2 (Q3)

https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour

119

Design Research edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 4 FTE

(+ 1TPGer, and a half time researcher who is on the Reading Team)

Time spent: strengthen 60%, focus 30%, experiment 10%

120

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Design Research

Objective: Design Research Workshop

Objective	Measure of success	Status
STRENGTHEN Design Research workshop to build collaboration capacity with product teams. Team members involved: 4 (plus 3 non team members and IT / Admin support)	Workshop happens, and product teams learn more about collaboration with Design Research and vice versa.	Workshop completed: 40 participants attended (details on next page)

121

Q2 - Design Research

Objective: Mentoring

Objective	Measure of success	Status
STRENGTHENMentor people to do quality design research (those not on DR team)	One person outside the design research team does a complete design research project to DR team standards, expanding the number of people who do design research well.	Ongoing: Conceived of the prototype lab end of December. We initiated the first sessions on January 15. They will be ongoing every other week. Sherah has been participating in Design Research activities including being the lead in creating a methods menu for DR Workshop.

Prototype labs is bi-weekly design research series. It is being implemented to enable consistent evaluative research in a regular cadence for the Reading team to iterate concepts and get to know users.

Prototype Lab, besides being a great opportunity for mentoring, it is an attempt to address coordination difficulties with the multiple teams of the Reading team and provide a regular cadence of research and stronger understanding of readers.

122

Q2 - Design Research

Objective: Improve Participant Recruiting

Objective	Measure of success	Status
STRENGTHEN Improve participant recruitment process	Implement changes to reduce steps in recruitment (of design research participants) process to make recruiting more efficient.	Completed: (see next page for details)

Provide recruiting and scheduling for designers doing design research within teams and for the Prototype Lab.

Establish relationships with external recruiting firms to schedule international participants for deep dives.

Participant recruiter taking notes for design research sessions as needed.

123

Q2 - Design Research

Objective: Design Research with Product Teams

Objective	Measure of success	Status
FOCUS Collaborate with product teams to implement design research projects.	Design Research completes design research projects in collaboration with product teams to inform human centered product development.	11 research projects completed

Learning:

We potentially have too few QA professionals for the need within product teams. Design and Design Research end up doing QA before usability testing and before release.

124

Q2 - Design Research

Objective: Personas

Objective	Measure of success	Status
EXPERIMENT Persona interviews	12 persona interviews completed	5 interviews completed We noticed some patterns in Readers learning online, and what paths they take to learn.

We are currently solving for the issue of participants who agree to participate in research and then don’t show up as scheduled.

We deferred Persona research to get the Prototyping Lab rolling and because there were many unexpected and ad hoc research requests during this period.

125

Analytics edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 6 FTE, 1 PT

Key performance indicator: Velocity

Quarter: 911 (~906 last quarter)	October: 426 (>10% MoM)	November: 227 (<53% MoM))	December: 258 (>13% MoM)

126

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Analytics

Objective: Launch Pageview API

Objective	Measure of success	Status
Launch Pageview API Alpha API is operationalized and ready to be queried by outside world A blog post is published on Wikimedia’s blog Team members involved: 3	Usage of new API (next slide) Community feedback (not so SMART but very relevant)	The API was successfully launched Python, JavaScript and R clients written by community members Blog post published here

Learning: It is great to work in projects that have very positive community feedback.

Do capacity planning much before launch date, we had to drop hourly resolution as the system did not have capacity for it.

127

Q2 - Analytics

Objective: Replace wikistats pageview reports

Objective	Measure of success	Status
Use Hadoop infrastructure to replace computation of wikistats on http://stats.wikimedia.org (wikistats) Team members involved: All, at some point	Replace 5 of wikistats most popular pageview reports defined here	Reports updated: Monthly pageview reports for all wikis and projects, normalized/raw, mobile/desktop/combined. Available here

We consolidated pageview definitions: we have only 1 pageview definition for pageview data since May 2015

Pageview dumps have also been updated to match the new definition (daily/monthly per-article and per-project pageviews)

Learning: New definition has less variability (good) and is better at detecting internal traffic and robots

128

Q2 - Analytics

Objective: Program Global Metrics using Wikimetrics

Objective	Measure of success	Status
Deploy to wikimetrics an easier way to collect program global metrics for Learning and Evaluation of our grants programs Team members involved: 2	800 human hours saved by automation Feature adoption	Missed Code complete but not deployed. Reason: Wikimetrics is deployed on labs and we need to refactor our puppet code to do away with the self hosted puppet master setup, in order to reliably deploy and keep the service running

Currently deployed at: https://metrics.wmflabs.org/reports/program-metrics/create/

129

Q2 - Analytics

Objective: Build the infrastructure to support a generic EventBus

Objective	Measure of success	Status
Build infrastructure Scalable Event Bus Team members involved:2	EventBus service up and running in production using shared schema repository and new Kafka cluster.	Done. Documentation: https://wikitech.wikimedia.org/wiki/EventBus

A stretch goal for Analytics was producing real events to this new service. This work was done by the services team, and is ready to deploy. However, the holiday deploy freeze kept the new MediaWiki code that produces to the service from being deployed before the end of the quarter.

Learning:

Anticipate the deploy freeze that happens at the end of every calendar year earlier. Allow more room when teams depend on your work.

130

Performance edit

Quarterly reviewQ2 - 2015/16

Team size during this quarter: 5 FTE

Key performance indicator

Time to first paint	819ms	% from Q4	~52.37 YoY

131

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Performance

Objective: Availability

Objective	Measure of success	Status
Availability	Make MediaWiki code ready for concurrent deployment across multiple data centers.	Done with all high-level changes we anticipated for MediaWiki, but unknown add’l work remains.

132

Q2 - Performance

Objective: Thumbnail do-over

Objective	Measure of success	Status
Solve all (5-6) thumbnailing problems on a VM.	Move image manipulation (thumbnail rendering) out of MediaWiki by designing and implementing a standalone, horizontally-scalable imaging service based on Thumbor.	Done. Vetted by TechOps; on the roadmap for initial deployment in Q3.

(Some) problems with current stack:

Thumbnails of all sizes are stored forever, until the original gets deleted. Unused and rarely used thumbnails are never cleaned up.
Lots and lots of purge messages may be needed to clean up the thumbnails for a given media delete, to the point that the end-user request may timeout.

What we want: far future-expires for images; webp support; operational simplicity; support for additional image transformations (rotations, cropping, etc.)

133

Architecture edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 0.5 FTE

Time spent: strengthen 50%, focus 50%, experiment 0%

Key performance indicator

? - we may choose RFC flow, but no metrics gathered yet

134

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Architecture

Objective: Prepare WikiDev '16

Objective	Measure of success	Status
Focus: Prepare WikiDev '16	Demonstrate that we can develop a modern system in an inclusive, consensus-oriented, open manner. Large number of diverse RfCs, with ample discussion prior to WikiDev '16	Preparation happened, summit resulted :-)

Learning: Sharpened collective thinking on good meetings and how to build consensus

135

Q2 - Architecture

Objective: Strengthen: Improve ArchCom utility

Objective	Measure of success	Status
Strengthen: Improve ArchCom utility	Set up Architecture Committee to succeed. Maintain one public IRC meeting per week	10 IRC RFC review meetings, plus 3 “Agenda bashing” meetings for WikiDev ‘16. Most were at the usual time (14:00 PT) Phab query for meetings

Learning: Quote: ”[With] archcom being effective now (Which is great btw, so don't get me wrong) - I have noticed a trend where decisions are made in irc meetings, with little to no public discussion on the mailing list”. -- Brian Wolff on T118932, November 30

136

Q2 - Architecture

Objective: Strengthen: Improve ArchCom utility

Objective	Measure of success	Status
Experiment: ArchCom naming	Think through our loaded naming scheme, possibly renaming bits (e.g. “Architecture Committee” and “RfC”)	I didn’t get around to proposing anything on this front.

Learning: it’s ok that this one fell off. “ArchCom” and “RFC” seem to work well enough for now.

137

Operations edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 17 FTE

Time spent: strengthen 40%, focus 40%, experiment 20%

Key performance indicator

Availability	99.979%	+0.037% from Q1

138

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Technical Operations

Objective: Encrypted PII

Objective	Measure of success	Status
Focus: Encrypt PII on cross-data center links Team members involved: 5	Encrypt cross-data center traffic for MySQL/MariaDB, Kafka, Cassandra, (Varnish) HTTP Decommission udp2log (in favor of Kafka)	Cross data center traffic for MariaDB, Kafka, Cassandra and between Varnish cache tiers is now encrypted. Legacy (and unencrypted) logging solution udp2log has been decommissioned.

Learning: It has been confirmed by recent security vulnerability announcements (Juniper and Fortinet VPN backdoors) that we should not rely on transport link encryption by (proprietary) network hardware, as we have been hesitant to do since the start. We will continue to rely on FOSS implementations (TLS, IPsec) for encryption of PII data.

139

Q2 - Technical Operations

Objective: User auth security

Objective	Measure of success	Status
Strengthen: Security: Strengthen infrastructure user authentication Team members involved: 3	Establish org-wide workflow for creation and deactivation of user accounts in Production Implement (but don't enforce) two-factor authentication on SSH bastion hosts Migrate unmaintained OpenDJ LDAP to OpenLDAP	Organization-wide on/offboarding process has been implemented. Two-factor authentication is implemented and under experiment by all root users (using Yubikeys). OpenDJ has been replaced with OpenLDAP.

Learning: First results of using Yubikeys for ssh authentication are promising, however some usability and reliability issues remain. Further experimentation with the OTP setup will continue into the next quarter.

140

Q2 - Technical Operations

Objective: Incident monitoring

Objective	Measure of success	Status
Strengthen: Incident monitoring infrastructure Team members involved: 2	Upgrade incident monitoring and expand it to codfw Reduce monitoring check execution lag to 0 Create new abstractions for incident monitoring in Puppet	Software options were evaluated and we decided on a migration to Shinken (distributed monitoring). This migration has been prepared but didn’t complete in time. New monitoring abstractions have been created in puppet, work is still ongoing.

Learning: FQ2 is a challenging quarter every year with reduced time available due to the holiday periods, relatively high amount of staff taking unused vacation days, and deployment freezes. We should take this into account for our goal planning for this quarter.

141

Q2 - Technical Operations

Objective: Bare metal Labs

Objective	Measure of success	Status
Experiment: Support bare-metal servers in Labs Team members involved: 2	Allow provisioning of physical servers within the Labs network, governed by Labs Puppetmaster and LDAP user accounts	Labs infrastructure and Puppet manifests now support bare metal servers in the Labs context. One bare metal server has been provisioned.

Learning: A separate small new test cluster was built for testing infrastructure changes to Labs itself, which proved very helpful for this goal and many projects in the future. OpenStack Ironic was also evaluated for this goal, but our current reliance on Nova-Network would imply a significant effort and create more technical debt until our migration to OpenStack Neutron completes.

142

Release Engineering edit

Quarterly review

Q2 - 2015/16

Approximate team size during this quarter: … 6

Time spent: strengthen 9%, focus 28%, experiment 0%, Maintenance/Other 63%

Key performance indicator

Time to merge in MW Core	~6 mins	-45% from Q4	(still analyzing/making sense)

143

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Release Engineering

Objective: Reduce CI wait time

Objective	Measure of success	Status
STRENGTHEN Reduce CI wait time	CI cluster responds to spike in queued builds by starting and registering additional jenkins slaves (+) Migrate majority of CI jobs to Nodepool (-)	Nodepool is up and working. However, the majority of CI jobs are not yet migrated (waiting on a solution for caching dependencies).

Learning: More collaboration is needed to address some of the more annoying problems (see: caching solution for dependencies).

144

Q2 - Release Engineering

Objective: Reduce number of deploy tools

Objective	Measure of success	Status
FOCUS Reduce number of deploy tools from 3 to 2	Migrate all Service team owned services and MW deploys to scap3	Not completed. We are behind schedule in migration of services. The Dev Summit discussion, however, was very fruitful. Migrated some services on Beta Cluster to scap3 pre Holidays.

Learning: After the slow beginnings to this project (slow due to other priorities taking time) we didn’t allocate enough time in Q2 for “the last 10%” of functionality needed by Services.

145

Q2 - Release Engineering

Objective: Retire Gerrit and Gitblit

Objective	Measure of success	Status
FOCUS Retire Gerrit and Gitblit in favor of Phabricator	Decommission Gitblit (-) Code review RFC: creation, publication, discussion, feedback etc (+)	Not completed. Gitblit is still live. Differential Migration RFC is published, has been discussed, and feedback being addressed (mostly from the Dev Summit).

Learning: Happy surprise- The discussion at the Dev Summit about the migration was predominantly positive and there were no new blockers identified. It was a very encouraging conversation.

146

Q2 - Release Engineering

Objective: Release MediaWiki 1.26

Objective	Measure of success	Status
FOCUS Release MediaWiki 1.26	A quality MW 1.26 successfully released	Released.

Learning: Less of a learning and more of a perennial known: The releases we make for third-parties are still sub-par due to a number of reasons, one being internal priority setting (it is far less than 1 FTE per year).

147

Services edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 4 FTE

Time spent: strengthen 40%, focus 40%, experiment 20%

Key performance indicators

REST API usage	248 req/s mean	15% up from Q4	∞% YoY
REST API HTML uptime (catchpoint)	99.97%	99.82%	∞% YoY

148

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Services

Objective: API build-out

Objective	Measure of success	Status
Strengthen: Build out the REST API	Provide cached & purged REST API entry points for at least two high-traffic API use cases, and work with users to speed up page loads.	New entry points: page summary, wiktionary definition, math SVG & MathML renders. Math used in production web; apps and hovercards (web) in process of switching over to summary & definition end points.

Learning:

Need a well-defined thumbnail API. Started discussion & proposed strawman.

Broad support for maximizing caching with more REST end points at dev summit.

149

Q2 - Services

Objective: Event bus & change propagation

Objective	Measure of success	Status
Focus: Set up an event bus and change propagation solution	Key change events like edits, renames, deletes available in a publish-subscribe event bus. Basic change propagation / pre-generation functionality for REST services operational.	EventBus operational, MediaWiki event production just deployed. Change propagation service deployment delayed by dependency on event bus.

Learning:

EventBus collaboration between Services & Analytics initially suffered from an unclear division of responsibilities. Clarifying responsibilities and dependencies helped to make progress.

150

Q2 - Services

Objective: API-driven web front-end

Objective	Measure of success	Status
Experiment: Prototype an API-driven web front-end	Gather information to inform our longer-term front-end architecture by prototyping a service-worker / node.js based front-end.	Prototyped a service-worker front-end and established performance characteristics. Worked with Reading team & held a productive session on the topic at the dev summit.

Learning:

Session at the dev summit showed broad support for the direction & a preference for progressive enhancement.

151

Security edit

Quarterly reviewQ2 - 2015/16

Approximate team size during this quarter: 2.1 FTE (+Moritz)

Time spent (strategic): strengthen 25%, focus 0%, experiment 75%

Key performance indicator

Critical and High Priority Security Bugs	REDACTED	REDACTED from Q4	n/a

152

All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Security

Objective: Static Security Testing

Objective	Measure of success	Status
Automated security static analysis of MediaWiki	Automatically scan core and one extension weekly.	Security team chose Veracode for scanning, and are submitting core and two extensions for each deployment branch.

Learning:

The quality of static security testing tools for PHP is still immature. The WMF could significantly contribute in this area, but the Security Team lacks the resources to pursue this.

153

Q2 - Security

Objective: Expand Training

Objective	Measure of success	Status
Expand developer training	Present secure SDL training for community and staff. (stretch) Develop and present security training materials for DevOps and Mobile	The team documented the optimal process for other teams to integrate security into their development process, but training has not been given.

Learning:

The team adapted to account for a security incident. Our goals should be more flexible to handle this in the future.

154