Wikimedia Meta-Wiki

Research:Growth and diversity of Technology team audiences

Tracked in Phabricator:
Task T171220
Created
21:40, 6 November 2017 (UTC)
Duration:  2017-November – 2018-March
This page documents a completed research project.


Program 4 of the Technology Department's 2017-18 Annual Plan involves improving the support that the Wikimedia Foundation provides to technical communities inside and outside the Wikimedia Movement who build upon and use our software products and platforms. These users include "code contributors, documentation contributors, bug reporters, API consumers, volunteers who build innovative solutions to on-wiki workflow issues, researchers who examine the data generated by the Wikimedia projects, value-added vendors who provide services and support based on Wikimedia free and open-source software products, and true 'third parties' who install and use FLOSS software produced by the Wikimedia movement on their own computers for various reasons[1]."

Overview

edit

Summary of findings

edit

Key observations

edit
Awareness of audiences
Tech teams are aware of their external audiences, and work hard to support them.
Audience overlap
There is a lot of audience overlap between teams.
Ad-hoc communication
Communication with external audiences is ad hoc.
Importance of researchers
Researchers are recognized as an important, and diverse, external audience.
Value of offline interactions
Offline interactions--e.g. participation in hackathons, meetups, and professional and academic conferences--are seen as effective ways of learning the needs of existing audiences and introducing Wikimedia Technology platforms to new audiences.

Challenges

edit
Shared platform ownership
Shared ownership of technology platforms across teams can create ambiguity around who is responsible for external audience support.
Limited bandwidth
Teams have limited bandwidth for performing outreach and maintaining end-user documentation.
Diverse communication channels
The diversity of communication channels used to interact with audiences can hinder outgoing and incoming communication.
Limited data
Limited data about platform users and usage can make it difficult to track adoption by audiences.

Recommendations

edit

The following list of recommendations is not intended to be exhaustive. It is focused on addressing the kinds of issues over which the Wikimedia Technology teams have some degree of direct control, and which can plausibly be addressed with existing resources (or near-term resource growth projections).

Prioritization and goal-setting
Develop a Technology-wide strategy for prioritizing audiences and formalizing growth goals on a per-audience basis.
Documentation and communication support
Provide teams with increased support for end-user documentation and collecting input and feedback from audience members on an ongoing basis.
Define responsibilities and processes
Clarify audience support responsibilities and processes, within and across teams.
Increase offline communication
Increase tech team participation in conferences, workshops, and events attended by current and potential audiences.

Definitions

edit
  • Technology product or platform: technological infrastructures or outputs of technology teams that are intended to be used directly by external audiences.
  • External audience: Individuals or organizations that use particular products or platforms developed and/or maintained by Wikimedia Technology. The current research focuses solely on external audiences--those who are outside the Wikimedia Foundation.
  • Usage: In this research, usage means more than simple consumption. Usage involves building something with or on top of a Wikimedia Technology product or platform. By this definition, Wikipedia readers are not included in the class of users we are focused on, nor are Wikipedia editors whose only interaction with Wikimedia Technology products is through directly editing the content of Wikipedia websites.
  • New/potential audience: External individuals or organizations that are not currently using Wikimedia Technology platforms, but who may be interested in adopting our platforms in the future
  • Adoption: the use of Wikimedia Technology platforms by new audiences for specific purposes, or the increased use of our platforms by existing audiences.

Problem statement

edit

Outcome 2 of Technology's 2018-19 Departmental Program #4 involves measuring the adoption of Wikimedia technology by external audiences. The Technology department currently lacks a set of broadly-accepted, reliable, and meaningful measures for tracking the adoption of our technology by members of these audiences, which limits our ability to:

  1. measure the current and historical usage of our technology platforms outside of WMF,
  2. identify opportunities to reach new audiences and/or serve our existing audiences better, and
  3. measure adoption of Technology platforms (on a per-team or per-product basis) over time, including evaluating the impact of any future interventions we make to grow or more fully support our external audiences.

Research questions

edit

This project is the first step towards developing these kinds of metrics. For each of the teams we work with on this project, we seek to answer the following questions:

  1. Which audiences are you currently serving? Who else do you believe you should be serving?
  2. How does your work support the audiences you currently serve? How could you serve these audiences better?
  3. How are you currently measuring adoption from these external consumers? If you don't, do you have any notions of size/adoption rate?
  4. What kinds of audiences are you serving (e.g. individuals vs institutions/organizations)? What is the geographical distribution of these audiences?

Methods

edit

This study will consist of interviews with team leads in Tech involved in building platforms with an externally-facing audience and get them to articulate who their core audience is, and how to characterize adoption of their platforms.

At least initially, this research will not involve interviews with members of all Tech department teams. Rather, we will focus on teams that have well-defined external audiences, beyond other WMF teams/departments, and in some cases beyond the Wikimedia Movement altogether. We are conceiving external audiences for Tech teams as researchers, developers, data consumers who build other platforms or products on top of the software products (e.g. MediaWiki), research/analytics resources (including publications, specifications, dashboards, data APIs and datasets), and platforms (e.g. ORES, Toolforge, WikiData Query Service) we develop and maintain.

The anticipated output of this research will be an audience map for the audiences identified in the interviews, along with how the team that identified this audience currently communicates with and/or tracks tech adoption from this audience. Common communication channels and adoption metrics used or proposed by multiple teams will be highlighted for further investigation and evaluation as possible KPIs.

Timeline

edit
  1. October 2017: Develop interview protocol
  2. November 2017 - January 2018: conduct interviews with team leads
  3. January - March 2018: Analyze and report results

Teams involved

edit

This list includes teams for which we have already identified products or platforms that have external audiences. This list may change during the course of our study, as we refine our definitions of audiences, products, and adoption.

  1. Analytics
  2. MediaWiki Platform
  3. Research
  4. Scoring Platform
  5. Search Platform
  6. Services
  7. Cloud Services

Results

edit

See Research:Growth and diversity of Technology team audiences/Report for the full report of findings.

Stakeholder matrix

edit
Audience types
Team Platforms discussed Communication channels Researchers Platform developers Platform consumers Platform partners
Research Research papers, curated datasets, code libraries, WM data dumps, APIs, Analytics dashboards public mailing lists, Research newsletter, research.wikimedia.org, personal correspondence, metawiki, @Wikiresearch, blog posts, conferences and lectures. academic researchers, data storytellers, NGOs Code library contributors Media entities, research educators External collaborators, Grantmaking organizations
Scoring Platform ORES, JADE, WikiLabels Wiki-ai mailing list, mediawiki, village pumps, GitHub issues, IRC, Phabricator, conferences, lectures, and hackathons. academic research analysts, model builders direct contributors to ORES platform, developers of client applications ('local technologists'), WikiLabels campaign organizers Orgs like Google and Kiwix who want to know which versions of an article to index ,
Cloud Services ToolForge, Toolsadmin, CloudVPS public mailing lists, IRC, wikitech, Phabricator. academic researchers, Wikimedian researchers, the 'open knowledge-curious' Bot and tool maintainers, documenters on wikitech
MediaWiki Platform MediaWiki MW Stakeholders Group meetings and mailing list, mediawiki, public MW mailing lists, conferences. Researchers, research educators, data storytellers, NGOs Volunteer extension developers, Mediawiki consultants Semantic MediaWiki project, Mediawiki Stakeholders Group, WikiApiary project
Analytics WikiStats2, RESTbase data API endpoints, datasets, EventStream, Browser Stats dashboard, Reportcard dashboard, WikiMetrics Phabricator, wikitech, IRC, public mailing lists, personal correspondence. academic researchers Wikimedia movement volunteers, Wikimedia chapters & user groups, Orgs that use Wikimedia data (Google, Facebook, Microsoft
Security Security reviews, vulnerability reports Phabricator, IRC, conferences, hackathons, personal correspondence. Developers in the community, independent developers, GSOC participants, MW consultants. WMDE
Services RESTbase, REST API mailing lists, Phabricator, IRC, mediawiki. Researchers who use the data API endpoints RESTbase service developers External MediaWiki users who want "the full Wikipedia Experience", Orgs like Google that use the REST API to crawl Wp
Search MW Search API, WikiData Query Service, ElasticSearch mailing lists, mediawiki, Phabricator, IRC, conferences Wikimedian researchers ("editors who are trying to pull more information together about certain domains") Open Source Connections (contracted to improve search models) External MediaWiki users, organizational Search API users (e.g. Google), Orgs with SPARQL-compatible databases (e.g. museums) WMDE, BlazeGraph


See also

edit

References

edit
  1. ↑ "Wikimedia Foundation Annual Plan/2017-2018/Final/Programs/Technology - Meta". meta.wikimedia.org. Retrieved 2018-03-23. 
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Research:Growth_and_diversity_of_Technology_team_audiences&oldid=24695647"