Global Development/Catalyst Program Strategy

I have documented below the general strategy for the Catalyst Programs that we are developing in the Global Development team. This is a draft and subject to further refinements, but captures the general approach which has guided our work to date. --Barry Newstead (WMF) (talk) 19:04, 30 May 2012 (UTC)[reply]

Background on the Catalyst Strategy

Wikimedia penetration world map

The Wikimedia strategic planning process conducted between July 2009 and June 2010 placed a major focus on the improvement of Wikimedia's projects on a global basis. A major focus was placed on growing projects to better meet the needs of the large majority of people around the globe who were not in a position to contribute to or benefit from Wikimedia's free knowledge projects. The map of Wikimedia's reach provided a clear picture of the inequitable reach of our projects.

While the strategic plan set out important goals, it didn't identify the specific programs that would deliver the impact we are seeking as a movement. This is because there is no precedent in the movement for this sort of growth. The strategy was to create small teams that would help the movement figure out what works and then help to integrate these approaches into the way that we work in the movement, in a manner that is aligned with our volunteer-led community concept.

Impact goals

Catalyst Project Approach

The catalyst programs aim to:

grow editorship in their target countries and corresponding language projects
learn how to grow editors further in target countries and beyond
increase readership of Wikipedia in these countries (including via mobile)
diversify the worldwide community of editors, now mainly in Europe and North America

More specifically, the catalyst programs aim to:

support the acceleration of growth of small-medium projects in Indic languages and Arabic and demonstrate approaches for other similarly situated language projects
support the Portuguese community to restart community growth in order to strengthen the quality and breadth of the Portuguese language projects with a focus on wider participation from Brazilians
support better representation of domestic Indian topics in English Wikipedia and demonstrate approaches to further opening English Wikipedia to diverse contributors that better reflect the global distribution of the English language

Program goals

The catalyst programs seek to generate real impact for the movement as laid out above. A prerequisite to the achievement of these goals is the identification of programs that actually work. There are no proven programs within the movement that have been successful in achieving the goals above...none. The emergence approach that has been so powerful in building our successful projects has yet to prove successful in achieving wider success (in terms of creating truly great encyclopedias and other knowledge projects) in the languages and geographies above. There has been extensive effort in outreach and Wikipedia Academies, but neither have been evaluated to date on their impact on the projects. There are very promising initiatives happening in the movement:

The Wikipedia Education Program created out of the Public Policy Initiative has generated promising results in the United States and is in pilot stage in other more than ten geographies, where we have mostly seen good results and faced new challenges that we need to tackle
GLAM work has gained substantial momentum in many countries, but we have not to date evaluated the impact on the projects, so we don't know what the potential for impact is
Photo walks and competitions (highlighted by Wiki Loves Monuments) have generated enthusiasm in many countries and led to real high quality media contributions to the projects, however we have not evaluated these programs and do not have a clear view of the potential to link these initiatives to deeper contribution over time including the growth of new editors and articles on Wikipedia

The role of the catalyst programs is to serve as learning laboratories with the program goal of creating proven program models that can be utilized effectively across the global movement. The projects will follow the scientific and entrepreneurial methods that we see in places like Silicon Valley. The projects will be rigorous endeavors that look for innovations that will make a difference (from any source), follow the data, stop doing things that aren't working and most importantly, they will learn and iterate. It should be expected that there will be more failure than success, but that in failure there will be valuable learning that helps get us closer to success.

What success looks like

The easy answer (and the one we hope for) is that the catalyst projects will directly contribute to the impact goals laid out above.

The more complex, but important answer, is in the way that the projects contribute to the impact goals. If the projects can demonstrate what works and what doesn't in a rigorous manner, then the global movement will truly benefit from the work. For example, if they can demonstrate what effective outreach looks like and what the essential elements are, then the whole movement can benefit by using these proven approaches. Conversely, if they demonstrate that outreach is ineffective in generating contribution to the projects, then the whole movement can benefit by spending less time and money on outreach (at least, under the banner of editor growth goals). When we look back on the catalyst projects, there success will need to be measured by their contribution to the knowledge base.

General design of catalysts

We are approaching the Catalyst Programs with five elements:

1. A variety of efforts and experiments in building the number of editors. The model is to conduct a series of small pilot projects and test their efficacy and efficiency before expanding them within a geography and across geographies. These pilot projects should be done with the existing community (where there is interest) and should be implemented transparently. For each project there needs to be:

a clear set of “SMART” goals (specific, measurable, achievable, relevant [growing the number of active editors], time-bound) and learning questions to answer in the pilot
clear design for the pilot with defined methods and articulated assumptions, so that success or failure can illuminate future projects, and inform which elements work and which do not

an open process to make decisions, which reflects:

consultation with the community
how the project fits into the Wikimedia plan for that geography
securing the resources required for the project to succeed
community support for the initiative (not a vote or veto, but people who want to be involved and help)

a named team, including:

leader or person responsible for securing agreement, managing the project to success, and delivering all documentation after the end of the project
key decision-maker who approved the project, ultimately accountable for its success
people playing key roles, such as field work, database management, analysis, etc.
mentors who can advise the project, perhaps bring their experience of prior projects

a schedule, with:

a defined process to secure any necessary agreements and resources
a specific start date, once all agreements and approvals are secured
an end date, at which an assessment of success or failure can be expected (even if, at that date, the assessment is to extend the project)
milestones between start and end dates to track progress, capture learning, and make decision to suspend pilots that are clearly failing
time after the end of the project to document the experience to inform future decisions
open documentation of progress vs. SMART goals and learning questions

2. Careful tracking of individuals progress through user to active, collaborative editors, including:

encouraging users to create accounts
linking names and email addresses to user accounts
tracking individual engagement with catalyst projects through user account identification
timely, short surveys of participants at targeted points in the process

3. Rigorous analysis of:

Individual behavior through to active, collaborative editor, including what (and what doesn’t):

attracts individuals to become an editor: e.g. it may be easier to attract editors with a “ground force” giving talks in universities or at the Calcutta book fair, ads on Google, etc.
retains each editor in the community: e.g. are new editors most encouraged by connections on Facebook, mentors, push emails, notes of thanks, lack of warnings, etc.
builds affiliation with both with the existing editor community and the broader Wikimedia community, including local chapters and The Wikimedia Foundation

Community behavior to spot impact trends and understand the causes, such as peer and network effects; the analysis can build on the understanding of how existing communities of active editors grew, for example within existing networks of “geeks” and “gamers”
The cost of each effort, to compare the efficiency of different approaches; costs to be considered include out-of-pocket expenses, the time of paid staff or consultants, time of volunteer community members, and the strain on the existing Wikipedia editor community
The time taken for each effort to deliver results

4. Instilling the learning into each new round of pilots to grow the community of editors so that:

the lessons of success and failure can be cross-pollinated and instilled into all new pilots
resources can be focused onto efforts that can scale effectively and efficiently
failures do not harm or alienate the community (and result in misapplied resources)

5. Open documentation of the whole process to enable contributions and input from the broad community, and support timely cross-pollination of lessons. This will include:

a page on meta and on the relevant projects for the catalyst program which lists all projects, links to the page for each project, and links to pages covering findings
a page on meta and on the relevant projects for each project which includes all of the information in 1 above
a page on meta and on the relevant projects summarizing the findings from 2 and 3 above to enable 4
regular updates on relevant mailing lists and village pumps along with blog posts to keep the community informed of progress and findings

Sample activity areas for piloting

Wikipedia Education Program
Student clubs and on-campus programs
Outreach to new communities
On-wiki project supports for small communities
On-wiki project supports for newcomers
Addressing barriers to community growth in projects
On-wiki contribution campaigns/competitions
Off-wiki contribution campaigns via media/social media
Social media supports for community and newcomers
GLAM/other institutional partnerships focused on contribution

Organizational setup

The catalyst projects are setup as temporary teams of consultants or staff (where this makes sense) in three geographies: India, Brazil and the Arabic language region. The specific design of each project will vary based on the local context. Where practical, it might make sense to partner with third party organizations to support the projects.

The teams will typically comprise a group of 4-5 consultants with specific programmatic functions and coordinated by a National Program Coordinator or Director. The actual size of the teams might vary based on the scope of work that is defined. If there is the capacity to do more pilots more quickly then it might make sense to add consultants on short contracts or for longer term commitments.

Setup in India

The India Program team comprises five consultants as of 15 April 2012. They each are have an independent consultancy agreement with the Wikimedia Foundation. Due to operational complexities and uncertainties in India, the Wikimedia Foundation continues to alternative options for the setup of the India Program including the potential of partnership with a third party organization.

See India strategic plan for further information.

Setup in Brazil

The Brazil Program team comprises two consultants as of 30 May 2012. Oona Castro and Everton have independent consultancy agreements with the Wikimedia Foundation. The current plan is to form a representative office of the Wikimedia Foundation in Brazil. This office would house a small staff and is very likely to work from a co-working space in São Paulo.

The staff will cover the following areas/functions:

Revitalizing the Portuguese Wikipedia editing community
On-wiki attraction and cultivation of new editors
Outreach and localized community support
Education Programs
Institutional Partnerships
Communications

See Brazil Program page for further information. Brazil strategic plan will be developed soon.

Setup in Arabic Language Region

The Arabic Language Initiative comprises three consultants as of 15 April 2012. One consultant provides general advisory support and regional outreach support. Two consultants provide program support to the Wikipedia Education Cairo Pilot and university outreach in Cairo. The current plan is to engage additional consultants to coordinate and support programs in different parts of the region, and to pursue partnerships with non-profit organizations with aligned missions and programmatic capabilities that would support the catalyst strategy.

See Arabic Language Initiative strategic plan (draft) for further information.

Timeline and milestones

The catalyst programs are part of the Wikimedia Foundation's five year strategy. We are entering the third year of the strategy and are reaching a point where we have teams in place (not quite, but close enough), where administrative issues are no longer a major distraction, and we have learned valuable lessons from our first year of activities in India. Each program team has (or will soon develop) a clear plan with milestones.

Decision points

We expect the catalyst projects to be assessed based on their work over the next three fiscal years. The key decision point will be 2014/15 before the start of the new strategy cycle. The catalysts need time to build their capacity in running pilots, work with the communities on projects in the design process, study the context in the focus projects and places, create the tools for monitoring programs, etc. Making a decision on the viability of a catalyst project earlier than this point will be premature.

The near term decision points ahead will focus on the choices about which pilots to launch, expand, and stop. These decisions will be made by the Chief Global Development Officer on the advice of the National Program Coordinator and Program Directors as appropriate.

Sustainability/Exit strategy

The long term role of catalyst programs depends on whether this model of work is effective at both achieving the impact and program goals. This is still an open question.

It is our hypothesis today that there is value in having teams that run high impact programs in specific countries. In addition, it is our hypothesis that there is a long term role for catalyst projects to focus on piloting and evaluating new programs rigorously. The sustainability/exit strategy differs for each hypothesis.

Sustaining programs in catalyst regions

Our ideal approach is that effective programs that achieve impact goals will migrate out of the catalyst program structure as those programs mature in the specific region. There are at least three ways for programs to migrate:

Local chapters take on governance and management of programs
Community groups take on governance and management of programs that don't require long term staffing
Community groups partner with third party NGOs or create new organizations take on the leadership and management that require long term staffing

The decision on the best approach depends on various factors that will need to be evaluated over time based on the characteristics of the program and the status of partner organizations in the geographies.

Sustaining catalyst programs

If the catalyst programs prove effective in achieving their goals and there are continued important program pilots to design, execute and evaluate in the individual geographies, then these programs should be sustained through continued funding from the Wikimedia Foundation (via the Funds Dissemination Committee process) and through grants from third parties.

The catalyst programs will be assessed individually in the 2014/15 fiscal year (Note: India Program will have an assessment in 2013/14 as it is one year ahead of the others) and then every two fiscal years to enable sufficient time to design, execute and evaluate a set of pilots. It may be determined that there is either insufficient work to do in one or more geography and then that catalyst program would be closed down. It may also be determined that a catalyst program should be launched in a new geography based on a strategic opportunity for impact and for learning.