Brazil Program/Reports/2012-2013

The purpose of the Catalyst Program was to increase access, editorship and content, as well as improve the latest one in the Wikimedia projects. So far, the Wikimedia movement has been pretty focused on improving our main and most accessed project, Wikipedia, which is also a window for people to get to know all other Wikimedia projects.

We had established relatively ambitious targets for the first year. We had been advised that could be hard to reach and to predict, considering some targets are beyond the control of our direct action. We unfortunately didn't reach them in all their extension and some might say that sounded pretty obvious when they were first established. However, it still sounds helpful to have numbers to aim at, as reference to put our efforts in fulfilling our general goals.

What we did achieve was an increase in traffic to Wikipedia in Brazil, an emerging country that reads Wikipedia more and more. We also achieved to engage professors and educators into the mission of sharing knowledge in Wikimedia projects. We also managed to create the basis for next developments and partnerships in Brazil, through a local grant partnership with Ação Educativa, the organization in charge of the project and current recipient of the grant. We also engaged with the community bringing their knowledge and experience into our planning, and also encouraging partnerships and participation in events together with volunteers.

The purpose of this short report is to bring an overview on some of our achievements, failures and challenges, in order to create share with the large Wikimedian community worldwide what we did, how we've been developing the catalyst program and how we have been dealing with our greatest challenges.

As a former Wikimedia Foundation contractor, I would like to thank the Portuguese Wikipedia Community, and the Brazilian Wikimedia community, for all support, help, criticism, watching behavior and collaboration for the construction of our work. As a current grantee, I would like to thank the Wikimedia Foundation for the opportunity of building and continuing the development of this project, and also for the understanding and support to such a long path we took to build it thoroughly and slowly, constructing trustworthy relationships with our local partners and the community.

Oona Castro

Our resources back in the past and now

During the first semester of 2012, we had only one person working for the education program and the head of the catalyst program was hired in March, making a team of two people. During the first semester of 2012, we carried out a selection process and in the beginning of 2013 we had one more staff member and one short term contractor. By July 2013, we had 2 part time staff members and 2 full time staff members. After moving our program into a grant partnership with Ação Educativa in São Paulo, Brasil, one part time staff member was added into our team and we can count on the very welcoming support from their executive coordination as well as professionals from other areas such as administrative, legal, communications, publishing. We have received a grant of US$ 500,000 to develop our proposal, which was built throughout 2013 and reviewed by the Wikimedia community in September, 2013.

Challenges and opportunities

Back in 2012, the first three months were dedicated for on-boarding actions and a for planning the project with participation of volunteers. And the primary diagnoses led us to the conclusion that most attempts to bring new editors into the Portuguese Wikipedia faced a few challenges: a) one related to the understanding of the rules and policies, re-which we should facilitate and improve welcoming messages and tutorials; b) a small and overloaded community to help and improve new articles; instead, it's almost impossible to watch all changes and the fastest solution many times is to delete them in case they don't attend some requirements (and still, as we saw, 2013 has made 15 thousand articles with no sources); c) more knowledge on how to contribute is required by the new user than when Wikipedia was not one of the most accessed websites and this may discourage him/her.

It's quite normal that as the projects grow in quality and quantity of articles, the more is required by any user, even the new ones, with relation to knowledge on how to edit. While Wikipedia faces the challenge of managing newcomers and all types of contributions, given its huge exposure and recognition, its sister Wikimedia projects are extremely welcoming to new contributors, but face the challenge of not having a large enough active community to bring new people interested in collaborating. We want to face challenges in both ways in Brazil: to enlarge those small communities and raise awareness on all Wikimedia projects and to support the Wikipedia community (which is often also the entrance door for the movement) to build better conditions for the collaboration of the existing community as well as to bring in new editors and content into the project.

As the Brazilian population’s access to internet has increased considerably in the latest years, reaching now almost 50% of penetration, of which almost 70% have daily access to it , we see we have an opportunity for growth. We are willing to bringing more diversity to the Wikimedia global movement and more content in the Portuguese Language. The global south is still under-represented in the Wikimedia global community. Despite Brazil already represents about 4% of the global internet population, we are only responsible for about 2,9% of Wikimedia page views and 2.2% of page edits.

Background, challenges and achievements

The Brazil Catalyst Program throughout its first year and a half has established a team, a plan and built an organic relationship with the community, supporting the online Wikipedia Community in various ways, piloting an Education Program and sharing learnings out of it, building a strategy to improve decision making process through data analysis, supporting the improvement of communications channels, raising awareness on Wikimedia projects and developing a better understanding of the main challenges.

However, there is a lot to be done, and much to be improved. We needed to shift our approach and methodology for the implementation of the catalyst programs after the Narrowing Focus targets were established, and that meant postponing many activities. Hiring, planning and partnerships with organizations also took much longer than we imagined they would take (during 2012, we were only two in the team, establishing the basis to allow real kickoff of the program and piloting the education program. Now we have all set: an ongoing local grant partnership with Ação Educativa and a team of 5 people (being 3 part time and 2 full time).

In March, 2012, the amount of articles in the Portuguese Wikipedia was of about 715.000 articles. We have now more than 812.000 articles. In comparison to other wikipedias (as the Spanish or Polish ones), this growth may seem relatively low. However, we should highlight there was no relevant use of bots for creation of articles during this time, and on the contrary, there was human action to eliminate articles that had no sources for a while and were still published. Alongside with that, we should also highlight volunteers efforts to add sources into articles, through the reference project. Despite such efforts, 2013 still records the highest rates on articles with no sources. With only about 40 administrators and 1500 active editors, the Portuguese Wikipedia community lays on a conflict between enhancing quality and increasing content and editorship. The way we've been addressing this is to offer support for the improvement of filters, in partnership with volunteers efforts, but on the other hand to develop partnerships with people and institutions committed to sharing knowledge as well as to promote Wikimedia projects on social networks, media and educational and cultural institutions.

As stated above, one of our achievements was the increase on access to Wikipedia in Brazil. Although we are aware Alexa isn't the most reliable metrics for access, this is a regular rank made that allows us consistent comparison. According to this website, in 2011, Wikipedia was the 15th most popular website in Brazil. In 2013, it became the 10th, going up to 12th and 11th position throughout 2012 .

This is a good reason to celebrate, because this means more people are using Wikipedia to access knowledge. And it also reflects the increase of internet penetration in Brazil . However, still according to Alexa, Wikipedia is mostly accessed by graduate people, rather than those with no college or some college in comparison to general internet penetration. Besides that, the Portuguese Wikipedia is responsible for 2,02% of the general access to wikipedia.org, being on the 8th position in comparison to other languages. Yet according to Wikimedia stats, Brazil is responsible for 2,4% of global share in access to Wikipedia, sitting at the the 10th position (or 2,9% in the third quarter of 2013, sitting on the 8th position) . In our country, 82,8% of our readership is on the Portuguese Wikipedia (or 86,3%, in the third quarter of 2013) .

This means that we can celebrate the increase of access to Wikipedia in Brazil, and should continue promoting it as a source of knowledge in this huge and growing country. However, we also still face a lot of challenges: increasing the access to the Portuguese Wikipedia, participation and increasing and improving content.

A readership survey carried out in 2011 shows that readers in Brazil read more in other languages than the average of Wikipedia readers. This is not a bad thing per se, but we should be aware that if Brazilians are happy with the content they find in the Portuguese Wikipedia, they might look less for content in other languages. The second one is to increase participation.

While readership has comparatively showed a significant increase, editing hasn't. The Portuguese Wikipedia shows a “stable instability” - for years, we have seen no constant and consistent growth in participation in the Portuguese Wikipedia, nor any constant and consistent decrease.

In 2012, the curve of decrease in growth has changed the trend of 2011, showing room for growth. In 2013, during March, April and May, we finally saw growth in the numbers regarding new editors and active editors. In June, though, a very low decrease took place in the numbers regarding new editors (from 504 to 502) and a more significant decrease happened with regards to active editors (from 1598 to 1515). In October, the fall was very worrying. It's very hard to come up with definite conclusions on the reasons for these constant ups and downs.

Large media coverage in the first semester of 2012 has showed the power media plays to bring people interested in Wikipedia (at that time, we have received a lot of messages on how to contribute to Wikipedia), but retaining is still a challenge. The same readers survey mentioned above stated that better tutorials for first time editors could help Brazilians edit more. However, the main reason pointed out for little contribution is still something beyond our direct capacity to influence: that's people's “insecurity and lack of qualifications to be an editor”, as well as the fact that “they want to know their efforts will benefit their Wikipedia cohorts”. Another obstacle on that is that the “abundance of information on the site appears to overwhelm” readers. The survey also points out that Wikipedia is mostly used for personal reasons.

Considering this scenario along with the hard use Brazilians make of social networks, it is not difficult to understand why Brazilians would spend hours generating, sharing and reading content on other social networks and not on Wikipedia. It's well known Wikipedia's content requires much more time, research and confidence about the content production than casual sharing of informal content, and personal messages. It requires having the knowledge on how to source information, which is not vastly thought on secondary school in the Brazilian education system. Professors engaged in the Wikimedia Education Program report that it's even a challenge to teach how to write neutral and referenced articles to university students.

While we haven't achieved all our goals, such as provoking impact on retention of new editors in the Portuguese Wikipedia, we were happy to learn that the edits throughout the year as well as the average of content added per edit have increased in 2013.

Probably everyone who already worked to promote Wikimedia projects know how hard it can be to establish direct relations between actions and impact in the projects. For instance, we can measure how much content students in the education program added into our project (which is obviously very little if compared to the general edits by anyone, everyday), but we can't answer why there was a huge increase in participation in May and a fall in June.

Outreach activities, which we have done a lot in the first year and a half, is something that has been widely discussed about as activities that create very little impact overall on the projects. However, surprisingly, raising awareness on how Wikimedia projects work still seem to make much sense in Brazil. Not rarely we find out very few people know anyone can collaborate with Wikipedia. Therefore, although outreach activities are known for producing little direct impact in the short term, some opportunities and targets seem to be key to bring new contributors - not individuals particularly, but public institutions, large groups of people that, when decided to help Wikimedia grow, can help us create effective actions and projects to strengthen the projects. Working to foster collaboration on Wikimedia projects is not a science, then, but we can still bring theoretical and especially empirical knowledge to support us in the future of our activities.

Even considering a lot of challenges in 2013 we have seen a recovery (if unstable) on the Portuguese Wikipedia in the first semester, followed by another fall in the second semester . 2011 was a year in which the annual variation regarding new editors had been negative in all months. In 2012, the same indicator was positive in two months: May (just after a lot of media coverage on the Brazil Catalyst Project) and September, and the fall in many other months was lower than in 2011. In 2013, we have experience growth in three months: March (+10.79%), April (+4.48%) and May (+15.81%), and considerable lower falls in the other months so far (being -0,66% the minimum and -6,44% the maximum). It is hard, though, to attribute those general statistics to our work. While there is very little evidence of cause-effect relations to sustain our actions directly provoked changes in those trends, it’s not also 100% dischargeable the hypothesis that we collaborated with those results in a way. What we want and need to do more, from now on, is to track every result of our actions in order to better report on what has possibly been helping growth.

Active editors on Portuguese Wikipedia growth variation

New articles per Day on Portuguese Wikipedia growth variation

New editors on Portuguese Wikipedia growth variation

Very active editors on Portuguese Wikipedia growth variation

Education Program

During the year of 2012 we have piloted the Education Program in two phases: the first one started with one single contractor in December, 2011, engaging 5 professors from some of the most renowned universities in Brazil to develop class assignments for Wikipedia articles.

This first pilot served us with many learnings: we realized the less familiar the professor was with Wikipedia, the more difficult it was to get articles migrated to the main domain. Also, despite the efforts of some campus ambassadors, the courses in São Paulo did not achieve the results we expected. One of the courses in Rio was also called off by the professor, because she didn’t find wise to expose first year students to the task of creating new articles or significantly improve them. Instead, she ran a pilot with an extension course (a smaller, but voluntary and focused on Wikimedia course).

One of our impressions on the first pilot was that professors did not have a realistic idea of how much efforts doing class assignments for Wikipedia would require (and actually neither we did until our first experience), and also they weren’t committed to the same targets and metrics we had expected as results for the program. The other conclusion was that we should not define in advance which universities we should focus at. Instead, we should raise information on the plans made by the professors, their available structure and the existence of ambassadors to support them, to make the decision on where we should focus our attention. Additionally, we also realized we should not restrict the program to professors of our previous acquaintance circles.

Workshop for professors selected in the open call in the Brazilian Wikipedia Education Program

Having all that in mind, we decided to launch an open call for Brazilian professors. We would select 15 proposals to provide support to during the second semester. We received 36 proposals, out of which we selected 16. However, many of them were impacted by the national strike led by Federal Universities. Only the private and state ones could really develop the project throughout the second semester of 2012. This has also impacted our plannings for training (originally one in São Paulo and one in Rio) and we had to adapt the training sessions to a diversities of calendars, organizing 4 training sessions: in São Paulo, Rio, Paraná and Espírito Santo. Because of the professors strike, during the first semester of 2013, many selected courses were still going on, students were developing articles and professors revising them. Some professors gave continuity to this work, but we decided to improve our support structure before opening a new open call.

The Wikiprojects Bot

We’ve put our efforts in improving online and print materials as well as to run a pilot with professionals from the Brazilian health system in order to improve articles in their respective areas - for that, we invested in the development of a bot, similar to the Teahouse bot, which will invite people editing health related articles to participate in the Wikimedicine project. After a long development phase and many tests, the bot got a flag and will be running this semester. The national meetup of the targeted departments of the health Brazil system has also been postponed for this semester, so we’ll take part in that, present the project and send over an email with instructions and tutorials to the participants.

The present/future of the program

We have been working on providing the material and encouraging professors to develop activities during their course period withing Wikimedia projects. While we have focused on the first year and a half on Wikipedia articles, now we are more opened to different approaches as long as they engage more people in the projects and/or bring new and good content. The challenge here is to be didactic, simple and attractive. But through examples of what can be done we believe we'll engage more university professors and high school teachers into the Wikimedia world, and produce results and the mid and long terms.

To know more about the program

Get to know more about the 34 courses to which we provided support and the systematized learnings. Among our findings, we have concluded that although a few ambassadors were extremely helpful in the Brazilian program, the engagement of professors themselves is key for the success of the program. In general, professors that were more directly engaged online achieved better results than the ones that relied mostly on ambassadors to get the program developed.

Localization and original creation of tutorials

We have rebuilt the main education program page in order to meet our current objectives of the program: make it more and more independent on us, creating a network in which professors help each other and community members and professors can coordinate the work they intend to do together. We've also localized the education program extension, though we are being conservative about that since we heard there will be a general remodeling of that.

Meeting at Faculdades Integradas Rio Branco on the localization of tutorials in Portuguese

The Brazilian team has been localizing and creating online and print materials in order to allow the sustainability and growth of the project, which so far has been too dependent on our consultants. We involved students at the Design Course of Faculdades Integradas Rio Branco to localize tutorials in Portuguese.

We have also supported the community in creating the Tutorial, through suggesting, offering translation and helping in the localization. After the kick off, the community ended up doing almost all the job and we’re now finalizing and suggesting localization and changes.

We have also produced, in partnership with the University Gama Filho, a series of videos to teach their distant learning students about the Wikimedia projects, as well as how to edit Wikipedia and engage in the Wikimedia Education Programa. We have invited the community to participate and the results can be checked out at Commons:Category:Vídeos_para_curso_de_tradução_da_Universidade_Gama_Filho.

Data analysis

The community engagement activities are carried out through a set of approaches: data analyses: supporting community in better understanding and evaluating the development of Wikimedia projects, especially Wikipedia, by providing data for analyses.

We are also supporting the community in building a reference page for the Portuguese Wikipedia on WMF ToolLabs.

Please check some of the products:

• Timeline relating general stats and events/changes in Wikipedia: This timeline was created from a list built by community members about the main events throughout the history of Wikipedia. The idea is to allow easy editing from the community to help anyone understand whether (and in which extent) changes on the rules, changes on the community, changes of technology impacted the growth of Wikipedia. It has been recently added the feature of automatic update through the addition of events on the Wikipedia page.

• Planning for analyses and experiments: http://meta.wikimedia.org/wiki/Programa_Catalisador_do_Brasil/Planejamento_2012-2013/An%C3%A1lise_de_Dados_e_Experimentos : we invited the community to colaboratively write the analyses and experiments agenda. This document shall now be reviewed to match this partnership project terms.

• Research on reverts, vandalism and effects of captcha ( phase 1 and phase 2): During the controversial removal of the emergency mode of captcha in Portuguese Wikipedia we developed a research on reverts and vadalism, helping the community to make data-driven decisions concerning vadalism fighting. It made the community rethink its concerns about growth and admin work, and led to a task force of Abuse Filter improvement.

• Geolocated Edits: we created a page that shows countries share of edits on Portuguese Wikipedia.

• Localizing Snuggle: we are working on the localization of the Snuggle for Portuguese Wikipedia with Aaron Halfaker. A beta version is being tested at http://pt-snuggle.instance-proxy.wmflabs.org

• pt:Wikipédia:Ptwikis : We are helping community to learn how to access wikipedia data using the tool labs databases and they are creating monitor tools in this shared development environment. The tools available in ptwiki can be found here.

• Mobile Edits count: we created in a community shared environment at tool labs a graph that shows the edits done using the mobile interface by registered and anonymous users. This interface is ready to show data from all Wikipedia, but the legends still needs to be internationalized.

• Monthly stats board: From a very basic and manual regular update of our monthly stats, we have evolved to an monthly stats board making graphs that show trends and comparisons of the main statistics regarding Wikipedia.

• Annual stats board: Based on the monthly stats data, we are analysing annual variation of the main statistics on Portuguese Wikipedia.

Hackathon: building data analyses community and improving filters for the Portuguese Wikipedia

• Education Program Evaluation: we started a pilot to measure students behavior during the courses offered in the context of the Wikipedia Education Program. We expect to have this data automatically generated with the WEP mediawiki deployed in the Portuguese Wikipedia.

• Translating technical content: a big amount of mediawiki's documentation isn't translated to Portuguese. We been translating some of this page to allow non-English speakers members of the community to have access to this information.

• Surveys on outreach activities: https://s.qualtrics.com/SE/?SID=SV_eXuaZkTsIRAx261&Preview=Survey&BrandID=wikimedia

Data analyses community building: supporting the community in developing skills to independently create and analyse data: a portal centralizing access to a variety of data (both raw and collected/organized) and how-tos was created (it’ll still be edited, but the release of the version 1.0 is almost done). Also, the portal regarding data analysis created on Meta has been translated into Portuguese, making it available for Brazilian researchers and volunteers eager to work on data analysis.

Community Support

• Office hours with community regularly on Friday evenings to discuss the projects, data analysis,
• Encouraged and supported the revamp of the Correio da Wikipédia, the Portuguese Wikipedia Signpost
• Connected high school professors to Brazilian Wikimedia movement to develop the Education Program in schools (as featured in the major newspaper

Education Program present at the International Free Software Forum at Porto Alegre, Brasil

• Liaised the deployment of Notifications tool within the Portuguese Wikipedia, by translating the tool, the FAQ, announcing its release and following up with feedback
• Supported the community on the Visual Editor feedback process
• Helped community to evaluate filters on Abuse Filter for improvement.
• Supported the creation of SQL queries for community members ran researches.
• Supported Wikiprojects revamp.
• Sent out media inquiries to the community
• Dozens of outreach activities have taken place, with regular online survey to assess the quality and the fulfillment of our objectives.

Our reports

Weekly reports (in Portuguese)

• Relatórios semanais de dezembro 2013 - PT
• Relatórios semanais de novembro 2013 - PT

Monthly Reports

• December Report 2013 - PT
• November Report 2013 - PT
• October Report 2013 - EN
• September Report 2013 - EN
• August Report 2013 - EN
• July Report 2013 - EN
• June Report 2013 - EN
• May Report 2013 - EN
• April Report 2013 - EN
• March Report 2013 - EN
• February Report 2013 - EN
• January Report 2013 - EN
• August 2012 - EN

Team meetings

• Reuniões da Equipe na semana de 10/12/2013 - PT
• Reuniões da Equipe na semana de 02/12/2013 - PT
• Reuniões da Equipe na semana de 23/09/2013 - PT
• Reunião da Equipe em 16/09/2013 - PT
• Reunião da Equipe em 09/09/2013 - PT
• Reunião da Equipe em 02/09/2013 - PT