Grants:APG/Proposals/2019-2020 round 1/Wiki Education Foundation/Impact report form

Purpose of the report

edit

This form is for organizations receiving Annual Plan Grants to report on their results to date. For progress reports, the time period for this report will the first 6 months of each grant (e.g. 1 January - 30 June of the current year). For impact reports, the time period for this report will be the full 12 months of this grant, including the period already reported on in the progress report (e.g. 1 January - 31 December of the current year). This form includes four sections, addressing global metrics, program stories, financial information, and compliance. Please contact APG/FDC staff if you have questions about this form, or concerns submitting it by the deadline. After submitting the form, organizations will also meet with APG staff to discuss their progress.

Global metrics overview - all programs

edit

We are trying to understand the overall outcomes of the work being funded across our grantees' programs. Please use the table below to let us know how your programs contributed to the Global Metrics. We understand not all Global Metrics will be relevant for all programs, so feel free to put "0" where necessary. For each program include the following table and

  1. Next to each required metric, list the outcome achieved for all of your programs included in your proposal.
  2. Where necessary, explain the context behind your outcome.
  3. In addition to the Global Metrics as measures of success for your programs, there is another table format in which you may report on any OTHER relevant measures of your programs success

For more information and a sample, see Global Metrics.

Metrics from our proposal

edit
Metric Achieved outcome Explanation
1. # of total participants 15,476 We nearly made our goal of 16,325, an impressive statistic given the pandemic.
2. # of newly registered editors 13,218 A larger percentage of our participants were repeat participants in our programs than expected.
3. # of content pages created or improved, across all Wikimedia projects 26,690 We exceeded our goal, mostly thanks to some active Wikidata contributors.
4. quantity: words added to the article namespace 11,582,343 We nearly made our goal of 12.8 million, an impressive statistic given the pandemic.
5. quality: number of articles increasing by at least 10 points on the ORES scale 3,681 We nearly met our goal of 4,620, an impressive statistic given the pandemic.


Global metrics

edit
Metric Achieved outcome Explanation
1. # of active editors involved 5 These are Wiki Education's staff members.
2. # of new editors 13,218 This is the same number as our "newly registered users" metric.
3. # of individuals involved 16,057 This is the same number as our "total participants" plus the instructors who taught with Wikipedia.
4. # of new images/media added to Wikimedia articles/pages 2,433 We had a total of 7,946 uploaded, but many are not yet used in articles.
5. # of articles added or improved on Wikimedia projects 26,690 This is the same number as our "number of content pages created or improved, across all Wikimedia projects" metric.
6. Absolute value of bytes added to or deleted from Wikimedia projects 76,836,067 We don't measure deletions, so this is only additions.


Telling your program stories - all programs

edit
 
Wiki Education All Staff meeting via Zoom, July 2020

When the COVID-19 pandemic struck the world early in 2020, it created havoc across every organization's plans. Wiki Education was no exception; the instability in the weeks and months following the initial declaration of a pandemic challenged our organization as a whole. The generous grant from the Wikimedia Foundation in 2020 represented 16% of our planned revenue; however, several of our major funders pulled expected renewals based on the economic instability following the pandemic, causing our annual revenue to be cut in half. To respond to this situation, we made the difficult decision to say goodbye to half of our staff and cut all non-essential spending. These painful cuts enabled us to survive 2020, and we are deeply grateful to the Wikimedia Foundation, the Stanton Foundation, and the Hewlett Foundation, whose renewals of our grants in early 2021 have set us up for success in the next year too.

In the first half of the year, we had made good progress toward our programmatic goals despite the challenges of the pandemic. Wiki Education was more fortunate than others in the Wikimedia movement because the majority of our programs were already based online. The support we provide to college and university instructors in our Wikipedia Student Program was virtual already, and the Wikipedia and Wikidata courses we taught in our Scholars & Scientists Program were already conducted via Zoom. In the second quarter of the year, immediately following the start of the pandemic, engagement in our programs decreased slightly as some participants needed to reduce commitments to adjust to new situations in their lives caused by changes stemming from the pandemic. By the second half of the year, people had adjusted and if they enrolled in our programs they stuck around at similar levels to pre-pandemic; however, our reduced staff capacity meant we couldn't support as many program participants as planned. Nevertheless, given all these challenges, we came very close to meeting our overall goals.

We couldn't be prouder of how our staff responded to the challenges of 2020, nearly meeting all of our impact goals despite the trials of a global pandemic.

Wikipedia Student Program

edit

Quantitative targets

edit
Measure of success Goal (2020) Total % completed Notes
Total Participants 16,000 15,063 94% We came remarkably close to meeting our goals, given the challenges of the pandemic.
Newly Registered 15,500 13,039 84% We came remarkably close to meeting our goals, given the challenges of the pandemic.
Content Pages Improved 15,500 13,910 90% We came remarkably close to meeting our goals, given the challenges of the pandemic.
Quantity 12,500,000 11,104,310 89% We came remarkably close to meeting our goals, given the challenges of the pandemic.
Quality Articles 4,500 3,537 79% We came remarkably close to meeting our goals, given the challenges of the pandemic.

What we did

edit
 
The Progress Tracker helps guide students through the article expansion process.
 
The new grading view highlights where each student editor is in their assignment.

The spring 2020 academic term, which roughly correlates to the first half of the year, was supposed to be our largest ever: We'd successfully onboarded 409 courses, our largest cohort to date. To prepare for this large influx of courses, we debuted two new technical features, the Progress Tracker and grading view for instructors. To refresh from our midterm report:

  • The Progress Tracker scaffolds the article building work students do. When a student chooses an article, the Progress Tracker will appear for that article with a number of useful links. Students can then quickly access a sandbox link for them to craft a bibliography, another sandbox link for them to draft their article, and training modules relevant to each stage of the process. It also links to the sandbox pages where classmates will add their peer reviews.
  • The new grading view for instructors includes links to the key sandbox pages where students complete the different phases of their assignment — preparing their bibliography, drafting their articles, reviewing the drafts of their peers, and editing live Wikipedia articles. The goal of this new view is to make assessing student work easier for faculty.

While the end result of the term missed our expectations due to the pandemic, we still had meaningful impact on Wikipedia content, and we found these new technical features helpful in supporting students and instructors involved in our program.

This trend continued in fall 2020, where we still supported large numbers of students but with reduced staff capacity. We debuted a new application process, and asked faculty who were interested in teaching with Wikipedia to submit their course page by a certain deadline. As part of the application process, we asked if instructors were okay having students stay in sandboxes until we could evaluate their work, which was likely to happen after the official end of the term. We accepted about a third of the courses under the condition that students would stay in sandboxes. This enabled us to — with significantly fewer staff — still support 322 courses in fall 2020.

The courses produced some excellent work, e.g.:

  • A chemistry student at the University of Michigan created a new article on RNA theraputics, an important article that's received thousands of page views as people seek vaccine information.
  • Rice University students collaborated to significantly expand the article on education inequalities in the United States, adding information about the history, racial and other factors, and potential solutions.
  • A psychology student at the University of California Los Angeles created a new article on social projection, which has received tens of thousands of page views since it was created.
  • A medical student at Boston University significantly expanded the article on Methylprednisolone, used to treat COVID-19.
  • A medical student at Vanderbilt University significantly expanded the article on respiratory syncytial virus, or RSV.
  • Other Vanderbilt University students created the article on Indigenous storytelling in North America.
  • A group of ecology students from Eastern Washington University collaborated to completely rewrite a high-trafficked but not high quality article, energy flow, adding images from Wikimedia Commons.

Not only did our Wikipedia Student Program produce excellent content for Wikipedia, it also provided a lifeline for faculty facing Zoom teaching, something that's revealed in our end-of-term instructor survey. In the open text field for the survey, faculty shared their experiences:

  • "Wikipedia worked extremely well as a tool in the online classroom. It offered an asynchronous learning element to my course which was otherwise largely synchronous. Using breakout rooms on Zoom, it gave students an opportunity for collaborative work, which they told me they appreciated enormously as a way to connect with fellow students in a relaxed classroom setting."
  • "I actually chose to begin this assignment this year because of the pandemic, I was hoping to give the students an opportunity to collaborate and to do some virtual group work — hopefully reduced the feelings of isolation!"
  • "Because we wrote as a team, the Wikipedia assignment ended up helping us to build an esprit de corps which was so welcome in these COVID-online times."
  • "In a semester where students were anxious, overwhelmed, and depressed, it was more important than ever to have assignments that students found meaningful, and I believe that this fell into that category."
  • "This assignment brought energy to a semester that was weighed down by all of the stress of the pandemic."

We are grateful that our program was able to offer engagement for instructors, many of whom were facing the challenges of virtual teaching and learning for the first time.

Celebrating 10 years

We would be remiss in noting that 2020 also marked the 10-year anniversary of the start of the program. To honor the occasion, we published a series of blog posts from staff reflecting on the achievements.

We didn't just want to reflect on our perspectives of the last 10 years, however. We also wanted to showcase those who helped make it happen:

  • Max Klein and PJ Tabit joined Wikipedia to support student editors; both are still active today.
  • Jonathan Obar started teaching with Wikipedia in the pilot program; he's still teaching in our program today!
  • Amanda Levendowski was a student in the pilot program; now, she's a proud Wikipedian and a law professor who assigns her own students to edit Wikipedia.
  • Tighe Flanagan took three classes in the pilot program and then worked for the Wikimedia Foundation helping to expand the program globally.
  • Kasey Baker learned to edit as a student in the pilot program and has gone on to bring hundreds of others to Wikipedia.

What worked well

edit
 
Help requests from students were less frequent in 2020 than in 2019, and were also more evenly spread throughout the Fall term.

Our strategy of continually identifying the most common points of confusion and adapting the Dashboard to fix them has continued to pay off this year. Our main technical focus in late 2019 and early 2020 was a set of features for providing step-by-step guidance for the key stages of Wikipedia writing assignments, with related updates to standardize the locations for draft and peer review work and improve the interface for instructors to see what students did in each stage of the Wikipedia project.

We are also quite proud of our help structure for students as well. In the 10 years we've been running this program, we've learned a lot about how to socialize new contributors to Wikipedia. As we hoped, the combination of interface improvements and refinements to our help materials is reducing the amount of time staff spend responding to common problems. We found a modest decrease in the rate of help requests throughout 2020, compared with previous years. The requests were also spread more evenly throughout the term, with a lower peak in the busiest month of November — which is especially beneficial for us, as peak periods are the main constraint on how many courses we can support each semester. We were able to succeed in the second half of 2020 in part because of the dedication of our staff, but also because of the groundwork we've been laying for a decade, in terms of quality onboarding materials, clear training modules, and a robust technical platform. With only two full time staff members dedicated to the project in fall 2020, we were able to successfully support more than 6,800 new contributors to Wikipedia.

Our successes in this area are summed up in part of one student's reflection essay:

As a newcomer to this successful online community, I'll be honest, I was really overwhelmed. Writing a Wikipedia article seemed like a daunting task that I wasn't up for. But eventually, I did, and my article is now a hundred times more developed than it was before. The trick? WikiEducation. I don't think I could have done this without it.

WikiEducation is a step-by-step learning resource that teaches students and teachers how to contribute to Wikipedia. It's helpful for Wikipedia users, because it gives insight into what the websites norms and policies are. As a newcomer, this leaves less room for doubt about whether or not "I'm doing it right." I could imagine that feeling overwhelmed and unqualified to write articles for Wikipedia are oftentimes a deterrent for newcomers.

Not only is it enormously helpful for users, but Wiki Education is also helpful for the site of Wikipedia itself, because it does a great job of socializing users. As we learned in class, one of the greatest dangers to an online community is newcomers that break norms. Newcomers that do not have a sense of their role in the community perform poorly, don't commit, annoy experienced users, and turnover quickly. However, it is unlikely that a newcomer that has completed the Wiki Education training modules would break norms, because the spinoff site does a fantastic job of laying the ground rules for the website.

A third reason why I believe Wiki Education is a great tool is because it adds more credibility to the website. If all users were given the opportunity to complete the training module, it might lead to a better reputation of the website. Knowing that the site was only contributed to by trained users might change the minds of those teachers I had in high school, who believed Wikipedia was merely a space for non-academics to mess around and edit articles.

We look forward to continuing to provide support for students like this, so they can have quality educational experiences while improving Wikipedia, for another decade.

What didn't work well

edit

As mentioned earlier, in order to support more courses with our reduced capacity in the fall 2020 term, we attempted something new: encouraging a third of courses to leave work in sandboxes. While this helped us avoid a major challenge on wiki, we found it did not significantly reduce the staff time necessary to support these courses as anticipated, and results from our instructor survey showed students were less motivated by the assignment. We spent significant staff time in the final weeks of December combing through the work left in sandboxes, and reached a few conclusions:

  • Moving out of sandboxes earlier in the term helps head off obvious problems that doom projects from the start (poor sourcing, improper tone, etc.). If these are caught and re-directed early in the term, students can still salvage the project and create something Wikipedia-ready. The sandbox only process missed that step, leading to large chunks of unusable content left in sandboxes.
  • For good content, many articles had intermediate edits made between when the students did the work in sandboxes and when we finally had time to review it, making history merges impossible from our end, and students after the end of the term failed to follow-up on moving the work themselves.
  • Many students ignored our requests to stay in sandboxes and moved content live anyway; challenges emerged, however, because their course pages didn't have necessary instructions about how to do this, leading to some improperly moved content.
  • We still spent significant staff time answering questions for these courses during the term, and the amount of time to review each article in sandboxes after the end of the term was not worth it for the minimal high quality content we were able to actually move live.
  • Instructors who had previously had typical assignments noted decreased engagement from students this term, when they couldn't move work live.

For all these reasons, we've significantly scaled back on our sandbox-only courses for the spring 2021 term. A few instructors actually liked it, so we'll continue letting those participate in the program that way, but we no longer believe it will be worth it from a staff time savings and quality perspective, as we'd hoped.

Connections to strategy

edit

The work our Wikipedia Student Program does aligns well with Recommendation 8, Identify Topics for Impact. The sheer scale at which our program operates means it has a large influence on content on English Wikipedia.

In 2020, 44% of all pageviews of Wikimedia projects went to the English Wikipedia. In other words, one Wikimedia project has nearly the same number of page views as all the other projects combined. This outsized impact of English Wikipedia is demonstrated in other ways too: As one of the global languages, emerging communities often use translated articles from English Wikipedia to build coverage on their language Wikipedias.

But English Wikipedia still has significant knowledge equity gaps. And so programs like Wiki Education's Wikipedia Student Program are critical to the Wikimedia strategy of knowledge equity. Addressing content equity gaps on our most read project, a project used as the basis for translation by emerging communities, is critical if we want to achieve knowledge equity.

In 2020, Wiki Education supported more than 60 courses on topic areas related to race, gender, and sexuality content gaps. Some examples:

Through this program's work, we're able to continually improve English Wikipedia's coverage of topics related to knowledge equity.

As we mark 10 years of the program, it's also a moment to acknowledge this program is succeeding today in part because of following Recommendation 10, Evaluate, Iterate, and Adapt. We've been continually monitoring the program, evaluating it, iterating based on what we've learned, adapting to external factors, and scaling as we go. We believe our program is an existing success story for this model, and we're excited to see it be adopted more widely in the Wikimedia movement.

Wikipedia Scholars & Scientists Program

edit

Quantitative targets

edit
Measure of success Goal (2020) Total % completed Notes
Total Participants 175 308 176% We ran more Wikipedia classes than planned after demand for these increased, leading to us exceeding our goals.
Newly Registered 120 145 121% We ran more Wikipedia classes than planned after demand for these increased, leading to us exceeding our goals.
Content Pages Improved 700 1,280 183% We ran more Wikipedia classes than planned after demand for these increased, leading to us exceeding our goals.
Quantity 300,000 451,000 150% We ran more Wikipedia classes than planned after demand for these increased, leading to us exceeding our goals.
Quality Articles 120 144 120% We ran more Wikipedia classes than planned after demand for these increased, leading to us exceeding our goals.

What we did

edit

Our Wikipedia Scholars & Scientists Program was an unqualified success in 2020. As a program designed to train people to edit Wikipedia through a series of weekly Zoom meetings, we had to change nothing about the program when the pandemic hit and everyone started sheltering in place. We exceeded all of our goals for this program in 2020, including generating $169,000 in revenue from leading these courses.

In 2020, we led a total of 20 courses via Zoom. Course size ranged from 5 participants to 22 participants. We have three major sales models:

  1. Institutional payer: In this model, a particular organization buys out an entire course, and opens enrollment up to their members for no cost to the participants. An example of this was a course we ran with the Society of Family Planning, where their members (medical experts) improved articles related to abortion and contraception.
  2. Individual payer: In this model, Wiki Education decides to host a course around a particular theme, and sells individual seats in the course. An example of this was a course we ran on improving articles related to the 2020 election in the United States.
  3. Hybrid: In this model, a funder sponsors a particular course for improvement of a particular model, but doesn't limit enrollment to a particular group. We had two courses sponsored by the WITH Foundation, who sought to improve Wikipedia's coverage of healthcare articles related to developmental disabilities. We opened participation to healthcare and disability experts.

In each of these models, the goal is two-fold: (1) generate earned income for Wiki Education, and (2) improve the quality and quantity of content on specific topic areas on English Wikipedia by training experts. We succeeded in both goals in 2020.

 
A participant in one of our courses uploaded this photo of a COVID testing site in Arizona to Commons.

In particular, the program shows the importance of bringing experts and guiding them to improving a particular topic area. In the series of courses we ran related to COVID-19 (which stretched into 2021), our results show the huge impact a program like this can have on a limited topic area. Participants edited articles on:

  • "COVID-19 pandemic in" 32 different U.S. states (plus D.C.)
  • "COVID-19 pandemic in" 7 different countries
  • "COVID-19 pandemic in" 6 different regions (e.g., San Francisco Bay Area, New York City)
  • "Impact of the COVID-19 pandemic on" 10 different topics (e.g., education, migration, the environment)
  • COVID-19 testing, Face masks during the COVID-19 pandemic, the Johnson & Johnson COVID-19 vaccine, Misinformation related to the COVID-19 pandemic, Social impact of the COVID-19 pandemic, and more.

Participants also created the articles on, among others, the COVID-19 pandemic in the Navajo Nation and the impact of the COVID-19 pandemic on people with disabilities.

We saw other improvements to Wikipedia content from our expert participants:

  • An OB-GYN participating in one of our Society of Family Planning courses added a section on cesarean section ectopic pregnancies to the ectopic pregnancy article.
  • A participant in our course with the WITH Foundation overhauled the article on the Mitrofanoff procedure.
  • Another participant from the WITH course overhauled the article on muteness.
  • Prior to the 2020 U.S. election, when the topic garnered more attention, participants in a course we ran on voting rights added sections about racial disparities and economic factors, among others.
  • A physicist enrolled in one of our courses sponsored by the American Physical Society added an important caveat to the Density functional theory, illustrating the important work subject matter experts can bring to Wikipedia.
  • A course we ran to improve biographies of women led to the creation of the biography of Indian civil servant Nilam Sawhney, which has received thousands of page views.
  • A participant in a course we ran with 500 Women Scientists created the article on Latinx mathematician Carrie Diaz Eaton.

What worked well

edit

In 2012, Jennifer Doudna helped make one of the most remarkable breakthroughs in biology — the CRISPR/Cas9 gene editing system, which changed editing genomes from science fiction to science. While Doudna’s Wikipedia biography looked substantial, a lot of her research was missing or paired with the work of her male colleagues. In 2018, one Wiki Scholar transformed the page so that it emphasized Doudna’s scientific achievements.

Then, on October 7, 2020, Jennifer Doudna was awarded the Nobel Prize for her work on CRISPR.

Thanks to the scholar’s additions, the article explains the revolutionary potential of CRISPR and the role Doudna played in this research. Had she not added this content, millions of readers who came to Doudna’s Wikipedia biography to learn why she was awarded the Nobel Prize might have left with more questions than answers. This work reminds the public of women’s scientific contributions, inspiring new generations to pursue careers in science.

Stories like this remind us that the impact of improvements to a particular biography may not be visible for months or years — but it's critically important to tackle Wikipedia's equity gaps, and concentrated projects like our Wikipedia Scholars & Scientists Program enable us to do that. As participant Yohanna White wrote in a blog post for us about her experience in a course:

Everyone has a different reason for why they choose to be bold. I choose to be bold because as an underrepresented minority in science, it is a source of empowerment to be able to express my knowledge. I look forward to taking my boldness to the next level: hosting edit-a-thons to recruit and inspire potential editors! I am grateful for the opportunity to have a seat on the table. It is truly an amazing feat that a group of diverse volunteers can maintain the world’s greatest encyclopedia, considering that all it takes to make a difference is to create an account and be bold enough to click edit.

What didn't work well

edit

At the start of 2020, we'd been running 12-week courses in the program. We found that was the appropriate amount of time to deeply embed people into the Wikipedia community, giving them a full understanding of consensus-based decision-making, the more obscure policies, and more. Along the way, they wrote two full articles, according to our curriculum. But we were having a difficult time both selling and filling these courses: 12 weeks was a really long commitment for people to make.

Midway through 2020, we shortened the curriculum substantially in favor of 6 week courses. Instead of a deep dive into all things Wikipedia, we focused the curriculum in on the topic area at hand. If the course was focused on writing biographies, we spent a lot of time discussing BLP and notability but not MEDRS; if the course was on medical topics, we focused on MEDRS but not BLP. Instead of getting a more wholistic approach to all Wikipedia policies, we provided a deep dive into successfully editing a particular kind of article. And instead of asking participants to write two articles, we asked them to write only one. We found this to be incredibly helpful; we ran more courses in the second half of the year than we did in the first half, and the courses we ran continued to have meaningful impact, just more focused.

We look forward to continuing the 6-week course model in 2021.

Connections to strategy

edit

Recommendation 1, Increase Sustainability of Our Movement, calls for revenue generation for the movement as Initiative 7. Wiki Education has been successfully running this program with a revenue generation model since 2018; our work to supplement our fundraising efforts with an earned income model is, we believe, a first of its kind in the movement. In line with Recommendation 7, Manage Internal Knowledge, we've also extensively documented our learnings on Meta, in Category:Wiki Education Foundation evaluation reports. In particular, for this program, we also documented our adaptations of it in an update, in line with Recommendation 10, Evaluate, Iterate, and Adapt.

But like the Wikipedia Student Program, we think this program highlights work from Recommendation 8, Identify Topics for Impact. In the "What we did" section, consider some of the improvements made to articles. Through our program, experts added sections to articles about women's health, developmental disability healthcare, racial disparities in U.S. elections policy, biographies of women. These are among many content gaps related to equity that still exist on the English Wikipedia. In fact, most of our courses have explicit or implicit focuses on fixing English Wikipedia's knowledge equity gaps:

  • We ran two courses with 500 Women Scientists, designed to improve the number and quality of biographies of women scientists on Wikipedia.
  • We ran two courses with the American Physical Society, designed to improve the number and quality of biographies of underrepresented physicists on Wikipedia.
  • We ran two courses with the WITH Foundation, designed to improve the quality of articles related to healthcare for people with developmental disabilities. Participants in these courses included those with lived experience who we empowered to write Wikipedia articles about conditions they have.
  • We ran two courses with the Society of Family Planning, designed to improve the quality of articles related to abortion, contraception, and pregnancy on Wikipedia.
  • Individual contributors in courses related to COVID-19, voting and elections, and science policy also tackled elements of equity as they worked on their articles.

Bringing subject matter experts to Wikipedia and empowering them in a structured environment like this is key to solving English Wikipedia's knowledge equity issues. And, as highlighted in the Wikipedia Student Program section, this work is even more critical when we consider what a large percentage of the world's pageviews go toward English Wikipedia.

Wikidata Scholars & Scientists Program

edit

Quantitative targets

edit
Measure of success Goal (2020) Total % completed Notes
Total Participants 150 105 70% Because we ran more Wikipedia classes, we ran fewer Wikidata classes, meaning we slightly missed this goal.
Newly Registered 100 34 34% We significantly underestimated the value that people who had existing accounts find from our courses.
Content Pages Improved 2,000 11,500 575% A few participants began using mass editing tools, causing this number to significantly exceed expectations.
Quantity 15,000 27,033 180% A few participants began using mass editing tools, causing this number to exceed expectations.

What we did

edit
 
We ran all courses via Zoom.

Our Scholars & Scientists Program has two branches: A Wikipedia branch and a Wikidata branch. Where we ran more Wikipedia courses than originally planned, we ended up running fewer Wikidata courses. A newer program, we hadn't fully tuned our expectations for the courses when we created our proposal for 2020. Nevertheless, we're extremely proud of the work we did in this project. We ran a total of nine courses in 2020, empowering 105 people to become Wikidata contributors.

Similar to the Wikipedia courses, these are six-session courses that take place via Zoom. Course sizes ranged between 7 to 22 people, with most courses having 10–12 participants. At the beginning of the year, we'd experimented with two different levels of courses where participants met once a week for six weeks. Midway through the year, we changed the model to an "Institute" model, in which the group meets twice a week for three weeks. We found this model kept participants engaged, increased our enrollment, and created great outcomes for Wikidata.

Our courses have attracted a wide range of participants who've had excellent outcomes, which they documented in testimonial posts on our blog.

These contributors and their classmates improved more than 27,000 statements on 11,500 items, including creating 665 new items in 2020. We're excited about the contributions these participants have made to Wikidata.

What worked well

edit
 
Cradle school definition

One important part of our Wikidata courses that isn't encompassed by the top-level statistics is contributions people make to Wikidata that isn't just item editing. One example of this was when a participant added a new form to Cradle to add school data to Wikidata.

Cradle is one of the many Wikidata tools designed to make mass editing easier. In Cradle's case, it's a form-based tool to show fields that represent statements on Wikidata. Cradle allows users to look at lists of things and create new items based on these data models. It’s an important tool because it encourages users to contribute as much information as they can (some of it required, most of it optional). It also saves newer users the time of guessing or searching for the correct property to use. If you’re new to Wikidata, knowing which one is the best one to use can take a while to find. Around 100 of these forms exist for Cradle, but until one of our participants created one, no form existed for school items.

It’s one accomplishment whose effects have global reach throughout all of Wikidata. With this new form anyone who is interested in contributing data about schools has a template they can work with. This lowers the barrier for creating new items and encourages a standard set of properties to be used for those items. This can help with consistency in item description and querying. Now users can enrich Wikidata with more data about schools, which benefits the entire community.

What didn't work well

edit

We had intended to create a self-directed Wikidata course in 2020. While we made some significant progress on this in the first half of 2020, our reduced capacity in the second half of the year led us to put that project on hold, and also reduced our overall capacity for teaching Wikidata courses.

Additionally, we continue to struggle at correctly setting goals for our Wikidata course. This year, we significantly under-estimated the percentage of people taking our Wikidata courses who already have Wikimedia logins. Many have tried to contribute to Wikidata on their own, but found it too difficult to navigate, and they are grateful for the support of our structured learning environment.

We also have difficulty estimating the impact each course will have on Wikidata. In 2020, we had a couple of participants who really got into Wikidata, using mass editing tools and otherwise engaging in significant numbers of statements edited and items improved. But these new super active Wikidatans aren't in every class, so it's hard to count on them for our numerical goals. Another challenge is some participants are mostly interested in learning to use Wikidata tools or the querying service, leading to less engagement in content creation on Wikidata. And counting items edited or statements, as we do, doesn't adequately capture work people do contributing standard queries, engaging in property discussions, participating in item modeling in WikiProjects, or other contributions.

As more groups in the Wikimedia grantmaking ecosystem engage in Wikidata programs, we would love to participate in a conversation around appropriate global metrics that showcase the impact of Wikidata programs. We encourage the Wikimedia Foundation's evaluations and grantmaking teams to take a lead role in facilitating a conversation around Wikidata metrics.

Connections to strategy

edit

Since our Wikidata Scholars & Scientists Program is also an earned income model, it also follows Recommendation 1, Increase Sustainability of Our Movement, Initiative 7, Revenue generation for the Movement.

But like most of our programs, our largest connection comes with Recommendation 8, Identify Topics for Impact. As a global, multilingual project, Wikidata welcomes contributions from around the world. While our courses are taught in English, we've been pleased to see our participant base come from the international community. In addition to the United States, participants came from Australia, Brazil, Canada, Colombia, France, Italy, Germany, and the United Kingdom.

The international nature of the participants leads to diverse content contributions as well. For example, the participant from Colombia improved more than 100 items related to Colombia, such as the item on Colombian scientist Ángela Restrepo Moreno.

Amarílis Corrêa, a librarian at the Faculty of Architecture and Urbanism of the University of São Paulo, Brazil, took our course in September, and blogged about the experience for us. In her post, she wrote:

My first use of the knowledge developed will be working on the GLAM-Wiki I’m helping to implement for libraries of the University of São Paulo (GLAM Bibliotecas da USP), aiming to open data of scientific and artistic productions of the scholars, bibliographic data of the journals published by the university, and its cultural and scientific heritage collections. I intend to stay in touch and keep learning through the conferences and collaborative spaces recommended during the course and others (like IFLA Wikidata Working Group), as well as to share the achievements of this GLAM.

Undoubtedly, I’ll give credit to this great experience I had thanks to Wikidata Institute, the instructors Will and Ian, and all the classmates.

We look forward to continuing to support Wikidata's global impact.

Dashboard

edit

What we did

edit

Improving the performance, usability and flexibility of the Dashboard — both Programs & Events Dashboard and Wiki Education Dashboard — was an ongoing priority throughout the year. Programs & Events Dashboard was used in 2020 by more program organizers to track more programs than in any previous year (2,342 programs, led by 1,558 different organizers). Increased usage for Wikidata — and the high-volume editing that comes with it — has required us to focus on performance, specifically for increasing the rate of data updates for large, long-running programs. We also completed a major project to reduce the JavaScript footprint of the Dashboard, which should particularly help those using the Dashboard via mobile devices and those with limited bandwidth. Improvements to the data update process also included a set of features for better transparency, making it easier for program leaders and participants to see when data updates are happening and to identify the causes of any unexpected results.

What worked well

edit

Thanks to continued investments in maintenance, monitoring and automated testing, we were able to keep Programs & Events Dashboard running throughout the year with few service interruptions. We were able to respond quickly to support requests, for example to enable new languages and to refine the system for creating and translating training modules (which now features a full set of Swedish translations for the Editing Wikipedia training modules along with a new set of custom tutorials for GLAM projects created by Wiki Movimento Brasil).

What didn't work well

edit

Our exploratory work on an Android app went more slowly than we hoped, and we did not bring it to the point where it could be field-tested. We stopped development in mid-2020. We also faced challenges maintaining and updating our custom question-and-answer site ask.wikiedu.org, which we've now deprecated in favor of an FAQ system built into the Dashboard itself.

Connections to strategy

edit

Our support of the Programs & Events Dashboard is an example of Recommendation 6, Invest in Skills and Leadership Development, in action. We've created a global technical tool that program leaders around the world use to manage their programs, collecting a standard set of metrics. There are also tutorials embedded within: The basic Editing Wikipedia modules that have been translated in several languages, custom-built modules like the GLAM suite created by Wiki Movimento Brasil mentioned above, and more meta-level trainings for Wikimedians, like on how to run an edit-a-thon. A global platform like this supports individuals' needs in a centralized location, and has become a piece of essential infrastructure in the movement.

The Programs & Events Dashboard also hosts event safety trainings written by Wikimedia's Trust and Safety team that could help support Recommendation 3, Provide for Safety and Inclusion.

Revenues received during this period (6 month for progress report, 12 months for impact report)

edit

Please use the exchange rate in your APG proposal.

Table 2 Please report all spending in the currency of your grant unless US$ is requested.

  • Please also include any in-kind contributions or resources that you have received in this revenues table. This might include donated office space, services, prizes, food, etc. If you are to provide a monetary equivalent (e.g. $500 for food from Organization X for service Y), please include it in this table. Otherwise, please highlight the contribution, as well as the name of the partner, in the notes section.
Revenue source Currency Anticipated Q1 Q2 Q3 Q4 Cumulative Anticipated ($US)* Cumulative ($US)* Explanation of variances from plan
Selling Services USD $25,625 $22,613 $84,740 $33,557 $44,540 $185,450 $25,625 $185,450 We have strategically pushed revenue growth in this area since submitting our plan.
Selling Impact USD $2,450,950 $390,492 $42,799 $335,215 $294,724 $1,063,230 $2,450,950 $1,063,230 The pandemic caused a significant drop in our foundation support.
Total USD $2,476,575 $413,105 $127,539 $368,772 $339,264 $1,248,680 $2,476,575 $1,248,680 We decreased our spending to account for a lower anticipated revenue due to the pandemic.

* Provide estimates in US Dollars


Spending during this period (6 month for progress report, 12 months for impact report)

edit

Please use the exchange rate in your APG proposal.

Table 3 Please report all spending in the currency of your grant unless US$ is requested.

(The "budgeted" amount is the total planned for the year as submitted in your proposal form or your revised plan, and the "cumulative" column refers to the total spent to date this year. The "percentage spent to date" is the ratio of the cumulative amount spent over the budgeted amount.)
Expense Currency Budgeted Q1 Q2 Q3 Q4 Cumulative Budgeted ($US)* Cumulative ($US)* Percentage spent to date Explanation of variances from plan
Student Program USD $348,971 $86,261 $84,417 $59,090 $59,053 $288,821 $348,971 $288,821 83% A decrease in revenue due to the pandemic forced a dramatic cut in spending in the second half of the year.
Wikipedia Scholars & Scientists USD $455,844 $122,029 $124,459 $44,708 $46,171 $337,367 $455,844 $337,367 74% A decrease in revenue due to the pandemic forced a dramatic cut in spending in the second half of the year.
Wikidata Scholars & Scientists USD $317,822 $82,308 $84,544 $39,255 $39,218 $245,325 $317,822 $245,325 77% A decrease in revenue due to the pandemic forced a dramatic cut in spending in the second half of the year.
Technology USD $398,517 $88,352 $69,551 $33,333 $33,300 $224,536 $398,517 $224,536 56% A decrease in revenue due to the pandemic forced a dramatic cut in spending in the second half of the year.
General/ HR/ Finance/ Admin/ Board/ Fundraising USD $830,536 $248,864 $155,869 $93,088 $100,104 $597,925 $830,536 $597,925 72% A decrease in revenue due to the pandemic forced a dramatic cut in spending in the second half of the year.
TOTAL USD $2,351,690 $627,815 $518,840 $269,474 $277,845 $1,693,974 $2,351,690 $1,693,974 72% N/A

* Provide estimates in US Dollars


Compliance

edit

Is your organization compliant with the terms outlined in the grant agreement?

edit

As required in the grant agreement, please report any deviations from your grant proposal here. Note that, among other things, any changes must be consistent with our WMF mission, must be for charitable purposes as defined in the grant agreement, and must otherwise comply with the grant agreement.

  • None

Are you in compliance with all applicable laws and regulations as outlined in the grant agreement? Please answer "Yes" or "No".

  • Yes

Are you in compliance with provisions of the United States Internal Revenue Code (“Code”), and with relevant tax laws and regulations restricting the use of the Grant funds as outlined in the grant agreement? Please answer "Yes" or "No".

  • Yes

Signature

edit
Once complete, please sign below with the usual four tildes.

Resources

edit

Resources to plan for measurement

edit

Resources for storytelling

edit