Grants:IEG/Wikipedia Massive Open Online Courses

status: not selected

Individual Engagement Grants	review grant submissions	visit IdeaLab submissions
eligibility and selection criteria

project:

Wikipedia Massive Open Online Courses

project contact:

User:Ragesoss

participants:

Sage Ross, Alex Stinson, and Ryan Malloy

summary:

Develop and run a course about Wikipedia on Coursera

engagement target:

English Wikipedia

strategic priority:

Increasing Participation

total amount requested:

30,000 USD

2013 round 1

Project idea

Update 2013-02-18

Tentatively (barring unforeseen institutional difficulties), the professor leading the course will be Rosta Farzan (w:User:Rostaf), a social computing researcher at University of Pittsburgh, and we'll use Coursera, which is already affiliated with the University of Pittsburgh. Together with Robert E. Kraut (who is also likely to be involved with the course), Rosta designed the Wikipedia portal for the Association for Psychological Science's Wikipedia Initiative (about which they have some forthcoming research). The plan at this point is to have the individual modules/lectures presented by many different Wikipedia experts of different types: Wikipedia researchers, Wikipedians with a deep understanding of one particular aspect of the project, technologists, etc.

Massive Open Online Courses (MOOCs) have taken off hugely in the last few years. The basic idea is: college-level courses offered for free from great teachers at many different universities, with video lectures, automatically graded quizzes, and in many cases, homework assignments evaluated by peers. The idea has been around for a while, but only recently has the course experience gotten good enough to be worth most people's time. The leader right now seems to be Coursera, which has over 200 courses and typical enrollment of tens of thousands of students (sometimes more than 100,000) per course; edX, Udacity, and Class2Go are other major platforms, and many universities are developing their own similar systems.

A Wikipedia course on a major MOOC platform could attract thousands of new editors. Since MOOC students tend to be self-motivated rather than grade- and degree-driven, a Wikipedia MOOC would avoid many of the pitfalls of the approach we've tried so far with professors assigning Wikipedia contribution in conventional university courses, where they follow the letter of the assignments but aren't usually invested enough to stick around. Students would, for the most part, sign up for a Wikipedia MOOC because of intrinsic interest in Wikipedia. (The kinds of people who would have become Wikipedians 8 years ago are now flocking to MOOCs to slake their self-driven thirsts for knowledge and community.) So the focus would be on recruiting Wikipedians rather than building content. It would explore the different aspects of Wikipedia and how it works—from our policies and culture to deletion processes to the sister projects, depending in part on the interests and background of the professor leading the course—with article writing only an optional element near the end (once the uncommitted students had fallen away, as typically happens with the majority of students enrolled in any MOOC). Students would evaluate each others' contributions (in sandboxes at first) against explicit rubrics based on what is and isn't good to do on Wikipedia. (With a big enough set of students, we could even allow students to work in multiple languages, with peer evaluation groups according to language so that students who speak the same language could help each other learn to contribute on any language Wikipedia. Although the major MOOC platforms are based in the United States, they have students from all over the world.)

The major stages are:

1. Finding the right professor (someone with a pretty deep understanding of Wikipedia) who is able and willing to head a Wikipedia course (and whose university will let them)

   I (Sage Ross) have several prospects just from reaching out privately to Wikipedian-professors I know personally, and I haven't yet reached out to the Wikimedia research network. This grant should be accepted provisionally, but only go forward once we reach the point of finding an appropriate professor and getting into a MOOC system... at which point the real work begins!

   Finding a professor: Done! Rosta Farzan, a social computing researcher at University of Pittsburgh, will head the course, but we plan to divide the course content up among many guest lecturers who present videos on their individuals areas of expertise. So we'll tap into the network of Wikipedia researchers, technologists, and outstanding editors to put together an interesting and diverse set of perspectives.

   We will be working to get officially onto Coursera in the next few months.

2. Developing the course content

   I have experience developing Wikipedia-centered course plans, from my own Wikipedia assignment as a graduate teaching fellow back in 2006, to the initial model syllabus used in the Public Policy Initiative, to a recent revamp of that sample syllabus that is the basis for many of the current courses on Wikipedia. I also have some experience in—and a strong desire to learn more about—educational video production. I also have a pretty broad knowledge of Wikipedia, how its various facets work, and what academic scholarship has had to say about it.

   Alex Stinson (Sadads) and Ryan Malloy (Cryptic_C62) also bring extensive experience with designing Wikipedia course content and instruction plans. Both have served as Campus and Online Ambassadors, and Alex has trained other Ambassadors and led academic outreach efforts, while Ryan has voice acting skills (check out this screencast he made) that will be put to good use to fill in the gaps between guest lectures.

   Robert Kraut, professor at Carnegie Mellon University, will help plan the curriculum and recruit experts to deliver individual course modules.

3. Running the course

   Making sure things go as smoothly as possible, both within the MOOC system and on-wiki, will be a very large project if the course is successful and has many thousands of students. I've got experience doing something similar with the Education Program.

   In addition to the grantee team, many experienced Wikipedians (on this grant page as well as privately) have offered to help make sure the course runs smoothly. The course content should also be of interest to many Wikipedians, so the enrolled students will likely be a mix of newcomers and established editors.

Project goals

The central goal is to take a first run at a Wikipedia MOOC, with three major outcomes:

• Experience and (freely licensed) course content that we can build on for future courses about Wikipedia (MOOC and conventional)
• Many new editors who contribute to Wikipedia during the course
• (Relatively) many new long-term Wikipedians who continue contributing after the course ends

Our preliminary plan at this point is that we will run the course on Coursera, with 10-12 weeks of course content, with video content contributed by a wide variety of Wikipedia experts. Enrollment in Coursera courses is not easily predictable (as both the number of users and the number of courses offered is rising quickly, and we don't have an easy way to calculate student interest in this particular course). However, anecdotal experience suggests that nominal enrollments on the order of several tens of thousands are common, with a few thousand students becoming active participants in a typical class.

Future goals (beyond the scope of this grant) include porting the course to other MOOC platforms, including ones more similar to Khan Academy where students work through modules independently at their own pace (in contrast to the defined start and end dates of courses on Coursera and many other MOOC platforms). Future efforts are likely to focus on open source platforms, in particular.

Project plan

Scope:

Scope and activities

The main tasks will be:

Developing a course plan

This will be done collaboratively amongst the professor and all grantees at the beginning of the project. I've done some preliminary work on this with Andrew Lih and Adrianne Wadewitz, but week-by-week and assignment-by-assignment planning will need to be done in close consultation with the professor heading the course.

Writing lectures

Depending on the professor's Wikipedia knowledge, lecture style, and preferences, scripting the video lectures and/or screencasts for the course may involve full scripting of lectures by professor and/or grantees, writing lecture outlines / bullets the professor can expound upon, or some mix. We'd want to do most or all of the writing before the course begins.

Video production

While approaches to video vary widely, we'd probably want a mix of live video of the professor and screencasts and/or slides showing the aspects of Wikipedia under discussion. The live lecture videos would be recorded by the professor, accompanying screencasts and/or slides would need to be recorded, and then it would get edited together to get to the final video product.

Getting the course content into the MOOC system

Since peer evaluation will likely be important for any Wikipedia MOOC, figuring out how to match the technology of the MOOC platform up with Wikipedia in a practical way will take some doing. Coursera, at least, has classes that have similarly complex peer evaluation components (such as design courses where students create websites and submit the urls so that peers can evaluate them according to specific criteria).

Managing the forums and issues that come up during the course

MOOCs typically have lively forums where students ask questions and discuss the course content, and some input for the course staff (professor and/or teaching assistants) is expected. Given the social and technical complexity of Wikipedia, there's likely to be quite a lot of attention required by the students, although some Wikipedians will likely be enrolled as well and can help similarly to what Online Ambassadors do in the Wikipedia Education Program on English Wikipedia.

Managing the on-wiki results of the course

Frankly, if a Wikipedia MOOC is successful, it's going to really shake things up. English Wikipedia has about 30,000 active editors per month. A very successful MOOC could conceivably double that. There will be problems, there will stresses on the community's processes and systems, and there will be a lot of need for communication and on-the-fly adjustments to the course to account for unanticipated problems on-wiki.

Evaluating and documenting the course

Since this would be just the beginning of Wikipedia MOOCs, we'd want to reserve some time to reflect on and document how the course went, and to analyze the contributions of the cohort of students who took it, and put in place a way to track that cohort over time.

Tools, technologies, and techniques

We'll most likely be using the Coursera platform, which has a nice set of capabilities for presenting video content and integrating in-video quizzes, longer stand-alone quizzes, and flexible peer evaluation-based assignments (where students evaluate the assignment submissions of fellow students against a grading rubric, for example). Furthermore, our professor works for the University of Pittsburgh which already has an institutional partnership with Coursera (see https://www.coursera.org/universities ).

We assume that most of the people who will be presenting course content (guest lecturers) will have or have access to personal or institutional audio/video recording equipment, but we may have to provide recording equipment in some cases, for which shipping around a DSLR and digital audio recorder should suffice. In some cases, it may make sense to fund travel for volunteer videographers instead.

Producing great videos, as well as quizzes and user-tested peer-evaluation assignments which provide meaningful assessment, will be the central work of the project. Theopolisme, who is an expert with video production, will advise us on the particulars of tools and techniques for the videos. As much as possible, we'll seek out guidance from experts in online teaching as well, to make sure we're designing effective quizzes and other assignments.

For the analysis of student contributions afterwards, I expect that the toolset being built by the WMF analytics team will have everything we need for a basic overview. A list of student usernames will be collected through Coursera, so that can simply be plugged in and then we can explore what the students did and continue to do.

Timeline:

This is a preliminary timeline to give an idea of the breakdown of activities. It will need to be adjusted based on the actual time we're able to schedule the course, which might conceivably start as soon as July 2013 or as late as October 2013, depending the platform and the professor's schedule.

June

Course design

• Outline the course topics
• Identify and reach out to guest lecturers / experts to present each topic
• Decide the basics of the assignments and quizzes
• Draft lectures for weeks 1–2 of the course (approximately five video lectures per week, 10–20 minutes per video)
• Do trial recordings of the first few lectures, to work out the production workflow

July

Content creation

• Draft lectures for weeks 3–6 of the course
• Continue securing guest lecture volunteers
• Coordinate the production of guest lectures for those who volunteer to present them
• Write quizzes and assignment instructions for weeks 1–4, implement them on Coursera and test them
• Record additional lectures to complete the video slate for weeks 1–4
• Post-production to produce final videos for weeks 1–2

August

Content creation continues

• Draft lectures for weeks 7–10 of the course
• Record lectures for weeks 5–8
• Produce final videos for weeks 3–5

September

Course begins, content creation continues, course plan adjustments based on early results

• Record lectures for weeks 9–10
• Produce final videos for weeks 6–7
• Run Weeks 1–4 of the course, interact with students in the forums, monitor for problems, and adjust course plan based on early feedback and problems

October

Course continues, content creation wraps up

• Produce final videos for weeks 8–10
• Run Weeks 5–8 of the course.
• Produce new content as necessary based on course plan adjustments

November

Course concludes, reporting and analysis

• Run Weeks 9–10 of the course
• Reporting and data analysis

Budget:

Total amount requested

USD $30,000

Budget breakdown

Time and spirit of the grantee team

$25,350 (84.5%)

• Sage Ross: Project management and assorted course content creation, 6 months, 20 hrs per week (plus planning and coordination work between now and the start of the grant period in June, and this covers the cost of childcare for 2 while Sage is working as well): $12,000

• Ryan Malloy: Recording assignment videos, instructional screencasts, forum management, 4.5 months, 30 hours per week: $9450

• Alex Stinson: Course planning, lecture drafting, assignment design, on-wiki coordination, 3 months, 20 hours per week: $3900

Video production expenses

$3000 (10%)

• Video production (potentially including travel for volunteer videographer, recording setup for Cryptic C62, other unanticipated equipment expenses, and/or professional editing): $3000

Equipment & shipping

$1650 (5.5%)

• Recording equipment (so that we can ship equipment around to guest lecturers that don't have their own):
  • Zoom H4n Handy Portable Digital Recorder : $280
  • Canon EOS Rebel T4i camera with lens: $910
  • MegaGear camera case bag (to keep camera safe during shipping from person to person): $50
  • Dolica GX600B200 tripods: $60
  • (4x) 32 gb class 10 SD cards @ $35 each: $140
• Shipping of camera equipment to 6 guest lecturers: 7 one-way shipments @ $30 each = $210

Intended impact:

Target audience

• Wikipedians who care about our decline numbers of active editors and/or want to learn more about the community in a structured environment.
• The students (most of whom, we expect, will be Wikipedia readers—including many who want to get involved in a structured way)
• Everyone in the world (in the long term)

Fit with strategy

Increasing Participation

MOOCs have an audience of self-motivated learners that is now in the millions and growing quickly. These types of learners have very similar motivations as members of the Wikimedia community. We need to meet them where they are and welcome them into our community as both editors and informed communicators and advocates for the project.

From anecdotal reports, a majority of students using Coursera and other MOOC systems are outside the United States. We anticipate at least some course assignments that include the option of organizing into student groups by native language to discuss and contribute to Wikipedias in languages other than English.

Sustainability

If we get a first Wikipedia MOOC going successfully, it could be repeated and built upon, and we could also expand and diversify, bringing on new professors with different perspectives on Wikipedia, getting Wikipedia courses onto different MOOC platforms, and generally riding this wave of change in the higher education system.

The great majority of effort and resources for this first course will be spent on developing the course design and video content, which will have a free license. Thus we anticipate future courses without major modification/improvement of the course materials could be run with minimal financial resources.

The modular course content could also be used to build more independent learning resources (a la Khan Academy, where students go through at their own pace) that, once set up, would provide an option for an always-open course that would require very little maintenance.

Since some of the videos will essentially be tutorials on specific aspects of Wikipedia, we also expect that some of the course materials will be useful on Wikipedia itself, replacing or supplementing existing help content.

Measures of success

The central measures of success are:

• Number of students who join Wikipedia during the course and stick around as active editors after the course ends.
• No excess cleanup burden on the Wikipedia community (ie, the students to more to help than hurt).
• Video lectures are produced that are judged engaging and useful by students, and accurate by experienced editors.

Secondary measures of success include:

• Number of students who are active editors during the course
• Reduction of Wikipedia maintenance backlogs (depending on whether the course includes any assignments that point students to cleanup backlogs)

Risks and challenges

Many newcomers at once

Aside from the basic challenges of planning and executing a MOOC, with all the content development and technical work that entails, the main risks center on the effects the students have on Wikipedia and its community. English Wikipedia has only about 30,000 active editors per month, so a successful MOOC that brings in several thousand new active editors (and tens of thousands of new editors who don't become active) could be highly disruptive.

Good course design will be critical to avoid such disruption. The central focus will be on teaching the students about Wikipedia, without assigning them to do things that will require extra cleanup. So early on, there will be nothing outside of sandboxes that students are assigned to do on Wikipedia. There will be quizzes about various key aspects of Wikipedia that will form part of the basis of their grades, and (likely) peer evaluation of sandbox work that will form most of the rest of their grades. By the end (when the less serious students had stopped participating), we can start to introduce mainspace editing, focusing on cleanup of stuff that's already bad. So the main idea is to teach people what they need to know to become Wikipedians—if they so desire—without requiring anything as part of the course that will create big messes.

Lots of content to develop

The preliminary plan includes developing a large amount of video content for the class, which will probably stretch the capacity of the professor, grantees, and volunteers. However, MOOCs vary widely in length and amount of lecture content, so we should be able to scale back the amount of video and/or length of the course if necessary.

By focusing on having course modules delivered by many guest lecturers, we should be able to put together plenty of compelling material—if enough Wikipedia experts are willing to contribute. As a fallback, the grantees are willing and able to develop as much lecture content as necessary, although we expect to be able to find plenty of guest lecturers, between the collective professional and community connections of the grantees and professors and additional outreach to Wikipedia researchers and other experts.

Technical aspects of Coursera are unfamiliar

The technical side of setting up and running courses on Coursera is not (yet) familiar to us, and some of the things we want to do (extensive peer evaluation for student assignments, and pedagogical research to test the effects of variations on the course material) are at the complex end of what Coursera can do.

However, Rosta is an accomplished programmer, the grantees are all broadly competent with web technology, and volunteer participants have extensive technical expertise as well.

Since Coursera caters to many different kinds of professors with different backgrounds and levels of technical expertise, we expect that the technical aspects will not be too much of a problem.

Participant(s)

General interest list

• w:User:Kevin Gorman, heavily involved in academic outreach in the bay area, a lot of experience with instructional design for Wikipedia-based assignments. Currently a student at Berkeley, though graduating this summer. Guess I am now technically the regional ambassador for California. Available in whatever capacity I'll be useful in.
• User:ezalvarenga, I will not have much time, but I would like to analyse the results, learnings and see what we can measure with this trial. And, of course, if possible, help as a wikipedian with my volunteer account (user:everton137) when possible.
• w:User:Dcoetzee. I've also been very involved in the Wikipedia Education Program in the Bay Area, and as a Commons admin my strongest skills lie in the understanding of file uploads and licensing and copyright. I have my own video equipment so I could record a guest lecture on this topic. I'm also very interested in forum related roles and could also do screencasts and help develop assessments. Please use me where I am most needed. :-) Dcoetzee (talk) 22:11, 19 February 2013 (UTC)

Professor/expert roles

Professor

Rosta Farzan, Assistant Professor, School of Information Sciences, University of Pittsburgh

Guest lectures / course modules

• Robert E. Kraut
• others to be recruited

Grantee roles

Project management

ragesoss (Sage Ross)

English Wikipedia and Wikimedia Commons admin, editor of The Signpost (2009-2010), Online Facilitator for the Public Policy Initiative (2010-2011).

Presenter/Voice actor/Content development/Forum management

Cryptic C62 (Ryan Malloy)

6+ years of content editing on the English Wikipedia, with 9,000 edits and 5 Featured Articles to date, along with extensive experience as an FAC reviewer. Three semesters as a Campus Ambassador to the IASL. Outside the wikisphere, former video tutorial host for eHow and the Worldwide Center of Mathematics.

Curriculum organizer/Assignment design/Content development

Sadads (Alex Stinson)

English Wikipedia admin with 6+ years of editing experience and early campus and online ambassador for the Public Policy Initiative. Currently a graduate student in English and writing instructor at Kansas State University. Has experience developing instructional content and online support materials for students relates to Wikipedia and other classwork.

Possible volunteer roles

Most of these need to get done by someone, but that doesn't necessarily mean a grantee. If you're interested in volunteering, or have a particular strength in one of these areas but would need grant support to take on the role, feel free to list yourself as either a volunteer or potential grantee.

Course planning

(ie, which topics will be covered in lectures, and in what order)

• ?

Lecture/script writing

(please indicate which specific topics you're keen on writing)

• ?

Assignment design

(including where in the course plan each assignment fits)

• ?

Screencasting

(these will probably done after the lectures are recorded, in sync with them so they can be combined with the lecture video & audio, although other workflows are possible)

• ?

Video production & effects

(especially, combining video and/or audio lectures with screencasts and still images)

• I do a lot of professional videography and things of that ilk, so I'm happy to help with this. After talking with Sage, it looks like I'll be able to assist with nearly all steps in production from a filmography standpoint, including helping to direct the videos and later editing them and establishing a consistent format. Theopolisme (talk) 16:29, 17 February 2013 (UTC)

Forum coordination

(trying to make sure the overall forum dynamics are healthy, being the official voice(s) of the course on the forums, and coordinating the efforts of other forum helpers)

• ?

Forum help

(the more the merrier)

• Dcoetzee (talk) 22:11, 19 February 2013 (UTC) (I can also do forum coordination is no one else steps up to the plate on that)

On-wiki individual help

(the more the merrier)

• ?

Data analysis

(we'll almost certainly have a list of student usernames, and WMF will have some self-serve tools for pulling statistics of defined user sets, so this is something anyone with [a willingness to learn] basic data analysis skills could contribute to)

• ?

External communications/media relations

(this could generate some media interest, so someone with media experience might come in handy)

• I interned with the Foundation's communications team a couple summers ago, which didn't involve much direct media contact, but did involve fastidious tracking of press coverage of Wikimedia, writing pre-interview briefings for interviewees, drafting external communications, etc. I've been involved in a number of other projects in non-Wikimedia arenas that have involved significant direct media contact. Kevin (talk) 22:39, 19 February 2013 (UTC)

Other

(if you'd like to help in some other way, just describe what you'd like to do)

• ?

Discussion

Community Notification:

• Village Pump (proposals) on English Wikipedia
• Education Noticeboard on English Wikipedia
• Wikimedia Education mailing list
• Esplanada (similar to village pump) on Portuguese Wikipedia
• Wikimedia Brasil mailing list

Endorsements:

Do you think this project should be selected for an Individual Engagement Grant? Please add your name and rationale for endorsing this project in the list below. Other feedback, questions or concerns from community members are also highly valued, but please post them on the talk page of this proposal.

1. This is not going to be an easy idea to pull off, but if it is successfully pulled off, it has the potential to be utterly game-changing. It's obviously not guaranteed to work, but I think it's very well worth giving it a shot. I would be significantly more dubious of the idea if it was suggested by anyone other than Sage, but Sage is... well... Sage. If anyone has a realistic chance of pulling it off, he does. I'll assist in whatever way I can. Kevin (talk) 08:45, 12 February 2013 (UTC)
2. Ditto on Kevin's point that Sage is well situated to coordinate such an effort. Accessing individuals who are committed to knowledge and are already motivated would be extremely important in the long run for building a larger volunteer community. I would like to endorse this effort! Sadads (talk) 15:04, 12 February 2013 (UTC)
3. Worth trying out. Editing Wikipedia is a form of life-long learning. Fred Bauder (talk) 16:24, 13 February 2013 (UTC)
4. Agree completely with Kevin. If done well, this could do amazing things. And Sage makes it much more likely to be amazing! Let's give it a shot; the potential payoff is well worth it. Andrew Gray (talk) 16:38, 13 February 2013 (UTC)
5. It's a worthwhile idea to pursue. --Piotrus (talk) 22:39, 14 February 2013 (UTC)
6. I am SO happy to see this! For me, this is the ideal form of outreach, tapping avenues that are currently of interest to the pedagogy community. I would love to cover this in the form of a blog post on HASTAC if you're okay with it. Also, I've been closely looking at failure within Wikipedia, ordering knowledge as well as knowledge gaps. If you want to make it richer and also work on the philosophy and politics of making knowledge through consensus communities, I'd love to help develop that module. Cheers! Noopur28 (talk)
7. I strongly endorse to try a MOOC about Wikipedia or with assignments to use Wikipedia. Recently I've created the page for MOOC using Wikipedia and I am glad to see this proposal being submited. With my experience with more than one year as a coordinator of the Wikipedia Education Program in Brazil, I truely believe this is worth trying and I think it is very possible it will be a good way t bring new editors, since students who will do the course are those willing to learn. I hope I will have some time to help! Promising! --Tom (talk) 12:19, 15 February 2013 (UTC)
8. The project is exactly what Wikipedia needs, and with that said I strongly endorse it. I trust Sage completely, and think something really great can happen here. Theopolisme (talk) 15:54, 18 February 2013 (UTC)
9. Although I have an obvious conflict of interest as one of the potential grantees, I would like to say this: The people who are self-motivated enough to complete a MOOC may just be the people who are self-motivated enough to be valuable contributors, but never thought to try. If ever there were a net that was perfectly suited to catch a few hundred (or thousand) of those driven individuals, this would be it. --Cryptic C62 (talk) 17:54, 19 February 2013 (UTC)
10. I think this could have a huge positive impact on the number of well-informed new editors we get joining the project. Many people are afraid to jump in and edit without a more thorough understanding of things, and by taking this course they will overcome their fears. I suggest having a "badge" or "certificate" userbox for userpages demonstrating successful completion of the course. Dcoetzee (talk) 22:15, 19 February 2013 (UTC)