Wiki Education Foundation/Quarterly reviews/2015-Q3 Classroom Program & Educational Partnerships

The following are notes from the Quarterly Review Meeting for the Wiki Education Foundation's Classroom Program and Educational Partnerships team on March 25, 2015.

Present: LiAnna, Jami, Frank, Samantha, Bill, Eryk, Renee

Participating Remotely: Sage, Ian, Adam, Ryan

Please keep in mind that these minutes are mostly a rough transcript of what was said at the meeting, rather than a source of authoritative information. Consider referring to the presentation slides, blog posts, press releases, and other official material.

Introduction

Wiki Ed Quarterly Review (Q3) for Classroom Program and Educational Partnerships

Jami: Just a reminder about the emphasis of these Quarterly Reviews; this one is focused on the Educational Partnerships, with a short update about the Classroom Program. With the timing of these quarterly reviews, for the Classroom Program, it means we’re able to discuss things in the middle of the term, so we can change course if necessary.

Classroom Program: Student Work in the Sciences

Presented by Ian Ramjohn. Highlights of good student work from the Sciences.

Slide 1

Basin Studies is an LSU class making good strides in content.

Slide 2

Here’s some graduate student work.

The map of Borneo was from Commons; many of these maps already existed.

Slide 3

Conservation biology, also at LSU. Species articles are pretty standard format and easy to get into, the learning curve is easy, but they’re showing that it’s still great work.

Slide 4

Environmental Disruptors of Development. Instructor is a long-term contributor to Wikipedia, and understands how to work with WikiChemistry project, for example, to get community input. He knows to do it, which is the nice thing about working with people with a history on Wikipedia. It’s nice, too, when instructors can support students in an informed way. I think this course and instructor’s reflections will be interesting.

Slide 5

This Neurobiology course’s instructor has run student work on sandboxes through TurnItIn, to make sure there’s no plagiarism, also had reviews of work in the sandbox.

Discussion

Frank: If articles have a standardized structure, is that enough to make it easier to write better articles?

Ian: Yes.

Frank: If we know there are areas where article structures are more or less set, would having these templates make it easier to scale?

Jami: Yes, depending on our goals. More templates and toolkits, maybe in the course page system, is something we’ve been talking about. In theory, we’ve been thinking of a drop-down menu offering options for standard formats, i.e., books, species, biographies, etc. But it doesn’t lend itself to all types of articles. Sociology, for example, is hard to structure as a template.

Frank: Are we prioritizing Sociology over Science, then?

Jami: Not necessarily, it’s just where are our content gaps.

Frank: I’m not saying we target only one or two areas, but, considering there are professors who start with no experience, perhaps it would be better to give new instructors easy templates with clear structures before they move on to something more difficult.

LiAnna: However, we have to remember that we have to meet the professor’s goals and what they want their students to learn. They may want to cover something more in line with course goals than what they instructors might want to do. Our brochures right now tell you what’s a good idea for how to write these articles, for example, ecology has a very step-by-step format, whereas the sociology article is very open ended.

Frank: Of course we want to meet them, but we have to wonder how many biology instructors, for example, might come in if they had a simple template method to follow.

Ryan: We can provide direction based on what the goals are — like changing the difficulty levels on a video game. For example, do people want to write about psycho-logy or a psycholo-gist?

Ian: We saw at the art and feminism articles that having a format and structure can remove some of the learning curve.

Frank: From the outreach program, and from art and feminism, we’ve been thinking a lot more about campaigns as a way to drive people into filling content gaps. Campaigns could be combined with what we know about templates and structures. We could say “Hey we’re doing this campaign!” and provide a simple and direct way to enter into that campaign.

Jami: We do this in history outreach, and maybe we can use this to test this idea. “Hey, here’s how you do a historical biography,” for example, to history instructors. Also, I’d like to see how many instructors are concerned about the writing of Wikipedia and not necessarily the content. If we’re directing instructors to a specific species, for example, how many classes have the flexibility to focus on the areas we’re looking to target?

Frank: Right, what are the hot topics, and can we channel that into Wikipedia assignments, for example?

LiAnna: I just wonder if the staff time required to find the very limited number of professors who are doing that would make it worthwhile.

Frank: How many instructors have heard about this? I’m guessing less than 1%. If we can reach more people, we can more easily find that type of instructor who is interested in teaching these topics, interested in these templates, etc. I want to say that, as Lorraine Harriton said this weekend, we talk a lot about things that don’t work, rather than thinking big…

Jami: Let’s table this to the end.

Frank: OK

Classroom Program: Student Work in the Humanities

Adam is presenting quality work related to humanities courses.

Slide 1

These are some courses doing good work in humanities, students seem to know what they are doing.

Some of these are falling nicely into the gender gap, we have some good classes on say Yiddish theater and black theater.

Here’s Sophie Treadwell, added to a bit today. There’s some good theory and history articles, for example “Sentimental Comedy” here does a good intersection of theory and music history.

Marriage a la façon du pays was a nice stub created by a student; the instructor had sort of added a seed paragraph and it was immediately marked for deletion. So even today as I went to see it, they’ve expanded it into a really long, well-cited article.

Ramping up

A few courses getting ready to post work TODAY, or posting work with really notable improvements even just since this morning.

So that’s a brief overview.

Classroom Program: Goals, Expectations and Learnings

Presented by Ryan McGrady.

Spring 2015 goals

I’ll do an update on numbers and where we stand. I am erring on the side of oversharing but will try to go quickly… here’s our goals for this term.

Spring 2015 numbers

We’re about 32 short; we had a lot of organic growth.

Potential classes

We have a lot of people we’ve reached out to who are interested, but unsure about where those will end up.

Enrolled Students

We have a much higher student training completion rate this term, which is good news.

Other news

Incidents

There are some incidents that have showed up on noticeboards.

Hyolophagia was our first conflict; Ian e-mailed instructor, was one of the most unfortunate outcomes where editors were comfortable, but instructor wants to withdraw from the assignment.

Some neuroscience articles showed up as problems when editors weren't sure if talk page comments were part of a class. Worked with instructor to clear things up and ended up having a good talk about the course assignment template.

Incidents 2

There was an incivility issue, in which a long-term editor was uncivil to the instructor’s students.

We had some confusion on a talk page — and instructor just kept using last year’s, I called and we moved it over and that was that.

Incidents 3

In some cases we had to do some detective work, finding professors doing rogue classes.

Partnerships

Presented by Jami Mathewson.

I will talk about what Ryan has indicated, about our partnerships being a point of failure for recruitment. I’ll get to that but first here’s a review of the past three months.

Partnerships Slide 1

• Thanks to the rest of the team. Hopefully that gives you a bit of insight into the current state of the Classroom Program.
• Reiteration of why we have structured the Classroom Program/Educational Partnerships QR like this.
• Classroom Program: less to report in the middle of the term
• Educational partnerships: better to report successes/failures and get feedback in the middle of the term, so I still have time to make adjustments and amend my strategy.

Slide 2

• What I’ve been up to over the last 3 months.

Formalizing Partnerships

One of my primary goals: to formalize partnerships. I’m going to talk about how that’s looked different than expected in a bit, but for now, let’s just highlight the partners we do have.

• NWSA
• APS
• MPSA
• LSU

What this looks like: Memorandums of Understanding. Use these to spell out each organization’s expectations and verify commitment on their part to promote our program(s).

Slide 4: Outreach

Spent a lot of time doing outreach to potential partners, setting up meetings to see if they’re a good fit. Planning visits or putting together materials to engage potential partners.

There are about 10 universities who seem interested in partnerships.

Slide 5: Support existing partners

• Supporting existing partners to maintain enthusiasm on an initiative.

I’ve also been pulled into recruitment this term, because partnerships have missed our intended goals. So, I want to think about the limits of what we’re doing and why we aren’t getting more classes out of these partnerships.

Slide 6: Failures

So I don’t want to pretend like I am not running into barriers and problems with educational partnerships. Instead of framing this presentation in the context of “what I’ve done” and “what I plan to do over the next 3 months,”

I’d like to use this time to present a few goals that I have not met, speculate on the challenges I’m running into, and open it up to the group for ideas about moving forward.

Slide 7

• University partnerships. So I mentioned earlier that our only university partner is currently LSU. I was aiming to have 2 by 12/31/15 and another 3 by 6/30/15.
• What’s the problem? I’ve been thinking about this a lot, and here’s what I think.

Frank: So, why are we doing these university partnerships?

Jami: Great question. And leads us directly into the next slide…

Original vision

First, we approached university partnerships as a way to give universities some flexibility and autonomy for supporting more courses using Wikipedia. The idea was that we’d help them get a program started on their campus, and they would support instructors, design assignments, advise students, etc. In short, they would function as mini versions of our Classroom Program, with the expertise living on campus instead of just with Ryan/Helaine or one of us.

Discussion

Jami: I will say that 5/6 of the good examples we’ve seen from Ian have come from LSU, so in terms of quality, I think partnerships have yielded real results. We want to engage with people who have real expertise in our program.

Bill: So there are people on campus who engage as advocates for us, or…

Jami: Sort of, more like they are like having a Ryan or Helaine on campus.

The challenges

Then, last term, we saw several examples of incidents in courses where we had been less involved and left the support mostly up to campus faculty.

So we found that these faculty weren’t self-sustaining as much as we’d like them to.

For one, I think faculty are less willing to say no to instructors — to say, hey, that may work for you, but it doesn’t work for Wikipedia. When someone isn’t saying that, it’s causing problems.

Learnings

• Faculty do want to participate! There’s energy and enthusiasm, but not always to do exactly what we want them to do.
• They must be flexible with instructors and with the assignment.
• Like our past Ambassadors, they don’t want to do the daily tasks that we need done: enroll student usernames, ensure instructors follow our milestone model.
• We are making so many changes (which are needed and helpful!), but this is confusing for anyone who isn’t entrenched in our daily work. It’s simply hard to keep on message with our tools and recommendations if we don’t have the direct line to instructors when there’s a middle man.

Discussion

Frank: It feels like something has changed here about the champions model. Which is OK. I understood that this was about having faculty recruit, but doesn’t have to train instructors.

LiAnna: That’s two different programs; our campus champions are different.

Jami: Yes, I think though that what’s happening is that with recruitment, people go to whoever recruited them. If that’s me, fine, it it’s a champion, they may not come to us first.

Frank: Maybe it’s a communication flow problem? Maybe these people are like the people who are campaigning from Obama 8 years from now because nobody told them he’s not running anymore?

Jami: I get that, I think though that we have 40 hours a week and can keep up on the changes, but…

LiAnna: Let’s look at Obama as an example, Frank. There’s paid staff who do in-person outreach, they get talking points on a clipboard who go out and talk to people, but it’s all coming through a primary point person.

Jami: But I’m not the person at McDonald’s who tells everyone what the specials are.

Frank: Right, but you know, there’s certain rules, you won’t see someone selling creme brûlée at McDonalds. There’s certain guidelines. Do we need paid staff in regions? Paid staff in topic areas? Do we need 50 people and offices in every state for this to work?

Jami: OK, let me get back to the slides:

Learnings Cont'd

• I get so much feedback that people are nervous because they don’t have the Wikipedia knowledge and expertise to intervene when there is an issue in a class. So we either need to get better at providing this support and training, or we need to continue playing that role.
• A few other thoughts: this can work! Think of LSU. But even they are sustainable but not growing very much.
• They still have a focus: science. It’s difficult for LSU to expand outside of that. So there are still limits in place in expanding, even when campus faculty do a great job of supporting courses and producing outstanding work on Wikipedia.

Discussion

Frank: What do you think is the most important learning here?

Jami: All five are equally important! I will say… we lack flexibility; we turned away about 30 classes because they were doing things we couldn’t support. It meant fewer incidents, but also fewer courses. Even very motivated champions were reluctant to sort of wrangle people into converting course pages, for example. Instructors don’t care they way we do about things like having every student get a username, for example. We need that, but they don’t see the importance.

Frank: So what would you change about the roles people are playing now?

LiAnna: We did do this big push on campus — we visited campuses, invested a significant amount of time and got a handful of classes.

Frank: But, what do you think are roles these champions should NOT play?

Jami: They should probably support students instead of instructors.

Frank: So they shouldn’t take on our role for setting expectations and standards?

Jami: Except that they’re in an advisory position at the school.

LiAnna: Imagine you’re an instructional technologist and you have a partnership with us, and then you have a directive from your boss saying that your primary goal is to support instructors in anything they want to do. So an instructor comes and wants to do 1,000 student classes editing medical information. We say no, no, no, but they’re obligated to pursue it. They can’t say no.

Jami: Let me get to our next slide to talk about this a bit more.

Moving forward

• Options for how we can adjust our work with university partners:
• Develop a formal training/certification
• Expensive. Time-consuming. Can’t roll out in the next few months. If we do go in this direction, we’ll need to re-think our numeric goals associated with university partners.
• Give more autonomy to faculty partners
• Let them develop a classroom program on their own/with our guidelines/materials
• Let them make mistakes, fail, and have incidents. They can expand to more classes (like large classes, potential conflicts, and others that we might otherwise say “no” to).
• Narrow focus? It’s difficult to try to recruit and train and manage such a large range of faculty members. So maybe we narrow our focus to libraries?
• I’m actually going to ACRL tomorrow so I can start thinking about this: do we restrict our university partnership growth to libraries and take over the existing library projects happening in the US/CA?

Discussion

Frank: So… LiAnna made the point pretty clear that the role those people play as support for universities is inherently in conflict. So, are librarians in a different role?

Jami: I think so. Our question is how can a library help with the program. They support research, etc. I’m going to ACRL tomorrow, so I can speak more about this.

Frank: So librarians wouldn’t have as direct a relationship with instructors as the current support structure?

LiAnna: Not in the same way. Not all librarians are supporting student research skills, many are helping instructors as their educational support, etc. Also, I think this has been a really hard relationship to have. We had a very hard time getting MOUs.

Jami: We get some really smart questions about quality, but we don’t have very many conversations about growth through campus relationships.

LiAnna: But if we think we’re getting higher quality articles (and we are, for example, because of Becky Carmichael at LSU) then that’s a question of models reflecting people, which can be great in isolated cases but isn’t scalable.

Bill: From a sort of outside viewpoint, I wonder if these solutions are going to work. If someone on campus is representing us - the McDonalds/Obama whatever model — and they’re paid staff of a university. That’s always a conflict. If we want to solve this, we need paid staff. Maybe regionally. Also we’re following content models as a trade-off. Do we want more classes and more incidences, or better content and fewer courses? Do we want more Adam/Ian/Ryan/Helaines?

Jami: And we still don’t have a model for measuring quality. But last term we finally had some support, but we are finding out what our limitations are. So… let’s get into the next slide actually…

Open questions

• Do we continue pursuing university partnerships? In other words, can we still support them? Is this worth the resources it takes to recruit and on-board them if we don’t get a ton of classes right away?
• Things to think about:
  • We have a lot of requests for campus visits. Do we leverage this better for agreements? Do we make our MoUs more specific/strict? Require one faculty member to attend a training? To get a certificate?
  • There seems to be a lot of potential. But maybe not enough focus on individual opportunities and groups to build some infrastructure.

What are opportunities for:

• Faculty Learning Communities
• Libraries
• CTLs (Centers for Teaching and Learning)
• Deans of colleges
• Centers for Women in Science

Academic associations

I did want to spend a little bit of time talking about our other major target for partnerships: academic associations.

Original vision for academic partnerships

Our original expectations of these partners were that they would play the role of recruiter, connecting their members to us to support.

Discussion

Frank: I know we are running over time at the moment, but I want to emphasize that this is an extremely important discussion. I would like for us to take the time to understand and think through what we are talking about today. This is a turning point for this organization. We are setting our long-term goals very soon. We have to decide what programs continue and what ends. I do not want to make lighthearted decisions. We can’t be rushing through this decision just because we have meetings today.

Frank: So lets say we want to expand our goals by 20-fold over the next two years. I’d really like to understand what is possible and not commit to something that can’t happen.

LiAnna: We need a technical budget to expand to expand the program. Right now we need that funding before we can expand.

Bill: I am with Frank here. We need to know how, if what we have in place isn’t meeting goals, how can we grow? If we need tools, this is the place where we find that out.

LiAnna: It’s worth pointing out that recruitment didn’t start in earnest for this term until mid-February.

Jami: We set all of our goals, and we’ve slid off of that timeline for a variety of reasons. Which is why our strategy here is a long-term growth one.

Frank: My challenge is more fundamental than that. I know the history, but this isn’t the classroom program organization. If the classroom program isn’t getting to our goal, we need to find other ways…

LiAnna: I disagree completely. If we had the budget right now for tech projects, we’d reach those goals in two months.

Frank: But our approach was wrong. It’s not just support, it’s outreach that isn’t scaling…

LiAnna: I disagree. We didn’t do active outreach. We relied on organic growth, which we can’t do. We have doubled the number of classes in a month….

Frank: But it’s not scaling. I’m talking about what we need to bring in 1,000 to 10,000 new instructors. I know everyone has made sacrifices here, I’m trying to do, without being mean, is to challenge everyone and ask: What would it take to bring in 1 million instructors? I know that aspirational, but…

LiAnna: That’s not even aspirational, there’s not a million instructors in the US who would do assignments that fit what we need.

Frank: What has been done to move the numbers up has been exceptional. I am seeing and acknowledging that. And it worked because you are all strong advocates. If we had 20 of each of you it would be a model. But we can’t have that. But what we have done is not scalable.

LiAnna: But our partnerships are not a short-term solution. They’re long term. We turned down psychology classes — maybe 15 — but when the ball gets rolling with these associations, and as our technical infrastructure improves, we can start scaling. I believe the classroom program can scale. I think the outreach and recruitment can happen. Wikipedia is changing from a crazy tool to one that people understand intuitively now.

Frank: The short-term model of recruitment we have done is unscalable. That’s all I’m saying. I would like to suggest we start expanding into a scalable solution to meet these outreach goals.

Jami: My perspective on academic associations is that they are not worth giving up on yet. We may have assumed that the end point was getting those MOUs. That’s not our real end point. What happens is once that happens, energized instructors start discussing it, sharing ideas, talking at conferences.

LiAnna: We could hire staff I guess; but we don’t even have the capacity to expand very far beyond where we are. I do believe it scales, I do think it’s worth doing. We rely on organic growth. We haven’t done any motion to bringing people in. More staff time doesn’t increase the speed of which people come on board. We can’t look at the academic partnerships after three months.

Jami: We did try to do some outreach, get things in journals, meet their needs. But we see the trickle-down effects.

LiAnna: We’ve seen that things do scale up over time, even without staff attention. University partnerships may not work the same way.

Jami: When we have better ideas of getting into universities, it might be different — libraries or archives, or whatever.

LiAnna: We have never tried having someone on campus who is accountable to us and who is responsible for lining up with our expectations.

Eryk: I want to weigh in on recruitment and outreach. A lot of the stuff I did was outside of the box- social media, mailing lists, lots of different ways. I think we’re going to wake up in fall of 2015 with more than we can deal with. Because

Jami: there’s been a lot of people saying they want to do it later 

Eryk: Also, some partnerships came in at a point where it really doesn’t make sense to expect results yet. No one wants to stray from their current syllabus a week before they start the course. What we’ll see is that people will convert after this class, or the next class. The change won't happen on short notice.

LiAnna: We came into recruitment having worked with innovators who WOULD throw out a syllabus to pursue this avenue. And those people were the low-hanging fruit, the instructors who would throw out a whole syllabus on a week's notice. But we have gotten to all of those innovators. The next phase is people who like the idea but what to make sure they have real plans and want time to think this through.

Jami: And I think many partnership-lead courses have different motivations. They want to make an impact on Wikipedia.

LiAnna: This is different than the Public Policy Initiative. The best professors came on early and fast but they were the innovators and would throw out their current syllabus to get going. We had a rosy view because of these outliers.

Eryk: This is the first time we’ve done cold calls, and we’ve seen the results turn from zero to 60. Those results haven’t been discussed here in terms of outreach learnings.

Jami: We did follow ups with the Washington folks and they want to do follow up in the fall... We need to get started sooner because our onboarding involves outreach now. We could start recruiting now for the fall, but the big thing is we lack the course page system. We can’t act on this learning right now due to the tools.   LiAnna: Will the beta version be ready by Wikimania?   Jami: We don’t want to onboard people and then need to change it. To be fair, this happens every term.   Eryk: The timing has also been based on the innovator model and does not accommodate the hesitant professor. We need more lead time if we want to get the more cautious instructors to come on.

Jami: It accommodates the returning professor, and not the new one. We need to be out there in May.

Bill: Plus don't some syllabus' need to be approved farther in advance?

Challenges

It's been slow. We expected way more classes from these partners for the spring term. So what’s happening?

Learning

Long-term. My primary speculation is that these partnerships have very long-term goals. We put a lot more work in the beginning to get an initiative rolling, and we hope the initial enthusiasm will turn into classes right away. It takes several people participating to get the ball rolling.

Think of APS. Even though we don’t want many more psych classes to join right now, they constantly do. Because these instructors know each other, are at the same conferences, are following the same blogs, and are reading the same papers. Once several people participate, like in psychology, it slowly becomes a part of the classroom fabric, and more people actually consider Wikipedia assignments as an option.

Academic associations still have so much potential for closing content gaps. I think if we figure out a good way to create lists of content gaps—even if by engaging the association itself—we can recruit classes this way and give better direction to potential courses.

Top-down helps, like with COSSA.

Not ready to give up on these!

Closing / Other Thoughts

Jami: Do you guys have any other thoughts on academic associations? Any ideas about how to improve their efficacy? How to expand their initiatives or reach more members? Or do you think we should abandon them? Open to hear what you think!

Frank: I’d like to see some analysis on how people came in, and how that worked, if it was cold e-mails, mailing lists, visits, etc. I also wonder if we need to just think about the classroom program as one way to do this.

Frank: Also, I think the Quarterly Review needs to be open to longer, in-depth discussion. We need to reimagine the way we think about these quarterly reviews.

LiAnna: Though some may not need a more elaborate conversation. Communications and Digital Infrastructure are support roles, so maybe we don’t need as much time for them.