Grants talk:Project/Learning with Wikipedia Based Assignments

Eligibility confirmed, round 1 2017

This Project Grants proposal is under review!

We've confirmed your proposal is eligible for round 1 2017 review. Please feel free to ask questions and make changes to this proposal as discussions continue during the community comments period, through the end of 4 April 2017.

The committee's formal review for round 1 2017 begins on 5 April 2017, and grants will be announced 19 May. See the schedule for more details.

Questions? Contact us.

--Marti (WMF) (talk) 19:53, 27 March 2017 (UTC)

Questions from NickK

• • Hi NickK - I'll answer the questions in line here:
• Who are the survey participants? You are speaking about 1,600 students, are they already involved in Wikipedia Education Programme, are they potential students of teachers already involved in the programme or are they still to be recruited?
  • A few things here:
  • The 1600 participants were from Fall 2016, we expect to have many more participants in future surveys. They are students involved in courses supported by the Wiki Education Foundation, not the Wikipedia Education Programme. This gave us more access to recruit and an embedded survey system.
  • Future survey participants will be recruited from participants in courses served by Wiki Education during the semester / quarter (one in Fall and another in Spring). We recruit by sending emails to both the students and instructors as well as offering an incentive (entry into a drawing for a gift card). Students who volunteer for interviews in Fall will be scheduled for interviews in Spring.
    • @Zach (Wiki Ed): Do I understand correctly that courses are already confirmed by Wiki Education Foundation and that not having enough courses is not a risk? — NickK (talk) 22:50, 12 April 2017 (UTC)
      • @NickK: No, they're not confirmed and there is really no way to confirm them given the way higher ed courses operate. There is always a risk in having a low respondent rate but given the success of the previous research I believe the risk is extremely low. Wiki Ed's courses have been growing at a steady rate since their inception, Spring 2017 currently has 351 courses and 6688 participants, while Fall 2016 had 276, Spring 2016 had 215, and Fall 2015 had 162 courses with only 3803 participants. The data is all available on the dashboard. Given the trend for growth I believe our target for survey participation are very conservative. Zach (Wiki Ed) (talk) 12:30, 13 April 2017 (UTC)
• You are stating that this project is yet to be approved by the University Human Subjects Review Board. What is the probability it will not be approved, and if it happens, do you have any alternative plan?
  • My previous study was approved in under two weeks by the review board and this research is based on the previous research. This research is categorized as extremely low risk and slated for expedited review. I feel 100% confident I can navigate any issues and get full approval for the study as well as the public data release. Approval may be subject to changing some questions but would not be a problem for the study itself.
• What will be the basis of these two publications and four presentations? Will they be based on the study you made last autumn, or on the one funded from this grant?
  • Both. Academic research takes time to complete and disseminate, so any academic publications written from this data probably won't even make it past peer review until after Summer 2018 (although the intent is to write from Fall 2017 data). I'm currently drafting publications from the previous research as well as proposing presentations from the previous research. Once data is collected from the new research we will write on that data, but that data will not be ready until after Fall 2017's semester.
    • The idea here is to begin to generate a stream of data and publications on teaching with and student use of Wikipedia, which I would not only conduct but also write (and assist others with) on, present on (and assist others with presenting on), and disseminate the data for others to use. This has a bit of a long tail on it, but I expect Fall 2017's data to be used for years, and the more times we repeat the survey and interviews over time the more we can look at trends and overall student use of and experience with Wikipedia. It has incredible potential value not only for educators and educational research but also for those researching Wikipedia in general.

And not that much a question, but just out of curiosity: is it possible to read the Fall 2016 Pilot Study you mentioned in the proposal?

• Absolutely - can you email me at zach@wikiedu.org I can send you whatever you want. Currently I am assembling all the data and an executive summary so it is a bit messy and I'd rather post the public data when it is in the best shape possible for a wide variety of researchers to engage with. You can read a brief overview of the research here, which I wrote in the early months of the study.
  • OK — I can wait until it's ready than, that's not time-sensitive. Thank you! — NickK (talk) 22:50, 12 April 2017 (UTC)

Thanks in advance for your answers! — NickK (talk) 16:12, 12 April 2017 (UTC)

• If you have any other questions I'm happy to answer them! Zach (Wiki Ed) (talk) 17:31, 12 April 2017 (UTC)

Comments from Ruslik0

This is an interesting research project but I have a few questions/comments:

1. You mentioned "longitudinal research". Does this mean that you are going to survey the same students as in the Fall 2016 Pilot Study?

• • Sorry that is a bit confusing - its only sort of longitudinal because the shared cohort would be students who use Wikipedia assignments. However, we would collect interview volunteers and follow up with them afterwards for qualitative data as well. I made slight changes to the wording to clarify as this is more about repeating the same study with as many students as possible given the heterogeneity of hundreds of courses across North America.

1. As I understand this is a purely online survey?

• • The survey portion of it would be, yes. They would be given the opportunity to volunteer for interviews.

1. Can you name one or two journals where you want to publish your research?

• • My personal preference is for platinum OA journals. Since I have been working with others and taking second author, we're currently looking at publishing in Computer & Composition (securing funding for OA article processing charges now). For this research I plan on submitting to First Monday, and most likely the International Journal for the Scholarship of Teaching and Learning. Additionally I would look into Communication Education (NCA journal) and depending on some of the outcomes we would look into New Media and Society or Social Media & Society. This would be up to co-authors as diversifying the potential readership is ideal here.

1. What standards do you use when designing and performing surveys? I mean standards of best practices.

• I'm not exactly sure what you mean - my surveys are designed collaboratively with multiple researchers trying to address a variety of research questions. We are all professional researchers with years of research experience. We follow strict human subjects guidelines (I am CITI certified). We follow typical social science "best practices" (which these folks cover fairly well here, except we use a homegrown survey tool, not qualtrics). Hope that answers your question!

1. You mentioned "participation in a variety of Wikipedia communities" in the community engagement section. Can you name these communities?

• Sorry for the phrasing, I see how that was confusing - I meant both Wiki Ed and I personally have participated in different parts of the Wikipedia community for years and will continue to. Personally I am a member of the listservs and FB groups mentioned in "community notification," as well as I have presented at WikiConference North America and hopefully at Wikimania this year. It is my hope to engage with the Wikipedia Education Collaborative in redesigning this research, as well as engage more Wikipedia researchers to help answer questions about student participation in Wikipedia alongside larger questions of education outcomes. I'll modify my response there to clarify.

1. Do a "Wikipedia-based assignment" that you give to students involve actual Wikipedia content creation? If so you should probably fill-in "Do you have any goals around participation or content?" section.

• I do not give students the assignment - these are students who are already participating in a course with a Wikipedia assignment - I am not their instructor. This grant was written for the research portion, not for content creation. Wiki Education has some fairly impressive numbers for participation and content but this research would not affect student participation. The hope is that others can implement this research to better reach participation or content creation goals, but this is only the first step here.

Thank you. Ruslik (talk) 17:16, 12 April 2017 (UTC)

• You're welcome! Zach (Wiki Ed) (talk) 18:27, 12 April 2017 (UTC)

Round 1 2017 decision

This project has not been selected for a Project Grant at this time.

We love that you took the chance to creatively improve the Wikimedia movement. The committee has reviewed this proposal and not recommended it for funding. This was a very competitive round with many good ideas, not all of which could be funded in spite of many merits. We appreciate your participation, and we hope you'll continue to stay engaged in the Wikimedia context.

Next steps: Applicants whose proposals are declined are welcome to consider resubmitting your application again in the future. You are welcome to request a consultation with staff to review any concerns with your proposal that contributed to a decline decision, and help you determine whether resubmission makes sense for your proposal.

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.

Aggregated feedback from the committee for Learning with Wikipedia Based Assignments

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	4.4
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	4.1
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	5.7
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	4.9
Additional comments from the Committee: Education is a priority of the Wikimedia movement but there are not obvious outcomes from the proposed research besides improved understanding of learning and teaching with Wikipedia. There is no strategy for taking action that would be implemented and therefore no direct impact for Wikimedia projects. Because the previous research conducted (Fall 2016 study) has not been reported on it's also unclear to me what the impact of additional research would be or why it is needed. The impact is low compared with the budget Yes, it fits with Wikimedia's strategic priorities and probably will have an online impact. Education Programme is implemented in many countries all over the world, but we are lacking scientific research on its effectiveness. This project seems to be very impactful from this point of view and its learnings are likely to be useful for other countries as well. Measures of success relate only to collection of data and conducting research, not implementing action based on findings or improving education programs. The approach is iterative. The risks appear to be small. The success can be reliably measured. I don't understand the ultimate aim: increase retention rate, increase number of active editors, etc. What is the aim? I have already read results of other students' research on Meta where the main answer for low retention rate or level of editing is lack of time. The results of the survey showed that we need to increase priority of Wikimedia involvement, so, the problem is already identified. Do we just perform one more research here? Decent iterative project (survey) + research behind. I'm not sure this is the right time to be undertaking new research seeing as the researcher is still completing previous research (continuing with analysis; preparing papers and presentations) and ethics approval has not yet been granted. I have no doubt of their ability to execute the project, nor of their skill and experience. Overall this looks like a solid scientific study being done by experienced researchers. Too expensive specialists with lack of details. Grantees do not have significant history of editing, lack wiki skills and experience. It is realistic with participants having necessary skills; budget may be a bit on the high side. Target community is student editors, although it's unclear how they will be recruited and if they will be representative of student editor demographics. Project does not support diversity. There is a significant community interest. Seems to be endorsed by others, has target community. Well-defined target community with support from many folks working on education initiatives. Does not seem the right time for this research (especially given how competitive this round of funding is) and a strong enough case has not been made for the need for this research. I would also like to see the proposed research linked to more concrete action on how to improve the teaching and editing experience. I will recommend to fund this project as it seems to be a solid study/survey being done by experienced researchers, and the results may be interesting and informative. They can be used in the future to improve design of Wikipedia based educational problems. The ultimate goal needs to be revised and the intersection of the scope of this research should be checked against prior researches. The funding also seems excessive. I see no use for Wiki projects of this kind of research. I would support full finding as I think the entire project deserves it, but still would like the grantee to look into ways to reduce costs.