Grants talk:Project/Research Grant/A study on analysis of leadership wrt Gender and its impact on projects, individual and community growth in India

Questions from Joalpe

Thank you for the proposal. The study of event stewardship is indeed fascinating. I have some questions (being asked as I am wearing my hat as a member of the project grants committee):

1. Given the large budget, I would have expected a better preliminary research and more thorough literature review. Could you please provide a more compelling account on your hypothesis on the impact of leadership on women's participation on Wikipedia?
2. Could you provide a rationale for aggregating being an event organizer/facilitator/steward with being a movement leader? What is the theory of leadership you are relying on?
3. How reliable are the data source? What methodological pitfalls do you envision? You are expecting to deal with diverse sources ("dashboards of various activities, reports, etc."), but aren't you already sampling on successfully reported cases, thus creating an endogenous bias? What are variables from these events are you planning to collect, and why do they matter? And why the ones you do not plan to collect do not matter?
4. How will you identify male and female editors in the reports? What is the expected rate of unknown response? How will it be dealt with?
5. Could you please specify on the budget why you need items on line 8 to 12?
6. Why do you need two investigators for this work? What are the different roles? Do investigators have prior scientific publications on the gender gap in the movement?

I hope my questions make sense. Thank you. --Joalpe (talk) 03:39, 1 April 2021 (UTC)

Response to the Questions

Hi ‎Joalpe, Thank you so much for your valuable comments. Following are our answers to each question raised. We apologize for the delay on our part.

1. The suggestions in the grant template asked not to mention the details and to keep it intact to the basics, that's why the review wasn't shared in detail. According to a 2018 report, there is a clear disparity among the male and female program organizers (65% vs 35%) and affiliate organizers (74% vs 26%) (https://meta.wikimedia.org/wiki/Gender_gap/Knowledge#/media/File:Gender_of_Community_Audiences_for_2018.png). Community insights, 2019 (https://wikimediafoundation.org/our-work/addressing-wikipedias-gender-gap/facts/) tells us that 25.6% of movement organizers are women while 64% affiliate members are women. In a 2020 study, it was observed that gender representation is appalling within the organizational structures of governance and influence. 74% of the chapter and thematic organization board members are men, whereas 80% of user group board members are men. If we observe formal representatives of affiliate groups, 65% of chapter/ thematic organization and 74% of user group primary contacts are men. However, the problem of women’s low participation in editing Wikipedia is primarily impacted by the limited women leadership. A lack of female leads has also been identified as a reason for low women engagement in a report on Gender studies wrt to India by Bhuwana in 2020 (https://cis-india.org/a2k/blogs/bridging-the-gender-gap-in-indian-language-wikimedia-communities). Considering this aspect, it is fair to believe that a higher number of women leaders are expected to lead more women-centric initiatives and are also expected to attract more women to come forward to edit. Along with this, there have been reports on poor retention and growth of Women and women-led initiatives which has not been addressed in any of the previous studies on gender gaps. The aspects like the impact of the difference in leadership on the outcome, engagement, and growth of the project, have been ignored, therefore, this study has been proposed to derive a clearer view on the impact of gender gaps in leadership on projects with special reference to Indian communities.

2. Following the various studies we reviewed and also considering the undocumented aspects of leadership and its impact on project growth, we framed the proposed project. We reviewed leadership from the Learning and Leadership development report (https://meta.wikimedia.org/wiki/Movement_Learning_and_Leadership_Development_Project/What_we_learned#Leadership) as well and are going to consider the community organizers/facilitators.grantees/leads as “leaders” in this specific project. We will analyze the acquired data on project outcome, growth, and engagement for different activities which will be categorized on the basis of the leads and gender. The intention is to do a comparative analysis of the difference (if any) in the growth or engagement of programs led by females wrt males.

3. Methodological pitfalls might arise if the data reported is false or misreported. Since we are taking the data directly from the Indian Wikimedia communities, the chances of obtaining erroneous data are significantly less. Even so, we will be thoroughly checking the data obtained for any inconsistencies in their reporting to minimize any bias or errors. Some communities might be unwilling to participate in this study. Although the study is not binding to any community in any way, we will try our best to persuade more and more communities to participate in this exercise to get better and more reliable results. Another cause of bias might be the personal preferences of the investigators. We have a female and a male investigator in the team to minimize this bias. All the data and results will be cross-checked by both the investigators to reduce these types of biases.

It is true that we will be dealing with diverse sources including dashboards, reports, etc., but even the events which have significantly less outcome are reported. In that sense, we are not just including the successful case but also those cases where the outcome was less than the expected impact or outcome.

There are four most important variables we will be studying, which are as follows - a) Number of events organized by male and female leaders respectively in a particular community, b) growth of initiatives, c) Participation, d) editor retention. These variables play an important role when it comes to resource allocation and volunteer participation for Wikimedia. There are other aspects like the outcome, resources, support, etc. which are important to be reviewed as well but they have not been considered for the proposed study as this will make the research too extensive and hard to execute in a year.

Certain variables such as the age of leaders, their time period at the position, as well as various factors related to volunteers participating in the events are the additional factors we will not be considered for the present study. These factors might play a significant role in understanding the group dynamics of Indian Wikimedia communities, however, no previous studies regarding this have been conducted for any group so gender-based differences are a good starting point. These factors can be studied in future studies.

4. Our aim is to study the gender gaps in leadership for programs designed and led by Indian Wikimedia communities. We are not dealing with the gender gap on the editor level, therefore, this information will not be required.

5. At the end of our project, we are aiming to present the key findings of the documentary as well as the important responses of our interviews in form of a documentary. This documentary will (Budget Head 10) be essential in explaining the study as well as the results not only to other Wikimedians but will also help in spreading awareness regarding the gender gap. Some Open Access Journals ask for publication fees (Budget head 11) and we have also kept the budget (9& 12) for the interviews with the community organizers. Graphic Designer services will be taken for printed versions of reports and for larger community use for awareness and engagement. We can consider removing the ones found irrelevant.

6. There are two investigators in this project – a) Nitesh Gill, and 2) Dr. Praveen Kumar Yadav. Nitesh has been an exceptional and active Wikimedian with more than 1500 articles focussed on Women. She will be tasked with the responsibility of communicating with different Indian Wikimedia communities and gathering the data, providing insights from her experience and also information about projects organized by various communities. Additionally, she will also have a lead role in interviewing the female participants. On the other hand, Dr. Praveen is leading the study design & methodology, will be dealing with analyzing, and interpreting the collected data. Also, his role will include writing and publishing the results of the present study. Although the investigators do not have any prior publication in regards to gender gaps, both investigators are suitable for this project considering their research experience of Masters of Philosophy (Nitesh) and Doctorate of Philosophy (Praveen). Nitesh has been a leading figure in the gender gap movement in Indian Wikimedia communities, which makes her knowledgeable about the community dynamics as well as the problems faced by the women wikimedians in India. Dr. Praveen has been associated with research for the past six years and has published more than 25 research papers in various national and international journals. It’s a very good opportunity for both of us to use our respective expertise to explore this very crucial topic.

Thanks,

Praveenky1589 (talk) 19:59, 5 April 2021 (UTC)

@Praveenky1589 and Nitesh Gill:, regarding the last point -

6a). Can you please provide few evidences to strengthen your argument about Nitesh being a leading figure in gender gap movement.

6b). Can you please provide few evidences to strengthen your point about you being in the research related to social sciences. I can see here and here, that your field of research work is Forensic Science. In that case, considering your completely different background, will you consider someone from social sciences with prior experiences on gender gap research as your Project Manager? -- Bodhisattwa (talk) 13:18, 20 April 2021 (UTC)

Hi User: Bodhisattwa, these questions have been addressed in the section below. Thank you Nitesh Gill (talk) 21:14, 20 April 2021 (UTC)

Eligibility provisionally confirmed, Round 2 2021 - Research and Software proposal

This Project Grants proposal is under review!

We've provisionally confirmed your proposal is eligible for review in Round 2 2021 for Research and Software projects, contingent upon:

• confirmation that the project will not depend on staff from the Wikimedia Foundation for code review, integration or other technical support during or after the project, unless those staff are part of the Project Team.
• compliance with our COVID-19 guidelines.

Schedule delay

Please note that due to unexpected delays in the review process, committee scoring will take place from April 17 through May 2, instead of April 9-24, as originally planned.

• Please watch your talkpage, which will be the primary method of communication about your proposal. We appreciate your timely response to questions and comments posted there.
• Please refrain from making changes to your proposal during the scoring period, so that all committee members score the same version of your proposal.
• After the scoring period ends, you are welcome to make further changes to your proposal in response to committee comments.

COVID-19 planning for travel and/or offline events

Proposals that include travel and/or offline events must ensure that all of the following are true:

• You must review and can comply with the guidelines linked above.
• If necessary because of COVID-19 safety risks, you must be able to complete the core components of your proposed work plan _without_ offline events or travel.
• You must be able to postpone any planned offline events or travel until the Wikimedia Foundation’s guidelines allow for them, without significant harm to the goals of your project.
• You must include a COVID-19 planning section in your activities plan. In this section, you should provide a brief summary of how your project plan will meet COVID-19 guidelines, and how it would impact your project if travel and offline events prove unfeasible throughout the entire life of your project.

Community engagement

We encourage you to make sure that stakeholders, volunteers, and/or communities impacted by your proposed project are aware of your proposal and invite them to give feedback on your talkpage. This is a great way to make sure that you are meeting the needs of the people you plan to work with and it can help you improve your project.

• If you are applying for funds in a region where there is a Wikimedia Affiliate working, we encourage you to let them know about your project, too.
• If you are a Wikimedia Affiliate applying for a Project Grant: A special reminder that our guidelines and criteria require you to announce your Project Grant requests on your official user group page on Meta and a local language forum that is recognized by your group, to allow adequate space for objections and support to be voiced).

We look forward to engaging with you in this Round!

Questions? Contact us at projectgrants wikimedia  · org.

Marti (WMF) (talk) 05:34, 17 April 2021 (UTC)

Questions regarding financial expenses from Bodhisattwa

Hi @Praveenky1589 and Nitesh Gill:, I have questions regarding the financial part of the proposal

1. Can you please explain the reason behind the allotment of more than 75% of the entire budget grand total to the salary/remuneration of the Project Manager and Co-project Investigator?
2. Isn't it too much compared to Indian standards to allot the huge amount of ~1 million INR as salary/remuneration to collect data mainly by interviewing only?
3. Considering the completely different research background in Forensic Sciences and no prior experience to gender gap research, can you please explain the huge amount of ~1 million INR which you have alloted to yourself, being a completely newbie in social science research?
4. The travel and accommodation part is not understandable to me. Where and how will it be required?
5. Have you both done a pilot project before regarding this topic? If yes, please share the links. If no, please explain, why have you not considered doing a pilot before taking this huge grant project.
6. I have observed @Nitesh (CIS-A2K): is also a full time employee at CIS-A2K committing 40 hours per week. How will Nitesh manage 20 hours extra for this research? I would like to know the perspective of @Tito (CIS-A2K): as a program manager of the organization regarding this question.

Thanks in advance.. -- Bodhisattwa (talk) 13:17, 20 April 2021 (UTC)

User:Bodhisattwa, Thank You for showing your interest in our proposed project. Considering the specification that you have questions about Finances, I would like to believe that there were no concerns related to the proposal. For clarity, I am adding pointwise responses to your questions-

• As the proposed research is a data-driven study and does not require the use of any other tools, equipment, or space, and is extensively dependent on our intellectual abilities, that’s why there is no other expense except the remuneration.
• There are multiple dimensions to the observations here. We strongly believe that the remunerations specified in the budget are just and reasonable. The supporting facts are listed below- and the supporting arguments are as follows –
  • If you would review the proposal, it isn’t merely based on interviewing individuals. We intend to make the study substantial by supporting our findings with data & statistics. In the proposed study, we will be reviewing the available/shared data, conduct a meta-review (A meta-analysis is a statistical analysis that combines the results of multiple studies) derive inferences based on the observations, share recommendations based on the inferences. The interviews will be conducted to discuss the viability of proposed recommendations.
  • In the context of financial support to research standards, especially with reference to the Indian context, there are no limits or specified guidelines for remuneration/salary available in the case of research projects designed and proposed by organizations & individuals. However, you can review the same from various sources online.
  • Value of intellectual property is defined not by opinions but by the future impact of research on the society and research community. Considering the impact of the proposed research on our understanding of Indian Wikimedian communities, we believe the proposal is valid in the proposed terms.
• The principles of research methodology are the same across the different fields. Moreover, there are no prescribed guidelines in India against undertaking cross-discipline research projects. Additionally, forensic science is a sub-discipline of social sciences and the coursework for both is the same across India. Ergo, it is not accurate to dismiss the experience of the project managers in the field of research. We have the support and guidance of Professors and social activists for guidance in refining the project.
• Last but not least, we would like to point out the expertise of the Project Manager and Co-Project Investigator. Dr. Praveen Kumar Yadav has completed his doctorate, published more than 25 research articles in various national and international journals with good impact factors. With a citation index of around 86, an I-10 index of 3, and an h-index of 5, his capabilities of the undertaking and completing a research project are well established. Additionally, Nitesh Gill has completed his M.Phil and is pursuing her doctorate. She played a major role in bridging the gender gap on Punjabi Wikipedia. I hope this clarifies your disbelief of us being newbies in R&D.
• An integral part of research is to communicate it to various related groups (institutions, organizations, and research community). This is often done through research papers as well as events, conferences, and workshops. We will be presenting our research work at suitable conferences, events, etc. The travel and accommodation funds will be used for these purposes.
• We agree with your suggestion that a pilot study must be undertaken before applying for such a large-scale project. We would like to bring to your notice that we previously applied for a rapid grant for a pilot study before submitting this project. However, we were advised to apply for the project grant instead.
• All necessary information was reviewed, concerned individuals were informed and clarifications were gathered in advance, before submitting the proposal.

Thanks --Praveenky1589 (talk) 21:06, 20 April 2021 (UTC)

Hi @Praveenky1589 and Nitesh Gill:, thank you for your prompt reply. I never said that there are no concerns related to the proposal but I decided to focus on the financial part first.

1. For a proposal where actual research expense covering only ~3% of the total grant only and covering around 1000-1500 USD for the entire year, I am still not convinced why you would need so much money to remunerate yourselves.
2. I am still not convinced for ~1 million INR allotted for data collection personnel.
3. I am still not convinced for ~1 million INR allotted for project manager considering the meager amount of spending for actual research. Why are there even two persons required to do the data collection and analysis? Why can't there only one person do it?
   1. Your research on Forensic Science was scientific-tool based study and not a social study. This one is different. Anyway, if you insist, I would like to believe that this research will be properly handled by experienced researchers like you. However, I am still convinced, that a researcher with social science background with experience in gender gap study can do the job better. Have you considered to hire someone like that?
4. The accommodation and travel part is still unanswered. How and where will be that money spent?
5. Thank you for sharing about the pilot study proposal under the rapid grants. I guess, it is this one then. In the rapid grant proposal, you had allotted Rs. 90,000 for yourself for a study of 4 months i.e. Rs. 22,5000 per month. In the Project Grant you have sky-rocketed to around Rs. ~100,00072,000 for one person per month and there are now two persons. Please state your reasons to explain this jump. I guess, you could have done it easily for Rs. 22,500 per month single-handedly as proposed in your Rapid Grant proposal and the extra amount of remuneration for two persons are unnecessary.
6. Thanks for letting know that the the concerned employer was notified and he has cleared Nitesh. I wonder, what he has to say regarding the extra hours needed for the research. - Bodhisattwa (talk) 03:02, 21 April 2021 (UTC)

P.S. I was also wondering why you have mentioned yourself as third person in your reply like.... "Dr. Praveen Kumar Yadav has completed his doctorate, published more than....". It feels like someone else has drafted the reply on your behalf. Anyway, this is just an observation. :-) -- Bodhisattwa (talk) 05:08, 21 April 2021 (UTC)

User:Bodhisattwa, Considering the questions you have raised, there is only one identified concern which is about the finances of the proposal. We look forward to your observations on the methodology.

• With all due respect, our budget for remuneration is based on the review of previously granted projects and self-assessment of our abilities to design and execute an initiative. You can have different opinions, based on your expertise and understanding. We can agree to disagree.
• As we specified in our previous communication, we aren’t merely collecting the data.
• There have been some approved projects from the region in the past that had remunerations at a much higher level than compared to ours- Please see these - 1 , 2. Also, in the same cycle, there is another proposal with double remuneration- 3

So, it’s quite evident that this budget isn’t alien or unacceptable to Indian Wikimedia projects/proposals standards.

• We addressed questions about travel, which probably you missed. We would like to highlight again that- An integral part of research is to communicate it to various related groups (institutions, organizations, and research communities). This is often done through research papers as well as events, conferences, and workshops. We will be presenting our research work at suitable conferences, events, etc. The travel and accommodation funds will be used for these purposes.
• Yes, you have identified the proposal that we mentioned but you overlooked the proposal details. The hours haven’t been specified, the scope was smaller, the parameters were less and the work wasn’t as extensive as it is in this proposal. So, there is no substantial ground for setting equivalence here. A small correction, the remuneration hasn’t skyrocketed (which we cannot prove as there were no details in the previous pilot proposal). Following the current rate and considering the hypothetical future inflation, the remuneration calculates to be 72K INR which can be rounded off to 75K max but not 1 lakh INR for an individual. It will be 1.5 lakh maximum and that too for both.

There are two reasons why we have referred to ourselves in Third-party- It keeps the information neutral as self-praise is usually not recommended, and we do not want to associate individual ownership to the shared responses. We both discuss and share responses as a team. --Praveenky1589 (talk) 19:15, 21 April 2021 (UTC) --Nitesh Gill (talk) 19:16, 21 April 2021 (UTC)

Hi @Praveenky1589 and Nitesh Gill:, thanks for your reply.

1. Yes, you have every right to agree to disagree with me, and that is absolutely fine.
2. Yes, you mentioned earlier that the person will review existing resources too in addition to interviews and collecting data. I have noted that earlier, sorry that I have not mentioned that before but I was not and am still convinced that INR 1 million is way too much amount for this kind of job. You can totally disagree with me. That's fine.
3. I am still not convinced about the expenditure of remuneration:actual reasearch ratio should be that high. 75% is way too much. You can disagree with me. That's totally fine. But, you have not answered
   1. why a social science researcher having experience of gender gap study can not be hired as a Project Manager instead of you who is a Forensic scientist by profession.
   2. why the research cannot be handled by one person. Why one person cannot collect the data, review existing resources, meta review and analyze the data? I don't think the sample size is that much large that it cannot be handled by one person.
4. Sorry, I missed the travel and accommodation part earlier. Thanks for pointing that out.
5. Thanks for correcting the monthly remuneration. I mistakenly over-inflated, I am really sorry for that. You are correct, Its 72,000 INR per month as of now. I have corrected that in my earlier comments. Thank you also, for explaining the difference with the Rapid Grants proposal.
6. I hope, @Tito (CIS-A2K): has noticed the concerns regarding extra 20 hours needed for research for Nitesh. If he has not noticed already, I am pinging again.

P.S. - 1) Whataboutery should not be an way of argument. The grants you mentioned were different and this one is different. They cannot be compared altogether. It is not logical to compare a research grant with a capacity building grant or software development grant anyway. This one is not even an approved grant. There are other grant proposals and approved grants where remunerations has been set far less compared to these. I hope you have noticed that too. Have you considered them? The point is, all grant proposals have different perspectives and cannot be a justification for why you have set your remuneration so high.

P.S. - 2) It is neither a standard practice on-wiki to sign and mention yourself as third person nor a standard practice to show ownership to shared responses like you did earlier or you and Nitesh did now. Also, self-praise can still be done even with a third-person perspective, you have signed at the last, so that is considered as your comments. Anyway, this is just a trivial matter and no need to divert your focus further on this. -- Bodhisattwa (talk) 01:16, 22 April 2021 (UTC)

• Hello User:Bodhisattwa, yes, we noted it. For everyone in the team, while renewing a contract (generally July), the organisation notes and discusses the time availability as well. Thanks. --Tito (CIS-A2K) (talk) 16:28, 22 April 2021 (UTC)

Hello @Tito (CIS-A2K):, thanks for clarification on point #6. -- Bodhisattwa (talk) 02:44, 23 April 2021 (UTC)

Hello User: Bodhisattwa, About the research expenses, we answered to clarify why there is no other major expense in the budget, your observation is dependent on how you review it and is independent of our thought process.

• Because we believe we can deliver what we are committing. We have support from experts in the field and are open to collaborating with existing researchers. But some of the projects we reviewed, didn’t have social science experts but still, the researchers were able to execute the program. Also, we don’t think we should refrain from proposing what we intend to work on, in the hope for a social science individual with research abilities to take this up.
• Considering the vastness of the proposed study, different work aspects, for favorable work engagements (checking the reported hurdles), expertise and experience requirements and to balance the observations & conclusions by avoiding systemic bias we need two of us for the proposed study.
• Belittling on the grounds of limited knowledge of one’s competency or expertise also isn’t fair. The law of comparison says, only likes can be compared and our proposal is comparable at the basic levels to all the shared proposals. We justified our budget on the grounds of our value at work which wasn’t an acceptable reason to you.
• We don’t think there is any strict rule on communication practices. We explained our reasons and intentions. --Nitesh Gill (talk) 19:22, 24 April 2021 (UTC)

Hi @Nitesh Gill:, thank you for your reply. I think, as a community volunteer, I have addressed all my concerns regarding financial portion of the proposal and although, there are major disagreements, you have tried to address them from your perspective. So, I guess, I can now close this conversation from my side and leave the proposal to grants committee for their review, when seems fit. -- Bodhisattwa (talk) 20:31, 24 April 2021 (UTC)

Not realistic

Sorry but i have to say. If this research is to be conduct in the US or Europe, i would have understand but based on Indian standards (forget comparing to low income people), ~1 lakh rupee as salary per month is way too much for this research. Not realistic. Looks like this research will benefit involved researcher, rather than the communities. I have also concern that one researcher doesn't even involved in wikimedia movement (i see almost zero contribution from them, no wiki experience).

Even without any research, from my 8 year experience i can tell, like every big community (en, de, it etc) gender gap also exist in indian communities. Many communities even doesn't have female contributor. Same thing when it comes to leadership. I don't know how project goals 1-4 would benefit indic community. Let say you done that, now what. After that It won't magically decrease gender gap. To decrease gender gap, to increase female leadership first we need more female contributor. To achieve that we need to promote wikimedia to female population. Fund those who organise webinar, workshop etc which will actually have some effect. Right now this research looks waste of money. আফতাবুজ্জামান (talk) 18:30, 20 April 2021 (UTC)

Hi User:আফতাবুজ্জামান,

The title was a bit misleading to us as we thought you have doubts about our abilities to execute the program & deliver the listed outcomes. Following the details, the identified concerns are about- finances & need of the study. We appreciate you having vocalized your individual opinions and concerns here and we would like to share some information to support our vision & mission. The ability to conduct & execute a research proposal isn’t based on Wiki experience, but understanding its importance & to address this gap, we have Nitesh as a co-investigator who is one of the notable wikimedians in India with years of experience. Also, no such limitation was specified in the guidelines.

As you have already shared your agreement with the fact that the gender gap exists even in Indian communities, this is substantial ground to design initiatives to bridge the same. Every plan or model is based on a tried and tested hypothesis which is achieved through research. When forming personal convictions, we often interpret factual evidence through the filter of our values, feelings, tastes, and past experiences. The usefulness and acceptability of an assertion can be improved or diminished by the nature of the assertion, depending on various factors. The existence of the gender gap is a fact as it is verifiable, but questioning the need for research in India is a prejudice. While reviewing the existing reports for designing the programs, we could not find the data on- growth of projects under different leaderships, iterations & factors affecting those patterns especially with reference to Indic projects. The reason why we did a pilot study, designed this proposal, and intend to study this in detail.

As far as your concerns regarding Indian standards of salary are concerned, I am concerned about your ignorance towards the intellect and value we possess. This might provide you more clarity on value & earning- https://www.ecnmy.org/learn/your-livelihood/wages/who-decides-how-much-we-earn/ The proposed remuneration is not alien to Indian research and economy.

The project notification didn’t mention reserving funds for marginalized communities, so we fail to understand your concerns regarding the grant money affecting the low-income people

--Praveenky1589 (talk) 21:42, 20 April 2021 (UTC)

Sorry, i am not convinced. I feel this research is asking for money which is unrealistic (since the research & researcher all based in india). I am not ignorance, i know very well how hard it is to get new contributor especially female contributor in indic communities, don't believe me, just try once. We should fund money which will have some real effect e.g. organising webinar, workshop, promoting wikimedia to indian women etc, Not this much money to a research. Anyway, i raised my concern. But It's up to WMF. --আফতাবুজ্জামান (talk) 00:10, 21 April 2021 (UTC)

Sensitivity of the topic, methods to be used and definition of variables

This is an interesting idea, and would potentially inform how we plan and organize community efforts in India and other parts of the movement. As I am reviewing the proposal, I have a few questions and some concerns;

• Under the first sub-section, it was mentioned that "Issues such as female sexual and menstrual health are often overlooked by male editors" and cited with this source, which mentions, “I was looking at the Wikipedia Marathi content for a while, and I felt that the main area where people were not looking at were female sexual health and menstrual health. Some topics that which are considered taboo also need to be addressed,” Dhanushree said. I am unable to understand how that quote translates to saying that male editors ignore the topic.

The intention was to share an additional perspective. Though not recorded anywhere, Nitesh from her experience of working on WWWW projects shared that there was some hesitation that members shared about writing women health topics. We aren’t claiming ignorance, but mentioning a possible reason. We agree that Wikimedia communities have taken steps to curb this skewness in the editor ratio, however, it is still far from balanced.

• Is the hesitation with editors of both the genders or only a specific gender? The above statement is gender-neutral. KCVelaga (talk) 18:31, 1 May 2021 (UTC)

• Many of the studies referred to the proposal are from 2008, 2009, 2012; those were the times when there was hardly an organized community in India. In the last ten years, the movement in India grew very rapidly and several initiatives being led by women, did you refer to any recent studies that you might be building your study upon?

We agree with you that women participation in the last decade of the 21st century has increased, however, there is a big gender gap in the wiki editors as suggested by recent studies. (1, 3 2). It is also important to note that although the gender gaps in editors and contributors have been studied extensively, the gender gaps in leadership positions have not been studied. In our pilot, we observed differences in the patterns of engagement and outcome in some initiatives, the reason why we want to study this in-depth. We are compiling the report and will soon share the observations. One of the visible concerns being the gap in affiliate representation, we wish to study this more to substantially analyze the project engagement and growth patterns.

• Can you elaborate more on the variables "project engagement" and "growth patterns"? If these aren't variable by themselves and are based a set of variables that will be used to say what is engagement and growth, what would be those? Also, you have mentioned pilot, where are the results of this pilot? KCVelaga (talk) 18:31, 1 May 2021 (UTC)

• "comparison of individual and project growth for males and females if the required data sets are available" - if I am not mistaken tons of data is already available through various public tools such quarry.wmflabs.org, query modules of MediaWiki API, and xtools API. Have you surveyed what data do these tools offer that will be of help for your research? The Research Team also provides special priviliges to access non-public databases for research and tooling purposes, has this been enquired about? I think there should not a question of "if the data is available" because, from my understanding, the whole point of this research is to map the relation between gender leadership and their impact on projects - if no initial survey of what datasets are available and what will be used, I feel it might be a problem later on quantifying the impact purely based on qualitative information. You will need data analytics skills to retrieve and process this information, do any of the team members already possess these or do you plan to hire a data analyst?

Our aim is to establish the gender gap w.r.t. Leadership & project growth and also to ascertain the extent of this gap. Praveen has the required experience with the tools required for data analysis. If required, we have academicians to support us with the same. Thank you for the observations & suggestions. We knew about the support from the Research Team and about xtools, but will definitely explore the other shared resources. Also, by the availability of data we meant mapped contribution/outcome/ for activities.

• As I have asked above, how is "project growth" being defined for this research? For a project of this scale, I would expect a survey of datasets available and not available done already. KCVelaga (talk) 18:31, 1 May 2021 (UTC)

• How will the impact of other factors apart from gender leadership be isolated during the study? I can at least imagine two scenarios where the data might misinform; an event led by a women, but an editor is actually mentored by a male editor, and vice versa i.e. an event led by a male editor, but actually trained by a female editor. This is possible because, during events, while one person coordinates the organizing, a lot of other editors actually do 1:1 mentoring and doubt clarification with participants. So it may not always be about who is leading. There can also be other reasons why editors continue to edit later on - have these factors been thought about?

We absolutely agree with what you have shared here. When setting demarcations in leadership for programs, we can’t be aloof to the fact that no event/activity can be organized with exclusive male or female support and participation. We will classify sections on the grounds of lead, organizers, or grantee(s). For breakdown levels as well, we will follow the same pattern for classification.

• I wonder why only US$137.57 for "Statistical Analysis" - it is quite low compared to the potential work and other budget tabs.

Praveen will be dealing with the analysis of data as well as the interpretation of the results of data analysis. This section of the budget will be used only if any software or additional help is required.

• "The problem of women’s low participation in editing Wikipedia is also impacted by the limited women leadership. Supposedly, a higher number of women leaders must attract more women to come forward to edit." - I feel there is a conclusion already being made about the research results, which is potentially a researcher's bias. (highlighting as no source has been cited)

This is a hypothesis that we will be testing in the proposed study. This observation was reported in one of the recent reports on the Gender gap. There have been reports on many other factors influencing women participation as well. A study by Julia Bear and Benjamin Collier found that one of the reasons for lesser women participation was the fear of committing mistakes and the associated criticism with these mistakes (here) . It stands to reason that this fear can be mitigated by women in leadership roles as well.

It rather appears to be a conclusion rather than a hypothesis to be tested. KCVelaga (talk) 18:31, 1 May 2021 (UTC)

In order to address the bias, we have two unrelated background individuals as PIs and have plans to conduct blind testing for validation of outcome.

• How would you define the following variables:

• a "leadership role" in "activities and affiliates"?
• a "project headed by males and females"?
• "editor retention"
• "performance of affiliates"

It can be highly subjective and sensitive on how these are defined, and can completely overturn the results of the proposed research. All these definitions should be defined and be opened for community review, to identify potential biases.

We are aware of the scenarios where no event is exclusive to a specific gender, we will consider the community organizers/facilitators/grantees/leads as “leaders” in this specific project. We will segregate the data on the grounds of the available gender information. We are aware of the sensitivity, limitations, and associated factors, but considering this is an unexplored zone, we'd like to decide on segregation or comparative models on the grounds of our observations. Our intention is to study the growth, expansion, and impact of all such initiatives and the difference in patterns if any. Also, following the law of comparison, we will only consider intra-affiliate comparison for affiliate-centric analysis and inter-affiliate activities for general growth and expansion studies.

• "growth, expansion, and impact" - can you please explain how you would quantify these factors? Also, "editor retention" and "performance of affiliates" from my initial comment haven't been addressed. KCVelaga (talk) 18:31, 1 May 2021 (UTC)

• I am further concerned about data collection practices. Firstly, it would be good to know your sampling technique? While it is important to protect personal information, given the sensitivity of the topic, I think there should be a community panel to verify the data being collected such that the results are not skewed in anyone's favour.

We will follow different sampling methods for the different study objectives. We will use Stratified sampling as it improves the accuracy and representativeness of the results by reducing sampling bias.

Purposive sampling will be used for the review of the outcomes. Considering the review is qualitative in nature, this method helps in canvassing the public for opinions.

We are aware of the factors that may cause bias like- deviation from the specified methodology, omitting of challenging communities, low engagement or using outdated data, etc. We agree with your suggestion to have a community panel verify the data collected. We will send out the invitations for the same once we are there.

• The proposal seems to only consider "male and female" genders. An emerging understanding of gender is about non-binary genders, how do you plan to deal with this sensitivity in your research?

Thank you for your suggestion. As much as we are willing to consider this option, we are dependent on the available information. We can't ask people to associate themselves to a certain group for the purpose of study, so we will focus only on the data and activities where the gender of the leads have been identified and shared. Also, we intend to study the gender gaps and we are open to considering all depending upon the availability of information.

• I feel attempting at the national level, especially in a country as diverse as India is a huge risk, as several aspects of the research plan and execution remain unanswered. Even the suggestion was to apply for a project grant, it doesn't necessarily mean to scale up the project in the first go. I would recommend considering doing focused research of 2-4 languages in the first place, have a community review of the research practices and then scale it up for the country. In an ideal case, it would be good if you can create a research toolkit after working on two or so languages, which can be used by communities in their own contexts.

It is true that there are various factors and underlying parameters to be studied in this project, however, we will not study all these variables. Our main focus is to study gender gaps in leadership positions and its impact. We understand that even this will be a difficult task considering the number of Indian Wikimedian communities but as we already have some identified filters for data segregation in order to make the data sets comparable, that will make it feasible for us to complete it.

But considering your suggestion, we believe that the community panel can also prove assistive in reviewing the designed methodology.

Thanks for your time. KCVelaga (talk) 14:55, 21 April 2021 (UTC)

User: KCVelaga, Thank you so much for your insights, suggestions, and guidance. We really appreciate the in-depth review and observations. Looking forward to learning more.

P.S. Apologies for the delay in response. We had limited availability because of some personal engagements. --Praveenky1589 (talk) 18:21, 26 April 2021 (UTC)

@Praveenky1589: Thanks for your responses. I have asked a few follow-up questions. I feel given the uncertainties in the project and complexity of the Indian region it should be scaled down. After reading the proposal and the responses, the research plan remains unclear to me - it seems like several things are left to be decided after the project begins. I would also like to know more about how the connection between affiliate/community/activities leadership will be established with the growth of a project. KCVelaga (talk) 18:31, 1 May 2021 (UTC)

Aggregated feedback from the committee for A study on analysis of leadership wrt Gender and its impact on projects, individual and community growth in India

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	6.0
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	4.8
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	5.6
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	4.6
Additional comments from the Committee: The five goals are directly related to understanding patterns that affect diversity and power dynamics in the movement. I am not so sure about what we will learn that we don't already know, though. A research of this magnitude should have more specific hypotheses that are not just the observation of general trends. The project fits with Wikimedia's strategic priorities but its sustainability is questionable. In line with strategic priority of fixing gaps and getting to uncovered parts of population in terms of involvement. Though India is a huge country in terms of population, the research is quite local (that's not bad, just it is what it is) and won't be scaled to other countries/chapters. The project meets the wikimedia strategic properties. The project is sustainable, scalable or adapted elsewhere at the end of the grant. The reason is because project is feasible enough. It is unclear to me what we will learn here, especially because the proposal is not embedded in a strong research preliminary review. Thus, it demonstrates little potential for impact, relative to investment. The project is not innovative as it consists of rather standard research into the gender gap. The potential impacts are probably not very high - it is not clear how the research findings will translate into any change. The evaluation plan as well as the research methods are not well defined. I missed the hypothesis to be tested by this research. We do understand that the gap exists, why leadership is a key factor to be tested here? What other factors can influence female participation? What are the most important of them? I think that the preliminary research and review of existing fresh reports and analytics wasn't sufficient here The project takes an innovative approach to solving key problems. The potential impact of the project is greater than risk. The project has an evaluation plan that measures the outcomes of the project This proposal brings together an academic and a Wikimedian, which is a strong match. The only caveat is that they have no prior research work on Wikimedia. The project can be accomplished in 12 months although the budget seems excessive. The necessary skills are probably present. One of the participants isn't an active wikimedian and the budget is huge. It was pointed out on the talk page that it's not clear why 2 investigators are needed, that they lack relevant experience (working in forensic, not gender gap analysis before) and the remuneration itself is huge. Actually looking at the projects, it can be completed in 12 month. The budget is realistic enough but If the fiscal sponsorship bit is taken out that would save us some money because it is not really needed and moreover the contingency stated can support in the case of crises. The participant has the necessary skills for the projects, also the involvement of experienced community members to increase the capacity of the project. It would be interesting to frame this work within questions that would also come from affiliates, as they are the ones that will be investigated. A more participatory research framework would lead to greater results. Questions raised from community members are central and have not been sufficiently addressed. The project supports diversity but the community engagement is low. I see mostly objections on the talk page and they are reasonable Basically this research is target the wikimedia community in India and other sister communities like Punjabi and Tamil. Looking at the team of people working on the project they do have community support. The project supports diversity based on the scope of the research looking at the impact of gender gap of which the primary focus is females. Once the project is over, we will be able to empirically say whether a gender gap exists among the leadership of Indian Wikimedia Community, at a cost of 32'000 $. Some preliminary steps appear to be required: - a better understanding of the academic literature on the gender gap on Wikimedia, in order to create a set of more specific hypotheses (the way the proposal is written right now is too generic and might not lead to any impactful finding); and - a more participatory framework, learning from questions that affiliates ask themselves. I would also like to have more clarity on the audiovisual product that is being considered: what is its envisioned audience? What will be its format/length? The project does not clearly spell out the research methods, its evaluation plan is vague and the budget seems excessive. I think that the project needs more preparatory work. We are a non-profit with donations coming from our readers and we need to spend efficiently every dollar we receive. If something could be done for less, it should be done. If we have another opportunity that will create more impact with the same resources, we should choose it. Here, I believe that the requested amount is too high, that the hypothesis isn't strong enough and wasn't carefully selected. Some of the project members have relevant expertise, some - not, not clear why they don't attract experienced scientists and why we need to pay here Yes this project will be something I will like to support because of what it seeks to do by conducting research related to the skewness of women on wikimedia project and how to increase representation of women when the research is done. I support full funding for this project.

Opportunity to respond to committee comments in the next week

The Project Grants Committee has conducted a preliminary assessment of your proposal. Based on their initial review, a majority of committee reviewers have not recommended your proposal for funding. You can read more about their reasons for this decision in their comments above. Before the committee finalizes this decision, they would like to provide you with an opportunity to respond to their comments.

Next steps:

1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback carefully and post any responses, clarifications or questions on this talk page by 5pm UTC on Tuesday, May 11, 2021. If you make any revisions to your proposal based on committee feedback, we recommend that you also summarize the changes on your talkpage.
2. The committee will review any additional feedback you post on your talkpage before making a final funding decision. A decision will be announced Thursday, May 27, 2021.

Questions? Contact us at projectgrants wikimedia  · org.

Marti (WMF) (talk) 04:21, 5 May 2021 (UTC)

Responses to the committee feedback

To my understanding, there are four key issues raised in the committee review -

• Unclear goals and methodology of the project - The methodology of the project has been discussed in detail. Following are the objectives of the proposed research -
  • Goal 1 - To evaluate the number of males and females in leadership roles in events and activities in Indian Wikimedia communities.
  • Goal 2 - Meta review of participation, outcome, and continuation of projects headed by males and females in the last 5 years (2016 - 2020).
  • Goal 3 - To study the impact of these gender patterns on the performance of three Indian Wikimedia communities.

The research methodology will be divided into the following steps-

• • Data collection - Data related to events and activities of three Indian Wikimedia communities for the years 2016 - 2020 will be analyzed. The communities selected for the proposed study are- Hindi, Punjabi, and Tamil communities. The reason why these have been selected for the study is- Hindi is the most widely read language in India. The Hindi community has also been actively organizing various events and activities in India. The Punjabi community is the first recognized Wikimedia affiliate in India. Tamil community is one of the widely engaged & active communities (not a recognized affiliate) in India. Also, Punjabi and Tamil communities have been leading the two iterations of Project Tiger edit-a-thon in India. The diverse backgrounds & engagement of these communities will provide us with a variety of factors influencing projects and growth.
  • Category-wise data segregation - Data will be segregated on the basis of projects (WikiData, Wikipedia, WikiCommonsetc.). In the next step, the data will be further organized into four categories - a) events led by males, b) events led by females, c) events where males and females have been co-organizers, and d) events where the gender of the organizer is not known. Note: Following the law of comparison, ergo, all the comparisons will be made for activities falling in the same categories.
  • Intra-community comparison - Using the data matrix generated in the previous step, data analysis will be performed to compare contributions of males and females in leadership roles in respect to similar events and for each year within the same community. In this manner data for all three communities will be compared to study intra-community patterns.
  • Inter-community comparison - The comparative data metrics and relative ratios of engagement patterns will be compared for the three communities. Considering multiple factors that affect engagement and growth, the data values will not be directly compared, while the relative patterns of engagement & growth will be calculated and compared.
  • Result compilation- The results will be compiled based on the above-mentioned criteria and observations.
  • Community review and consultation - The compiled results will be presented before a committee for the review and consultation. The committee will comprise a minimum of 10 members. These members will include 2 members from each community, 2 external reviewers, and 2 gender gap experts. Communities (Hindi, Punjabi & Tamil) will be asked to nominate their representatives for the review committee. Two long-time wikimedians with an impeccable contribution to Wikimedia will be asked to participate as external reviewers in this study. Two researchers who have been associated with the gender gap studies (one with the international community and one with the national community) will be asked to be part of this study as gender gap experts. The committee will be asked to review our results and provide us their feedback. Necessary improvements will be made on the basis of their recommendations.
  • Publishing results - Once the results have been finalized by the review committee, the results will be published in high reputed journals.

• Impact of the study - Before searching for the solutions to any problem, it is necessary to establish whether the problem exists and to what

extent this problem exists. The same is true for the problem of the gender gap. Although many studies have been conducted to study gender gaps in editors of Indian Wikimedia communities, no such studies are available to study gender gaps in leadership which is responsible for organizing events and its impact on editor performance.

• Budget - Considering the previous suggestions, Nitesh Gill has stepped down from her engagement with this project. That is why the budget has been reduced to half. The revised budget is shared below.

S.No.	Category	Number of Units	Cost per unit (in dollars)	Total cost in dollars
1.	Project Manager -20hrs per week (52 weeks/12 months)	1	12/ hour	12480
2.	Storage Devices (external hard drives)	1	55	55
3.	Private contractor for Statistical Analysis	1	137.57	137.57
4.	Graphic Designer	1	1500	1500
5.	Research Expenses (stationary, facilitation, and publication expenses)	1	690	690
6.	Honorarium/ Merchandise for Participants	15	14	210
	Total			15,072.57
	Contingency expenses (7.5% of total expenses)			1,130.44
	Fiscal sponsorship (10% of total budget)			1507.257
	Grand Total			17,710.267

• Forensic background of project investigator - It is true that I have no previous experience with social science research. However, during the review of literature, I was shocked and surprised to see that many of the previous research projects have been led by project investigators who either did not have any experience in any field of research or had no research experience let alone social science research especially with reference to the Indian context. There were some studies conducted by field experts, but is it advisable to keep waiting for someone from the field to realize the need for this study to be conducted? I have completed my doctorate studies including Research coursework which is the same across all social science research. Additionally, I have sufficient research experience which will help me conceptualize and complete this project successfully and efficiently.--Praveenky1589 (talk) 16:40, 11 May 2021 (UTC)

Round 2 2021 decision

This project has not been selected for a Project Grant at this time.

We love that you took the chance to creatively improve the Wikimedia movement. The committee has reviewed this proposal and not recommended it for funding. This was a very competitive round with many good ideas, not all of which could be funded in spite of many merits. We appreciate your participation, and we hope you'll continue to stay engaged in the Wikimedia context.

Comments regarding this decision:
This project aims to characterize the role of gender in leadership around Indian Wikimedia communities and projects. While much research has focused on gender gaps in different aspects of Wikipedia, we appreciate your focus on leadership as a novel focus. The goals are relevant and in line with the strategic principle of knowledge equity. In addition, the project plan seems adequate to address such goals

However, the proposal still needs further development. First, the proposal would benefit from a clear definition of leadership in this context. From the section on the proposed solution, it could be inferred that leadership corresponds to the organization of community activities, however, this approach to leadership should be clarified from the introduction.

Second, since it was submitted under the research category, we would like to see a reference section added that points to related research, not only media articles. It will strengthen your proposal to review related research references to guarantee that your research will fill an existing gap in the state of art. Should you want to develop this proposal further and resubmit it in a future round, we suggest you review the following and consider how they might shape your thinking about your approach:

• the last edition of Wiki Workshop included relevant papers on Indian Wikimedia projects [1] and gender gap in Indian Wikimedia communities [2],
• a research project on bridging the gender gap in Indian Wikimedia communities was recently completed [3],
• an earlier work, based on an ethnographic study of Indian Wikipedians, already focused on the issue of gender in motivations to contribute to Wikimedia projects [4].

In addition, since the applicants are interested in the role of leader gender in attracting participation and retention of volunteers, it might be relevant to review and collaborate with other Wikimedia sponsored projects on understanding what causes drop-off and retention of editors, e.g., [5, 6].

References:

[1] Sneha Puthiya Purayil (2021). Languages of Knowledge Infrastructures: Learnings from Research on Indian Language Wikimedia Projects. WikiWorkshop 2021

[2] Bhuvana Meenakshi Koteeswaran (2021). Bridging the Gender Gap: A research study on Indian Language Wikimedia Communities. WikiWorkshop 2021

[3] https://meta.wikimedia.org/wiki/Research:Bridging_the_Gender_gap:_A_report_on_Indian_Language_Wikimedia_Communities

[4] Chakraborty, A., & Hussain, N. (2018). Mapping and bridging the gender gap: an ethnographic study of Indian Wikipedians and their motivations to contribute.

[5] Marc Miquel-Ribé, Cristian Consonni and David Laniado (2021). Wikipedia Editor Drop-Off: A Framework to Characterize Editors' Inactivity. WikiWorkshop 2021 https://meta.wikimedia.org/wiki/Grants:Project/Eurecat/Community_Health_Metrics:_Understanding_Editor_Drop-off

[6] https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Editor_Retention

Next steps: Applicants whose proposals are declined are welcome to consider resubmitting your application again in the future. You are welcome to request a consultation with staff to review any concerns with your proposal that contributed to a decline decision, and help you determine whether resubmission makes sense for your proposal.

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.

Marti (WMF) (talk) 22:06, 27 May 2021 (UTC)