Grants talk:IEG/Women and Wikipedia

Reminder, proposals due March 31

Hi Mssemantics, thanks for starting this draft, looks like a good beginning! I just wanted to remind you that the next IEG proposal deadline is less than 2 weeks away, so I'd encourage you to use the button at the bottom of your proposal page to create the rest of your proposal and begin filling in details about your project plan, budget, etc as soon as you're able - we're happy to give feedback or answer questions as you continue to develop this plan, so please feel free to ask for help here :) When you're all done and ready for review, just update the status in your infobox from DRAFT to PROPOSED. Best of luck, Siko (WMF) (talk) 20:28, 18 March 2014 (UTC)

Thanks Siko! I'm meeting up with my professor early next week to discuss and plan. I'll update the proposal soon after. --Mssemantics (talk) 19:15, 20 March 2014 (UTC)

Strategy docs

Linking some early strategy documentation, in case it is useful....agree that more data would be useful driver for driving more strategic decisions :) Siko (WMF) (talk) 22:20, 18 March 2014 (UTC)

Fantastic! Thank you. :) --Mssemantics (talk) 19:16, 20 March 2014 (UTC)

Ideas regarding data collection and solutions

I've been thinking an indepth study was needed, including regarding issues below, so hope my comments will help you.

You write "My solution is to provide the Wikimedia community with aggregate qualitative and quantitative data that can be used to assess existing efforts to address the gender gap and for designing future interventions or technology enhancements that promise to address the gap." Here are some issues/comments regarding both collecting data and formulating solutions:

Data collection:
Once technical barriers are overcome, past discussions on the GenderGap discussion list and among various women in person (and Sue Gardner's 2009 article) show that the biggest issue women who take the time to edit face is the combative/competitive atmosphere with covert and even open hostility by males towards editors perceived as females. This may be especially true if perceived females disagree with males and/or if the article deals with controversial issues, current events or political/economic power topics where competition is rife and/or many males still feel male voices are the serious ones. Some women may not feel free to admit to this - or even recognize it - without researchers working proactively to address the issues in the following ways:

• Some women editors may not feel free to talk about such issues if there are males in the same room or if the interviewer(s) are or include males.
• In either case, questions that elicit women's comments about such hostility may be necessary, for example:

• Did they register with a gender neutral user name? Why: by happenstance or intentionally? Did they early on identify as female on their user pages or else where? Did they later mentioned they were female? If so, did then notice a change in behavior towards them?
• Do they feel that editors (especially those assumed to be male) treat them differently than those assumed to be female and in what ways? If they are new to Wikipedia, what issues have they had in other mixed sex internet forums or elsewhere on the internet?
• What topic areas do they edit in and have they found more problems in some than in others? Do they find edit summaries, talk page and other discussion comments, etc. are more dismissive or hostile to them that to editors perceived as males? Do they find editors ignore their comments or refuse to discuss things at talk pages? Do these assumedly male editors seem to use double standards regarding behavior of females, for example, charging "incivility" more frequently than they do against editors perceived as male? If they ever received criticism from administrators, or even a block from one, did they feel it was unfair compared to criticism or blocks imposed on editors perceived to be male? Etc.
• Towards the end of interviews, if they have not brought it up themselves, one might ask: Have they been sexually harassed, asked for dates, etc. on or off Wikipedia? (A problem I've heard a number of women have had.) Have they been harassed or threatened in other ways on or off Wikipedia? (A problem I myself have had a lot.)
• What measures have they used to deal with any of these issues as relevant? Personal talk pages? Article talk pages? Asking for help from Wikiprojects or administrators? Other Wikipedia:Dispute resolution alternatives? Do they bring these or any editing issues to noticeboards? Do they consider complaining about sexism or getting it taken seriously to be difficult?
• Other questions as you come up with them from personal experience or as women bring them up.

Formulating solutions:
Things which I offer as solutions which you also may find of interest, given editors' replies (per some on my user page:

• Open discussion of these issues and solutions at Wikipedia trainings.
• Sensitivity trainings, workshop exercises and a video or two on recognizing own sexism and how not to offend women on Wikipedia which all Help-related pages and many Wikiprojects should be encouraged to link to. (Many models for this sort of thing exist.)
• Bring back the "Wikipedia:Wikiquette assistance" noticeboard to deal with low grade incivility, especially sexist comments and obvious double standard behavior, and encourage its use, especially by women. It's silly to have to bring such minor issues to Administrators Incident noticeboard ("WP:ANI") - and most women probably will be reluctant to bring it that far anyway.
• Volunteer administrators get burned out easily, especially since they are so often harassed for imposing sanctions on editors who truly deserve them. Thus bad editors who persist for years drive out better ones - and women - and Wikipedia well may be doomed because of it. Wikimedia Foundation has to figure out a legal way to hire a couple hundred part-time admins to deal with problematic behavior and chronic policy violators. And let's make at least half of these hired administrators women, if we really want a more civil environment and more women editors!
• Administrators should more freely recommend mentoring or Wikibreaks to problematic editors, and more frequently impose more short blocks to give editors time to consider their problematic editing.
• All the Wikimedia Foundation gender gap projects must make it clear to women they have a right to point out issues of sexist language, harassment or obvious double standards. Women's complaints should be given the same respect as complaints by transgenders who even have had an Arbitration to protect them even from editors who refuse to use the pronoun they prefer. Yet even truly repulsive insults against women still must be taken to "WP:ANI" for action. Perhaps we need some women to step forward about a "class action" ANI on this topic.
  

That's all I can think of for now, but will add more thoughts if have them. Good Luck! 108.48.225.196 aka User:Carolmooredc -- 03:49, 1 April 2014 (UTC)

Thanks so much for this terrific feedback, Carolmooredc! I very much like your suggestions re: interview questions. (I'm working on an interview based project right now and am asking similar questions, but your additions are helpful. I'd actually love to interview you and talk about all of this in more depth.) Later today, I'll be responding in more depth to your ideas above; I was so excited by your feedback that I had to leave a quick note!--Mssemantics (talk) 17:08, 1 April 2014 (UTC)

Hi Carolmooredc. It's taken me longer than I anticipated to get back to this, but I wanted to respond to your suggestions re: solutions. I see this grant proposal as being limited in scope to six months. I also see it as being a pilot program evaluation. I want to share back the positive stories of editors who identify as women, and I want to collect data to understand better whether and in what ways WMF and Wikipedia community attempts to address the gender gap have worked and are working. I think the solutions (many of which you mentioned above) will flow out of this data. I'd love to talk to you about this in more detail if you've the time and willingness. I truly appreciate your thoughts and feedback. --Mssemantics (talk) 17:23, 7 April 2014 (UTC)

Feel free to email me via Tools/Email. Carolmooredc (talk) 09:52, 8 April 2014 (UTC)

Research notifications

Hi! Neat proposal :) You should definitely notify our research groups, too:

• Research Committee: https://meta.wikimedia.org/wiki/Research:Committee
• Research mailing list: https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

I'm curious if you planned to do any large scale data analysis. For example, some researchers can gain access to WMF aggregate data on editors who have identified their gender when they create an account or set their preferences.

We also have a lot of gender data from the annual editor survey that could be mined for trends.

Last, we have thousands of editors with userboxes on their personal wikipedia page that identify them as female. Those editors could be sent a link to a survey.

Or you could approach members of WikiProject Women Scientists (https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Women_scientists) or WikiProject Women Artists (https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Women_artists) or WikiProject Countering systemic bias (https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Countering_systemic_bias) which tend to include a lot of women, or at least editors interested in countering systemic bias and the gender gap.

Last, last thought, whatever you do, you should chat with User:Keilana and User:SarahStierch about their work and experience with gender issues.

Cheers! Ocaasi (talk) 22:11, 2 April 2014 (UTC)

Hi Ocaasi! Thanks so much for your feedback and your recommendations re: notification of additional research groups. I'll do that today. :) I'd love to do large scale data analysis in the future (including scraping data from user boxes), but since the scope of this project is limited to six months and there have been some surveys done, I'd like to focus on stories of what's working, why editors who self-identify as women stay, and then conduct what is, in essence, a program evaluation of WMF and Wikipedia community attempts to address the gender gap. However, I am, of course, open to feedback and suggestions. Also, I've talked with some women in the community in the process of doing my project re: women and Wikipedia, and they've been fantastic. That project is ongoing (e.g., I'm still conducting and transcribing interviews), and I've assured the participants confidentiality, so I can't comment as to whom I've spoken with, but I can say: the community has been open and kind. --Mssemantics (talk) 17:30, 7 April 2014 (UTC)

Eligibility confirmed, round 1 2014

This Individual Engagement Grant proposal is under review!

We've confirmed your proposal is eligible for round 1 2014 review. Please feel free to ask questions here on the talk page and make changes to your proposal as discussions continue during this community comments period.

The committee's formal review for round 1 2014 begins on 21 April 2014, and grants will be announced in May. See the schedule for more details.

Questions? Contact us.

--Siko (WMF) (talk) 20:43, 7 April 2014 (UTC)

Women and Wikipedia

Hi I am really happy to see this or indeed any proposal on the subject of Women and Wikipedia. That said, I am a bit concerned about your approach. It would be great to hook you up with Keilana and just do the research that she doesn't have time for. Spending money for you to visit random edit-a-thons in whatever cities is just a waste of time. Just browse through the pictures on Wikimedia Commons taken at edit-a-thons and you will see what I mean. These are heavily skewed towards younger (20-24) and older (over 50) white males, because those are our largest 2 groups of Wikipedians that actually attend such things. At edit-a-thons, people are generally there for some teaser; a presentation on a GLAM or backstage pass of some sort. People will not want to skip this in order to talk to you, and the organizers will not be willing to spend precious time touting your proposal. I really think you need to separate the interview process from the edit-a-thon process, and forget trying to find women at random edit-a-thons. If you could just spend the 6 months personally interviewing (by skype, phone, or snail mail) each and every woman who attended an arts&feminism edit-a-thon in February that would probably deliver more interesting results over a 6-month period. Also, the gendergap needs an article on Wikipedia because we are still not even clear what we mean by it. Yes, we have too few women editors, but why is that a problem exactly? How do you prove to people that problem is urgent and actionable? If you can tackle this first, then the rest will follow. Sorry if I sound negative, I am trying to help! Jane023 (talk) 07:37, 9 April 2014 (UTC)

I forgot to mention that it has been shown that editor retention of edit-a-thon attendees is close to zero! Jane023 (talk) 07:42, 9 April 2014 (UTC)

Hi Jane023. Thanks so much for your feedback and ideas. I don't think you sound negative at all. I had intended--but perhaps had not explicitly stated--that I would target/attend WMF sponsored edit-a-thons and mentoring activities geared toward editors who self-identify as women--such as the Art and Feminism meet-up day (in which I participated remotely and created my first [very brief] article!). I may need to make this more clear in my proposal. As far as the unsponsored meet-ups are concerned, I'd still like to talk to all genders represented to see if we might try to understand the issue from as many perspectives as possible (e.g., it might be valuable to know how editors who are not and do not identify as women perceive and talk about the gender gap) and also to see what is working at those unsponsored events. I'd love to hear your thoughts about this. --Mssemantics (talk) 00:02, 10 April 2014 (UTC)

Well that sounds like it could be a problem if there are no WMF-sponsored or unsponsored meetups planned in the timeframe you are applying for. Talking to all genders is of course possible for past events and I still think this would be more beneficial, though it would be handy for you to attend at least one so that you know what one is like. Do you have a schedule of events? Jane023 (talk) 18:52, 10 April 2014 (UTC)

There's a schedule available here, but the offerings are slim (not filled in) for the fall and winter. There, of course, other events like http://femtechnet.newschool.edu/wikistorming/ that come about, but you have a great point. I'd be more than happy to talk to participants from past events and attend fewer current events; however, as you note, I'd like to attend at least few sponsored and unsponsored events for the purposes of observation.--Mssemantics (talk) 14:44, 14 April 2014 (UTC)

O that is funny - I have been to lots of edit-a-thons, but none of them were on that page, which I never saw before I clicked on your link. I suppose that is as good as any list you can find. Jane023 (talk) 20:32, 14 April 2014 (UTC)

Jane023, just curious: How did you find out about the edit-a-thons you attended and have they been documented? If so, may I ask where? Thanks!--Mssemantics (talk) 20:51, 17 April 2014 (UTC)

I learn about through our Facebook page for WMNL, the grapevine, and the village pump on the Dutch Wikipedia ("De Kroeg"). Mostly direct mail these days I suppose, since I can help as well as edit. Jane023 (talk) 08:14, 19 April 2014 (UTC)

Thanks Jane023. That's good information for me to know.--Mssemantics (talk) 15:28, 19 April 2014 (UTC)

WMF vs. Wikimedia

In some passages you say WMF-sponsored, in others Wikimedia-sponsored. Please decide. Any reason to focus on WMF rather than Wikimedia in general? What do you even mean by "sponsored"? --Nemo 07:01, 15 April 2014 (UTC)

I had the same question, because of course most of the edit-a-thons I have been to were "approved" by WMNL or at least WMNL was aware of them, but they only sponsored maybe 2 of them (for coffee and a snack at the locale provided by the holder). WMNL is a chapter and a separate entity from WMF, but I suppose you could see as somewhat official. In any case, no matter what the edit-a-thon is or who it is organized by, the format will probably be the same: a group of random people being taught how to edit, and then offered materials to use editing, whilst the smaller number of experienced Wikipedians who are not explaining something to the newbies, are just grabbing the materials and editing right away. Jane023 (talk) 07:55, 15 April 2014 (UTC)

Thanks for your feedback. I need to clean up my diction so that I'm more clear. What I'm trying to get at is this: The events paid for by Wikimedia versus those that happen organically/informally and have no funding/budget. I want to understand better if the money is well spent, if the organization and format at different kinds of events is, as Jane023 notes, similar or not. Perhaps this kind of work has already been done or isn't needed. I'm definitely open to suggestions. --Mssemantics (talk) 19:48, 15 April 2014 (UTC)

Sponsorship

What sense does it make that you focus on unsponsored events and you plan to attend as sponsored individual? --Nemo 07:01, 15 April 2014 (UTC)

I can attend local unsponsored events without being sponsored, but as a full-time PhD student, I would not be able to attend any events outside of my immediate region without some kind of sponsorship. You raise a good point, and (as I've noted above) I'm open to suggestions. --Mssemantics (talk) 19:50, 15 April 2014 (UTC)

Method differences

Why focus groups in unsponsored events and 1:1 in sponsored events? --Nemo 07:01, 15 April 2014 (UTC)

My guess is that there may not be many women (if any) who attend unsponsored events like informal meet-ups. I had proposed 1:1 interviews at sponsored events because I had wanted to target events held for women. However, I could just as well do focus groups at those events, too. The idea is to have a range of voices, opinions, stories, and experiences. --Mssemantics (talk) 19:52, 15 April 2014 (UTC)

Transcription

Nemo raised the issue of paid transcription on another IEG, and I agreed with their point so I figured I should raise it here as well: transcription services are expensive, and I question whether they are a necessary expense. After all, this is the kind of work that members of our movement (and PhD students, for that matter) do on a regular basis, for free. If you feel that transcribing all interviews yourself is an undue burden, you might try DragonDictate software, or even explore the possibility of asking some confederates who believe in the cause of Gender Gap research to help... I guess I'm volunteering myself, aren't I? Shucks ;) Jtmorgan (talk) 19:15, 15 April 2014 (UTC)

Hi Jtmorgan. I agree that transcription services are expensive; transcription is also extremely time-consuming. (I'm in the process of transcribing now, and even for a relatively fast typist using a USB pedal, the rate is about 2-3x the length of the interview. So, a two hour interview becomes six hours of transcription.) Since I didn't include a stipend for myself and am, in fact, a full-time PhD student, I decided to budget for transcription services to be able to complete the work within six months. I've talked to several anthropologists/ethnographers re: using a software package like DragonDictate. The issue is that you can't "train" it to many voices: just one. So, essentially, you must repeat both sides of the interview and then check for and correct errors. This can bring the time down a bit, but it still means the total time for "transcribing" an interview via this method is about 2x as long as the interview. Not much of a gain. However, if community members (including yourself!) would be willing to volunteer, that would be fantastic. Confidentiality might be an issue, though, as voices are considered a way to identify someone. I'm open to more thoughts and suggestions. This is a puzzle I'd love to solve! --Mssemantics (talk) 19:45, 15 April 2014 (UTC)

I also responded to the comment by Nemo on the other proposal. To reiterate the crux of the issue, is the point of the proposed work simply to collect interviews and transcribe them? Or, is the point of the proposed work to generate original research and identify possible solutions based on original results? My belief is that the goal of this proposal is to generate original research; spending the majority of the time on transcription is probably not the best use of the researcher's time. I understand the trade-offs between transcribing and paying for transcription. If the reviewers simply want data collection and not any real analysis, then requiring the researcher to transcribe is fine. If, however, the reviewers by into the goal of generating original analysis and results, then it would be better to pay for transcription (or at least some of the transcription) to facilitate moving more quickly from collection into an analysis phase of the research. Dwmc (talk) 03:59, 16 April 2014 (UTC)

Online surveys

I think Jane had some valid points. Instead of traveling, could you make an online survey and recruit survey participants online, and could you do semi-structured interviews online? I think you would get a larger number of responses at lower cost.

Also, could you explain how your research is different than the previous WMF-sponsored large editor surveys in asking about the motivations of editors? I think the most recent WMF survey was in 2012. --Pine^✉ 04:22, 16 April 2014 (UTC)

Hi Pine. Surveys are more cost effective, though not necessarily easier to develop--as a well developed survey that results in the kinds of information a researcher may consider important as well as in the information he or she doesn't yet know to consider important are difficult to create. Surveys also result in a very different form of data/information than interviews and observations. They don't allow for follow-up or for the cropping up of unexpected yet extremely relevant details. The WMF-sponsored large editor surveys, in my mind, don't ask the same questions or seek the same kinds of information I'd like to uncover and understand. For example, the open ended responses were removed from the WMF late 2011 survey and the only direct question re: gender that was asked was essentially "Are you aware of the gender gap?" The survey did not ask questions regarding how a Wikipedian chooses to portray his/her online gender identity, if he/she has received criticism directly related to gender rather than quality of work, what kinds of work he/she may choose to do to avoid gender related conflict, etc. I think Carolmooredc has suggested a great set of questions above--questions that were not included in the WMF-sponsored survey.--Mssemantics (talk) 17:40, 16 April 2014 (UTC)

Mssemantics OK, but couldn't you recruit for interviewees online and interview them through video or voice chat? You might get more responses at lower cost. --Pine^✉ 07:34, 17 April 2014 (UTC)

Hi Pine. Yes, I can recruit editors online, but as I've learned from the project I'm currently working on, even using snowball sampling in Wikipedia is difficult. I'm not a "known" Wikipedian--just a fledgling. It's difficult to get editors to respond, especially those who don't see the gender gap as an issue. I'd be more than happy to do interviews over video and video chat (as I have been), but I still believe being present at, at least, a few meet-ups and sponsored events would greatly increase the number of participants and would serve to allow me to gather observations as well. Also, some of the women I've talked to thus far have stressed the importance of co-location. Interpersonal interactions in shared space is rich in ways that mediated communications is not. I am, however, open to thoughts and suggestions, of course.--Mssemantics (talk) 16:40, 17 April 2014 (UTC)

Hi Mssemantics, if you're going to attend a meetup in person I think it's important to establish first that there will be multiple editors present who are in the population that you want to survey. It would be disappointing to have you travel a long distance to a meetup and have only one or two female editors present. I think it would be reasonable to have you travel to Wikimania and the WikiUSA conference, but beyond that I have doubts that the money and effort would be worthwhile. I would encourage you to design more online recruiting and interviewing into your proposal. --Pine^✉ 07:50, 18 April 2014 (UTC)

Hi Pine. Thanks for your feedback. I agree with you w/r/t having an idea of the participants before arranging travel. I'm going to do some work on my proposal page over the next several days--taking a lot of the great comments here into consideration and making applicable changes. I appreciate your help!--Mssemantics (talk) 17:28, 19 April 2014 (UTC)

Yet another study

As an active female Wikipedia editor (retired after a long career in research management), I feel I've been well-and-truly studied. I've participated in a number of surveys and interviews about my experiences. It seems to me that we already have base-level data and we've tried the obvious interventions (like edit training, facebook groups, tea house) that can be bolted onto the side of Wikipedia. While well meant, I would nonetheless characterise these solutions as aiming to "fix the problem with women". Maybe the problem is not with women but with Wikipedia? I think we need to dig deeper. As mentioned above, we know women are turned off by the aggressive culture of English Wikipedia. But talk to men and you will find many of them are turned off by it too. Having delivered edit training, I know most of the trainees are women. I don't think getting women started is as much of the problem as keeping them active which is why "bolt-on" solutions rarely work -- the problem is in the everyday behaviour on-wiki and so the solutions need to be there too.

The thing that seems to upset editors (men and women) the most is having their edits reverted, a not unreasonable reaction when they have put in time and effort and perceive it as wasted. Since we allow any bad-tempered/arrogant/stupid/officious/busy/etc editor to revert a good faith contribution without requiring a reason and without any *constructive* help offered to the original contributor and with no consequences to persistently behaving in that way, it's easy to see why many editors eventually give up and walk away. I think it's time to move this issue on from studying women to studying editor interaction and, in particular, the revert interaction. I don't think the root problem is inherently a male-female one, but more of an aggressive culture that has the symptom of lower female participation because women are socialised to be less aggressive.

I think it might be more useful to study what interactions in Wikipedia made an editor (male and female) upset enough to stop editing (either for a short time or forever). While it can be hard to reach the ones who have walked away forever, it's possible to reach those who have returned. Let's try to pin down the precise characteristics of those distressing interactions. I think there is a qualitative approach; interviews that get people to describe some of the interactions that gave distressed them in enough detail to unpack what aspect really upset them. Was it just the act of being reverted? Was it the reason given (or lack thereof)? Was it the tone it was said with? Was it the conversation that took place on a Talk page? I think there is also a quantitative approach to explore the nature of reverts of good faith edits. For example, is reverting or being reverted a behaviour that is uniform across all editors in proportion to their level of activity, or are there subsets of editors disproportionately likely to do reverts and subsets of editors more likely to be reverted? If so, what are the characteristics of these groups? Are reverts uniform across articles and categories or do some topic/categories have unusually high/low levels? If so, what are characteristics of those articles/categories? When reverts take place, how much explanation is given (e.g. length of edit summary, inclusion to links)? How often are reverts accompanied by writing on the talk page of the user being reverted or on the article talk page? If we can find what are the characteristics of distressing behaviours and then see who/where/how those behaviours are occurring, we begin to know where/how we need to trial some kind of intervention. Kerry Raymond (talk) 00:20, 17 April 2014 (UTC)

Kerry Raymond thank you for those excellent comments. There has been some research done on the kind of topics that you suggest here, but what hasn't happened, as far as I know, is for anyone to emerge with a plan for what to do about the negative effects on editor population levels from reverts or hostile messages. If you have any suggestions I would love to hear them. For example, if we find that reverts and warning templates are used most commonly on English Wikipedia regarding edits relating to climate change, then what approaches should we try to deal with the negative effects of good-faith editors getting frustrated and leaving? --Pine^✉ 07:32, 17 April 2014 (UTC)

Hi Kerry Raymond. Thanks for your feedback. I agree with you in so many ways, especially w/r/t this not being an inherently male-female problem, but rather one of the nature of a culture that may be hostile to all genders who do not want to engage in an argumentative community. However, based on some of the interviews I've done thus far, there also seems to be a tendency for community members to bring gender into the discussion and attack editors who self-identify as female. This is, of course, disconcerting. There is a study looking at the male-female dynamic re: reverts (among other factors). The researchers actually found that users who identified as female were more likely to be reverted, but I think more investigation is needed, and I like your suggestions above.

Also, regarding the interviews and surveys you've participated in, were the findings shared with you? I've looked for studies on and off Wiki that feature interviews with female editors, but I haven't found that many. I'd love to be pointed in the right direction.--Mssemantics (talk) 16:35, 17 April 2014 (UTC)

Evaluation reports

In case you haven't seen them yet, I wanted to provide the link to the evaluation reports as a reference for metrics. EGalvez (WMF) (talk) 03:09, 17 April 2014 (UTC)

EGalvez (WMF) which metrics do you think are most relevant to this proposal? Thanks, --Pine^✉ 07:27, 17 April 2014 (UTC)

Thanks EGalvez (WMF)! I will take a look. Also, any feedback (as Pine suggests) is welcome.--Mssemantics (talk) 16:41, 17 April 2014 (UTC)

Hi Mssemantics and Pine, all of the inputs, ouputs and outcome metrics of the edit-a-thons and editing workshops reports could be helpful. There should be more information in the overview page (linked above) about how some of the metrics were calculated and some of the limitations. Let me know if you have other questions! Thanks EGalvez (WMF)!

I can see how that would be helpful for the objective of this study "To measure the success of co-located editing and mentoring activities.", but the primary theme of this study is the gender gap rather than co-located editing and mentoring activities in general, so I'm interested in hearing how you think those metrics are especially relevant to the gender gap, EGalvez (WMF) and Mssemantics. Thanks, --Pine^✉ 07:55, 18 April 2014 (UTC)

Hi Pine, I'd be happy to clarify. These metrics are relevant because Mssemantics wants to analyze "qualitative and quantitative data re: success of sponsored versus unsponsored events with regard to user edits and longevity of users." In order to be able to compare fairly across these events to examine their effects on the gender gap or user "longevity" (what we call "retention" in the reports), there should be some level of analysis on the inputs, outputs, and outcomes of the events. For example, the amount of work put into organizing an event, whether staff or volunteer, varies and can have a direct impact on the retention of users. Also, staff hours are inherently different than volunteer hours. Mssemantics also aims to compare between sponsored and unsponsored events, which--as Mssemantics clarifies above--means programs that have a budget versus those that do not. Importantly, there are other forms of financial support that are hidden but can be useful in their analysis, like resource donations (event space, prizes, etc.). The data opportunities are almost endless, so I wanted to offer up some concrete measures that we have previously used in the reports that could help with this project. As a side note, some of the outcome metrics might not be obtainable depending on the timeline of the events. Hope this helps! EGalvez (WMF) (talk) 17:04, 18 April 2014 (UTC)

Conveniently, we have a useful tool that could help with obtaining all this information! JAnstee (WMF) created a Tracking and Reporting Toolkit. Mssemantics, you are more than welcome to make a copy of this tool for your own purpose, but you would probably want to collaborate with the event organizers. Importantly, obtaining username information from in-person events (i.e. where a pencil/paper sign-up sheet is used) requires that the participants sign an opt-in form, which can be found on this page, too. EGalvez (WMF) (talk) 17:15, 18 April 2014 (UTC)

Thanks EGalvez (WMF)! If I receive funding, I'd love to talk to you more about this if you've the time and willingness. Pine, even though the focus is on the gender gap, a part of what I'm asking here is how is the community addressing the gender gap, what seems to be working, what doesn't seem to be working, and is it "worth" the effort/time/money? I'd love to consider these types of quantitative datasets in light of the stories of editors who self-identify as women.--Mssemantics (talk) 17:33, 19 April 2014 (UTC)

Summary of feedback

First, thanks for all of the terrific feedback. Given that the committee is currently reviewing proposals, I'd like to provide a summary here of the changes I would make to my proposal if awarded funding. However, I'll refrain from making changes to the proposal for now. I'd also like to acknowledge the feedback I received that I think is valid and important but out of scope for a six month project.

Data collection

I agree with Carolmooredc that it's important to include the kinds of interview questions she identified; I also agree the primary researcher and interviewer should be female. (I am.) I agree with Jane023 and Pine that I need to attend specific events organized to address the gender gap and that promise a "higher yield" of participants. For these reasons, I would change my target cities to include those that have WMF sponsored events planned for 2014. (Sadly, there are very few in the Pacific Northwest.) I would also host an edit-a-thon locally so that I might better understand the process, be able to specify the focus, and evaluate the effectiveness of an unsponsored event. Following on Pine's suggestion to add more online recruiting and interviewing, I would add this, too. Though I still believe collecting some data (e.g., observations, interviews, and focus groups) in person is valuable and necessary.

Evaluation reports

I appreciate EGalvez (WMF)'s feedback re: the available statistics and forms for considering the evaluation of both sponsored and unsponsored events. I would like to use these. I'd also like to scrape user data to track contributions post-event.

Yet another study

While I appreciate Kerry Raymond's feedback re: understanding why editors leave Wikipedia, I believe it's beyond the scope of the current proposal. I also believe we still need to understand the gender gap. As noted, there have been some quantitative studies that explore the gender gap, there is very little qualitative work that shares the stories of women editors, provides a content analysis of the ways in which the community discusses the gender gap (on Wiki, in mailing lists, etc.), and considers how effective attempts to close the gap have been.

Formulating solutions

I very much appreciate Carolmooredc's comments re: possible solutions. I believe that this kind of information will flow out of the data collection and analysis of this study.

Again, thanks for all of your feedback! --Mssemantics (talk) 16:55, 23 April 2014 (UTC)

Aggregated feedback from the committee for Women and Wikipedia

Scoring criteria (see the rubric for background)	Score 1=weak alignment 10=strong alignment
(A) Impact potential Does it fit with Wikimedia's strategic priorities? Does it have potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	6.8
(B) Innovation and learning Does it take an Innovative approach to solving a key problem? Is the potential impact greater than the risks? Can we measure success?	5.5
(C) Ability to execute Can the scope be accomplished in 6 months? How realistic/efficient is the budget? Do the participants have the necessary skills/experience?	7.3
(D) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support? Does it support diversity?	7.5
Comments from the committee: There are few areas more important to our movement than increasing the participation of female editors. Well-articulated proposal by a qualified and well-advised researcher could help add rigor to our laments about low female participation. Attendance at Wikimania may be worth considering, as there will be hundreds of highly active female editors there and there’s likely a lot to learn from the women who do edit and do thrive in our community. We already know that new contributors at edit-a-thons rarely stick around. The basic idea (interviews of attendees at edit-a-thons) only has the potential to reach a tiny group of potential female Wikipedians that fit the target of the study. Very few women attend edit-a-thons, just as very few women engage with the edit button on Wikipedia. You may be better off concentrating on a local outreach project that targets random women of the general public who are computer savvy and willing to share their credentials online in a user profile on Wikipedia. Some questions remain about whether the final product will provide useful return on investment. Data may not be actionable for improving Wikipedia’s gender gap, and sample size from in-person interviews may be too small to be useful. May want to consider designing the work so there is more online outreach, which can be done at large scale at low cost, and consider cutting some of the travel. Proposer's engagement with the community is impressive. Project is focused in the US, but it could be replicated in other countries if successful. If funded, proposer is requested to focus research discussion on actionable recommendations and remedies that our community can take.

Thank you for submitting this proposal. The committee is now deliberating based on these scoring results, and WMF is proceeding with it's due-diligence. You are welcome to continue making updates to your proposal pages during this period. Funding decisions will be announced by the end of May. — ΛΧΣ²¹ 00:00, 13 May 2014 (UTC)

Thanks so much for the helpful feedback and supportive comments. I've made edits to the proposal in response to community and committee feedback.--Mssemantics (talk) 01:05, 20 May 2014 (UTC)

Total number of interviews

Hi, what is the total number of interviews that you plan to evaluate for this research, including the interviews that you have already done? --Pine^✉ 19:37, 25 May 2014 (UTC)

Hi Pine. Thus far, I've conducted 16 interviews. I'd like to conduct at least 30 total for this project (in addition to surveys and focus groups).--Mssemantics (talk) 18:11, 27 May 2014 (UTC)

Research review suggestion

http://cyber.law.harvard.edu/events/luncheon/2014/01/hargittai-shaw --Pine^✉ 19:38, 25 May 2014 (UTC)

• Mssemantics here's another suggestion. I was effectively a co-author of this research along with two fellow Wikimedians. https://meta.wikimedia.org/wiki/Mind_the_Gap --Pine^✉ 08:40, 1 June 2014 (UTC)

Related comment

Hi! Thanks for your great iteration on the recent proposal. As a committee member I'm really pleased and impressed.

One comment arose in our discussions, expressing a desire to contextualize your learnings alongside consider recent research by Hargittai and Shaw:

• http://cyber.law.harvard.edu/events/luncheon/2014/01/hargittai-shaw
• http://civic.mit.edu/blog/natematias/eszter-hargittai-and-aaron-shaw-on-internet-skills-and-wikipedias-gender-inequality

One point raised recently is discussions is that a good number of women are not even clicking 'edit' and so are somehow getting turned off or feeling unwelcome or unfit before they even encounter the very challenging editing climate. Insofar as this applies to women who don't edit (but might) rather than the women who do edit (but face discouraging aspects and leave), I'd be very curious to see if we can distill or combine some insights that apply to both spheres. Thanks! Ocaasi (talk) 19:56, 25 May 2014 (UTC)

Hi Pine, Ocassi, and the committee! I am familiar with Hargittai and Shaw's work (I've actually watched the recorded presentation). They have hit upon some very interesting findings I'd also like to explore. However, some of these findings may be tied to larger social/systemic issues. Digital literacy and the digital divide in general are talked about quite a bit, but are not entirely understood. Moreover, structural oppressions w/r/t gender are pretty complex (e.g., "wicked problems"), as has been evidenced in the trending #YesAllWomen tweets this past weekend. All of this to say: Yes. :) I'd love to contribute to trying to answer this question. Understanding why the women who do edit edit is an important first piece. From my interviews thus far (though I'm still interviewing, transcribing, coding, and analyzing), I've also heard from these women why they might not continue to edit. I hope to glean some design and policy suggestions from my study. I hope that answers the question. --Mssemantics (talk) 18:21, 27 May 2014 (UTC)

Thanks Mssemantics. I may have more to add after the grant decisions are finalized. --Pine^✉ 06:42, 28 May 2014 (UTC)

Round 1 2014 Decision

Congratulations! Your proposal has been selected for an Individual Engagement Grant.

The committee has recommended this proposal and WMF has approved funding for the full amount of your request, $8075

Comments regarding this decision:
We appreciate the incorporation of a survey to gather more perspectives online and the other adjustments you’ve made in response to community and committee feedback, and we look forward to learning along with you as you share findings on-wiki and beyond.

Next steps:

1. You will be contacted to sign a grant agreement and setup a monthly check-in schedule.
2. Review the information for grantees.
3. Use the new buttons on your original proposal to create your project pages.
4. Start work on your project!

Questions? Contact us.

Thank you

Many thanks to the community and the committee for such terrific feedback and for this opportunity. I'm humbled to be given this kind of access to and support from the community for what I believe is important work. My goal is to honor the stories of the women who edit Wikipedia and to deliver a final product that is insightful, accurate, and actionable. --Mssemantics (talk) 17:33, 30 May 2014 (UTC)

Congrats! :-D Jodi.a.schneider (talk) 12:17, 31 May 2014 (UTC)