Grants talk:PEG/User:Samir I. Sharbaty/Egypt Wikimedians User Group/WikiWomen Prize

Evaluation by the GAC

GAC members who support this request

1. Support, but I do think this is nice only for a short term effort to "recruit volunteers with bells and whistles". Once a group of interested female editors gets together, I do suggest that other actions be applied to keep them motivated. Alleycat80 (talk) 20:40, 30 September 2014 (UTC)

GAC comments

1. What is the current average sallary in Egypt? This is just a question of what is the value of the prizes for Egyptian people. Is it a small fortune, or something close to average monthly sallary?
2. Regarding measurement of the success - I would opt to follow the after the contest activity of participants for longer time and see if they really are active contributors after say half a year and one year.

Polimerek (talk) 08:35, 20 September 2014 (UTC)

@Polimerek: Well, this varies from place to another and from one to another so it's difficult to set a number, but may be the average amount for a student or a fresh graduate to spend is around 500 USD/month. I believe that a beginner user of Wikipedia shouldn't be motivated by money only and that the prize must not be equal to her/his efforts because if we set a very big prize they'll not continue to edit without it in the future. But if you believe that the third and second prizes or even the first one are very small and should be raised tell me. Also it would be good if we hear from Alex her views to this regard. -- Samir El-Sharbaty ^{(talk to me)} 15:00, 21 September 2014 (UTC)

The prizes seems to be OK IMHO - usually in other similar contests they are set up to be substantial but not extremely high from local POV. Polimerek (talk) 15:54, 21 September 2014 (UTC)

I concur. Living in the middle east, I would know a thing or two about salaries. The prizes are not low - - they are quite nice, actually - - but not too high. Alleycat80 (talk) 20:40, 30 September 2014 (UTC)

Tony1 comments

Hi, this looks like a very good proposal—thank you! But why is the application so late? It's halfway through the duration of the activities.

Could you provide a breakdown of the social media advertising and the poster printing? Who would design the text/graphics for each type of publicity? I'm a little dubious about the impact of posters; would it not be more effective to spend the whol $200 on social media ads?

Page on ar.WP: nearly 400 hits – good. But the talkpage is blank!

I see no criteria for judging "contributions", for both participatns and jury. How will you balance article creation with improvements to article completeness and improvements to article quality (like referencing, adding images)?

Who will be on the jury, and does everyone understand that jury members should exclude themselves from judging anyone with whom they are close?

Thanks. Tony (talk) 12:54, 15 September 2014 (UTC)

Replies from Samir

Hi Tony, Thanks for reviewing the proposal and your comments. Following some discussions with the rest of organizers and with Alex Wang who revised the proposal carefully and talked to me via e-mail and a video call, and taking your notes into account, we have decided to make some changes and give some explanations:

• This is a pilot round of the competition and will be extended till the end of October. The start date of this proposal changed to be 1 October as this will be the actual date when we will start spending money, and the end date changed to 31 October. If this round was successful we will take the learned lessons into account to help us organizing new ones in 2015 and the next rounds would be longer to last for 3 to 4 months.
• The reason behind the late proposal is that we wanted to take the advantage of having 3 university summer courses in Egypt Education program about women and that they will last till the end of September only.
• This competition is targeted to beginner users with less than 1,500 edits who can not join the Producer prize with expert users.
• Walaa will take care of choosing the jury and every participant will prepare a summary report about their achievements on a public page on Wikipedia so that results can be transparent. The jury will choose winners depending on their whole work, for example if a user created 100 new articles and developed 2 featured articles is better than another who created 50 new pages and developed one good article, etc.
• Advertising budget reduced to 50 USD and we will mainly depend on word of mouth to promote the competition. The amount will be used for online Ads and printing some stickers as tokens for the participants.

Thanks, and please let me know whether this answers your questions or no and if you have any further questions or suggestions. -- Samir El-Sharbaty ^{(talk to me)} 17:14, 18 September 2014 (UTC)

Thanks, Samir. I would be more comfortable with a worked-out structure for judging the prize, in terms of criteria. That is a fundamental way of setting the standard, of communicating to participants what you want them to achieve. It's also probably a much more effective way of demonstrating impact at the end of the activities. I think your community engagement and quality of output would be on much more solid ground if there were a system of answering questions, providing encouragement and advice, on a dedicated on-wiki page. But it's a small budget and, as you say, a pilot. If you succeed in gaining funding, I wish you the best and look forward to the outcomes and your report. I will alert the en.WP gender-gap task force if you're funded; they will be interested. Tony (talk) 08:57, 20 September 2014 (UTC)

@Tony1:Well, do you mean to set a criteria for selecting the winners like giving 1 point for creating every new stub article and 20 points for developing every featured article,... etc, for example? I think the jury will do so, but I don't know exactly the numbers they use. If necessary I can ask them to set a table and announce it to the public. -- Samir El-Sharbaty ^{(talk to me)} 14:36, 21 September 2014 (UTC)

Samir, I guess I was referring to quality-of-product criteria, which would speak to both the judges and, more importantly, participants. Many possible participants will probably not be familiar with the criteria for featured content and good content (English-language google translation). You might, then, consider either linking to the official criteria on the competition page, or extracting the most relevant points from the criteria. For example, the last bullet of the FA criteria on ar.WP (appropriate length) could be removed, since you'll be judging contributions over a range of article maturities; the point about stability might be excluded as unnecessary in this context; other criteria could be shortened/simplified/rationalised, as simply good advice, even for creating stub+ articles. Framing this as "the highest standards, but we do as well as we can under the circumstances", would guard against frightening some people away by making them all feel they have to produce featured-standard content (even though some of that is in your measures for success). Analogy for good article and featured picture criteria on ar.WP. Tony (talk) 11:50, 26 September 2014 (UTC)

• Thanks for explaining this, Tony. I will update the contest page on Ar Wiki and will take your advice into account. Regards, -- Samir El-Sharbaty ^{(talk to me)} 22:20, 26 September 2014 (UTC)

WMF comments

Hi Samir El-Sharbaty. Thanks for all your work on this proposal and for your engagement in the discussion. We're excited about supporting more efforts in Egypt to encourage female editors and more women-specific content! We have the following remaining questions:

1. Does the "25+ new submissions" mean 25 new articles or 25 participants? We would suggest aiming for 40 new or improved articles, but please use what you think is most appropriate.
2. We would recommend assessing editing behavior over 2-3 months instead of just 1. In those months it would be good to do some type of follow-up with the participants to encourage them to continue editing.
3. What is the process for assessing featured articles on the Egyptian Wikipedia? What is the timeframe for having an article reviewed?

We look forward to your response. Alex Wang (WMF) (talk) 04:12, 25 September 2014 (UTC)

Replies from Samir

Thanks for your comments and questions, Alex. I'm happy to answer any questions you or the GAC might have.

1. By 25 new submissions I mean 25 participants. Also added creating +50 new articles to the measures of success.
2. The period in which participants will implement their goals + the assessment period will take more than 2 months. We also intend to extend the contest in the next rounds to take 3 months at least if the contest proves success. Also the jury will put into account the user's behavior (e.g. their talk pages) before and after the contest and the featured and good articles nominations takes months which will be another period for the community to judge their work in a longer period of time.
3. Nominated featured articles usually take months (4-5 month) for users to vote and comment on them. The jury will assess participants' work depending on developing and nominating articles and will not wait until they're accepted. The nomination page for each article usually shows if the article is more likely to be accepted or not.

Hope that this answers your questions. Please let me know if you have any further questions. -- Samir El-Sharbaty ^{(talk to me)} 22:44, 26 September 2014 (UTC)

Thanks for this explanation. It will be good to check in even after the grant reporting period (60 days after the end date of the grant) to see the editing behavior of the participants and discuss ideas for improving retention the next round. Alex Wang (WMF) (talk) 14:28, 27 September 2014 (UTC)

More so, if I may interrupt at the WMF section for a second; I think distributing prizes is a good short term effort, and I'll perhaps go one step further than what Alex said, and say that although I support this, I do agree that you should not build on having a competition with prizes like this every year; In my very humble opinion I think that you should work during this time to figure out other motivations for volunteers to create quality content on AR wikipedia, besides prize money. I can offer some ideas if you like...

Aggregated feedback from the GAC (to be completed at the end of the discussion)

Key:

• 1 = very weak or no alignment
• 3 = weak alignment
• 5 = passable alignment; room for significant improvement
• 7 = reasonably good alignment
• 10 = excellent alignment

Scoring rubric	Score
(A) Impact potential Does it focus on one (not all) of Wikimedia's strategic priorities? Does it have potential for impact on Wikimedia projects through new or improved content? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	8
(B) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	7
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Do the participants have the necessary skills/experience? Are there any external factors that may promote success?	8
(D) Budget quality Does the budget match the program scope? Is there strategic justification for material costs (printing, merchandise, etc.)? Does the budget reflect responsible growth for repeat grantees or a reasonable first investment for new grantees?	8.5
(E) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support? Does it support diversity?	7
Comments from the GAC: It's great to see an initiative aimed at increasing participation by women, improving content around female issues, and targeting new users. Online writing competitions are low-cost and high-impact! The project idea would be strengthened by a longer-term plan. We realize this is a pilot competition and will be iterated on in the future, but there also needs to be a plan to engage these new participants going forward, perhaps through some type of on-wiki mentorship. There is the risk that relatively high prize amounts will engage participants during the contest who will not contribute after receiving a prize (this has been seen as a challenge in Wiki Loves Monuments). Therefore, having a longer-term plan is important.