Grants talk:PEG/Criteria
Review and proposal for new structure of criteria edit
Alex, Asaf, and GAC members:
I've gone through the criteria without making any substantive changes. Purely as a collection of criteria, it's looking good; but I have a major conceptual issue and a few minor queries.
The edited criteria are duplicated here in green for easy reference as you read my comments below:
The Grant Advisory Committee and WMF staff review applications based on the following criteria:
- The fit with the mission and strategic priorities of Wikimedia.
- The potential for impact in the targeted Wikimedia projects (e.g. Spanish Wikipedia, Wikimedia Commons). What would the activities improve? What would be created or changed thanks to the work enabled by the activities
- The availability and readiness of volunteers and the availability of other non-financial resources required to implement the activities. Would the applicants be reasonably likely to be able to deliver the expected results? Is the scale commensurate to the size of the team and volunteer base? Has this been demonstrated?
- The track-record of the applicant(s) in managing grants (where applicable).
- The degree to which the actitivies would add new knowledge to or spur innovation in the Wikimedia movement (ensuring that the activities are not based on models already proven ineffective).
- How efficiently funds would be used.
- The quality of the proposed measures of success. Are they actually measurable?
- The sustainability of impact beyond the duration of the activities. Would the work keep on giving? Would the new infrastructure/capacity fuel subsequent work?
As so often with such judgmental criteria, the extent to which all criteria are mandatory is unclear. Some of them look as though they'd be expected of all proposals (1, 2, 3, 6, and 7); others look as though they'd apply only for some proposals—that they'd be regarded favorably if appropriate to a proposal, but that their absence wouldn't necessarily worsen one's chances of getting the funding (4, 5, and 8).
In 4, the "where applicable" quality is explicitly mentioned, but not in the others. This mandatory vs where-appropriate binary seems to be complicating the relationship between the strategic goals, these criteria, and the rubric. Why not structure the eight criteria into a binary system, for greater clarity? Something like this:
The Grant Advisory Committee and WMF staff review applications based on the following criteria:
Essential
- 1. The fit with the mission and strategic priorities of Wikimedia.
- 2. The potential for impact in the targeted Wikimedia projects (e.g. Spanish Wikipedia, Wikimedia Commons). What would the activities improve? What would be created or changed thanks to the work enabled by the activities?
- 3. The availability and readiness of volunteers and the availability of other non-financial resources required to implement the project. Would the applicants be reasonably likely to be able to deliver the expected results? Is the scale commensurate to the size of the team and volunteer base? Has this been demonstrated?
- 4. How efficiently funds would be used.
- 5. The quality of the proposed measures of success. Are they actually measurable?
Where applicable
- 6. The track-record of the applicant(s) in managing grants
(where applicable). - 7. The degree to which the activites would add new knowledge to or spur innovation in the Wikimedia movement (ensuring that the activities are not based on models already proven ineffective).
- 8. The sustainability of impact beyond the duration of the activities. Would the work keep on giving? Would the new infrastructure/capacity fuel subsequent work?
A few more points:
- Problem sentence in the lead. Even the lead awkwardly straddles the binary division:
"Grant requests should support Wikimedia's mission and strategic priorities. The WMF favors high-impact, innovative requests, and seeks to increase each grantee's capacity to develop and execute new programs and partnerships."
It seems to me that high impact is more generally applicable than innovation (the latter falls into the category of "a bonus if we see it, but not every top-class proposal will actually be innovative per se"). Normalising innovation (= new and/or cool, and/or a bit clever) as a requirement might scare people off, or worse, encourage them to trivialise the meaning of innovation; I've seen this in a proposal already ... workaday things labelled as innovative for the sake of it. Perhaps it's good enough to rely on Item 7.
As well, the second half of the sentence in question doesn't seem to cohabit with the first half very well ("and seeks to increase each grantee's capacity to develop and execute new programs and partnerships"). If you agree, new wording could be developed.
- Criterion no. 1. The target for the strategic priorities link is going to appear bizarre to most people. What exactly is that page? I've no idea.
- Criterion no. 2—one size fits all? No. 2 is in a grey area, since it's often hard for conferences, certain other event, and the FDC-like operating grants that increasingly show up for PEG money, to point directly and specifically to impact on the WMF's online projects, or frankly anything else. Maybe this is why applicants for conferences and annual grants find themselves inserting vague boilerplate text to pay almost automated lip-service to the stated criteria/priorities, mission—it's boring for reviewers, and wastes applicants' time too. What to do about No. 2? I don't know. Perhaps it can be resolved with additional wording in the bullet, or perhaps a footnote. I haven't thought through the possible wording. Do you see a problem, too?
- Criterion no. 7. "The degree to which the activites would add new knowledge to or spur innovation in the Wikimedia movement (ensuring that the activities are not based on models already proven ineffective)." Does the bracketed bit need to be there? It seems to say that if your activities are not based on an ineffective model, you're being innovative or adding new knowledge. This is not logical, and might also confuse translators.