Phase 1
Reviewing
  • Each reviewer should get access to a reviewer dashboard like the one presented below.
  • Submissions are members of a wiki category, potentially "IEG Tool Grants Applications 2014 R1" or something. There is a few such topical categories for each round.
  • The 'approved' column should contain links to the scoring form for the relevant submission.
  • The scoring form contains several single choice questions (score from 1 to 5 for example) and a comment box.
  • Only one person should see the scoring form of him/herself.
Phase 2
Aggregated anonymous output
  • After the reviewing period ends, the tool should produce output for each submission: average score for each form question and a list of anonymous comments.
  • Such output should not be public. It should be accessible by a administrative person who leads the reviewing committee.
Phase 3
Semi-automated posting
  • Optionally, the tool should have a button to post the averaged data to the tools' talk pages once the previous phrase completes (by manual click of a button).

Example of a reviewer dashboard

edit
Tools
name description approved? requested amount note
Grants:IEG/Map atlas a short description from the idea page (first template) not scored $1,000
Grants:IEG/Gadget to do Foos and Bars a short description from the idea page (first template) yes $10,000 should talk to candidate about X
Total (Tools) Total amount you funded. $10,000 1 scored, 1 not yet


Outreach
name description approved? requested amount note
Grants:IEG/More ponies in user preferences . no $55,000 exorbitant amount, no clear goal
Grants:IEG/Reach Bazz @ Belarus a short description from the idea page (first template) yes $35,000 should talk to candidate about X
Total (Outreach) Total amount you funded. $35,000 1 scored, 1 not yet


Total
$45,000 of $82,000 this round.