Grants talk:APG/Proposals/2012-2013 round1/Wikimedia UK/Progress report form/Q1

Hello community.

Our report IS quite long but we are doing lots of things. We would love to answer any questions you might have or receive your comments. Thanks to everyone who contributed to it. Jon Davies WMUK (talk) 11:12, 3 May 2013 (UTC)

Link to staff summary

We have published a staff summary of all 2012-2013 Round 1 Q1 reports here: FDC portal/Proposals/2012-2013 round1/Staff summary/Progress report form/Q1. We ask each entity to also review our general comments there. All the best, KLove (WMF) (talk) 23:59, 10 June 2013 (UTC)

Appreciation

We appreciate the time and effort you have devoted to preparing this detailed progress report. Here are a few activities we especially appreciated:

• Congratulations on placing three Wikipedians-in-Residence! We look forward to hearing how you support them and create a network of WiRs in the coming months.
• We were impressed to learn that your GLAMWiki conference attracted over 200 people. We hope to learn more about the impact of this conference in future reports and in particular how many attendees of this conference are active in the movement in the future.
• We would like to congratulate you on the creation of a five-year plan through a collaborative process.

With thanks, KLove (WMF) (talk) 23:59, 10 June 2013 (UTC)

We would like to learn more

We’re interested in learning a little more about a few things you’ve mentioned in this report.

• When describing “Program 12” in this report, you mention, “Project grants to UK community for developing work.” Could you provide us with a few details about these projects or some links to project descriptions? We are interested in learning more about your technical grants now.
• We appreciate that WMUK is reassessing how it will address low volunteer participation in some of its activities. Please keep us updated in future reports as your strategies develop.
• Please link to the documentation put in place for the Train the Trainers program from this report or this discussion page so that we may review it now.

Thank you for the additional details. We sincerely appreciate them. Cheers, KLove (WMF) (talk) 23:59, 10 June 2013 (UTC)

• Program 12 was intended to be used to grant money to the community for them to develop small software tools with. However, we haven’t made any progress with this budget in Q1, and we will be performing a review to see if the money can be spent better in other places.
• We will adjust our reporting to include this in future. We hope that the introduction of the VLE will help with this.
• We do appreciate that people will want to learn from this programme, as has been with the case of Wikimedians in Residence. And we do facilitate this where possible, for instance with WiRs we were able to release an example agreement so that other chapters can see what we’ve done. Unfortunately, we can’t share the documentation drawn up for the Train the Trainers programme publicly, as it contains commercially sensitive information. We will, however, release the results of the review of the Train the Trainers programme later in the year. Richard Nevell (WMUK) (talk) 14:22, 5 July 2013 (UTC)

Thanks for these comments and updates. They are much appreciated! KLove (WMF) (talk) 17:42, 10 July 2013 (UTC)

Suggestions for future reports

We found it difficult to understand WMUK’s progress from reading this report because of this report’s confusing approach to reporting spending by program area in a way that does not align with its proposal form and because metrics for many activities are not reported. In future reports, we request that WMUK report on its spending in a way that corresponds to the proposal form and include metrics in its reports. We know the reporting process can be time-consuming and so we thank you in advance for the improvements. All the best, KLove (WMF) (talk) 23:59, 10 June 2013 (UTC)

Reporting spending by program area

FDC staff cannot understand WMUK’s progress on spending from reading this report because programs are not presented in a way that matches how the budget is presented in the FDC proposal form. While for some programs in your report you list corresponding program areas from the FDC proposal, for other programs you do not.

Just a note that we realised this would be a problem and that is why we put the FDC reference in the banner ahead of each report line. Hopefully with the new 'magnificent seven' categories we will not be so confusing! Jon Davies WMUK (talk) 08:04, 11 July 2013 (UTC)

Thanks, Jon Davies WMUK, we appreciate this! KLove (WMF) (talk) 09:16, 11 July 2013 (UTC)

We tried to match these budget items to the programs listed in your proposal, but unfortunately found that they did not seem to correspond directly. For example, in future reports you can break down expenses as follows:

1. Community
2. Promoting free knowledge
3. Membership and public support
4. External relations
5. Finances
6. Governance, management and administration
7. Staff

Please note for future progress reports that additional details about spending by budget item are welcome but not required. We do need spending reported according to the categories listed above to understand your progress against your plan.

Thank you, KLove (WMF) (talk) 23:59, 10 June 2013 (UTC)

Reporting on metrics

We noticed that you’ve listed metrics for many programs, but have not actually included them in your reports on these activities. For example,

• Program 11 (GLAM and Outreach Events): Number of institutions, number of people attending activities, number of people remaining active after n months; All contact requests from cultural institutions answered.
• Program 13 (Extended reach): Number of people taking part, new editors trained; Number of events; Follow up on their engagement; A monthly report collated from either filled in ‘event organiser’ feedback form or informal feedback from the volunteers
• Program 14 (Wikimedians in Residence): number of people attending related activities
• Program 19 (University Outreach): number of people attracted and still engaged; Number of institutions collaborated with.
• Program 22 (Microgrants): total spend on microgrants
• Program 23 (Travel grants): total spend on travel grants

In future reports, we would like to see metrics like this reported on to better understand your chapter’s work and progress.

Best, KLove (WMF) (talk) 23:59, 10 June 2013 (UTC)

• We worked with the British Library, the Senate House Library, and Camden Libraries through Andrew Gray. He delivered 16 workshops, with 107 attendees. We do not yet have figures for have many people remained active as editors over the long term - this is not information which is easy to generate from the tools available to us. GLAM-Wiki 2013 could have been included under this, but as it had a separate budget we treated it as a separate programme, which means the figures could arguably be considerably higher. The work of Wikimedians in Residence could also be included in this, but likewise was treated separately because it came under a different budget. Adjusting our reporting may help with this in future. All contact requests from cultural institutions were answered, and has been specifically addressed by hiring a GLAM Organiser.
• Our main activity in programme 13 surrounded International Women’s Day and Wiki Loves Monuments. A seminar was delivered to coincide with International Women’s Day, and was intended to raise awareness; we do not know if it resulted in people signing up as it was not an event where people were editing. WLM involved preparatory stages and did not involve getting new editors trained. We did not manage to implement a monthly reporting system, however there will be developments to report on in terms of dealing with feedback in our Q2 report.
• At the end of April, our WiR at Tyne and Wear Archives and Museums had delivered events with a total of five lengthy one-on-one sessions with academics; he had been in post for a couple of weeks. We were unable to get figures for the number of people who attended events delivered by the WiR at the Natural History Museum and the Science Museum, as he was only part-way through his post and had not written reports on the majority of events when we were compiling the FDC report. The one for April can be seen here and there will be further reports included in Q2. Reporting will be much more detailed in Q2; the TWAM WiR has come to a conclusion, and the NHM&SM has done a lot more since.
• In Feb, Mar, Apr Wikimedia UK volunteers delivered four events to a combined audience of about 52 people, including heads of learning, attendees at education conferences, and members of the British Psychological Society. Three institutions were involved. Of these events, one was aimed at directly training new editors. We do not know how many of the six editors continued editing.
• According to the table under ‘spending during this quarter’, we spent £4.81 on microgrants (coming under ‘project grants’). We’ll need to explain why since we highlighted a lot of impact for the grants. Presumably it’s because spending was incurred the previous financial year in some cases while the results were delivered in Q1, while in some cases the expense has not yet been reclaimed. The output of the microgrant system was three Featured Articles on the English Wikipedia, two Featured Pictures on Commons, over 500 photographs uploaded to Commons, and nine core articles improved on the English Wikipedia.
• We spent £389.45 on travel grants in Q1. This mostly covered travel costs for volunteers to attend a hackathon in Amsterdam. Other grants have been allocated, but expenditure was not made in Q1. Richard Nevell (WMUK) (talk) 14:24, 5 July 2013 (UTC)

Response from Wikimedia UK

Thank you to the FDC for the time they have taken in reviewing the various reports of the Wikimedia chapters. Having written one, we appreciate that you have had a lot of information to deal with and in a wide variety of formats.

The comments across the chapters make interesting reading with some common themes - especially regarding metrics and reporting structure.

We were obviously aware that treating our internal budget headers as ‘programmes’ would create a very long report. In part this is because of the latitude given in the instructions for filling out the report, which gave us the freedom to construct the report as we saw fit. The decision was to make sure there was ‘too much’ information, rather than too little - but we made sure the budget headers corresponded closely to the key initiatives detailed in the bid to the FDC for funding in the first round. We adjusted the template to indicate exactly which part of the FDC bid they section referred to. Each budget supports a discrete activity. We note that you have made your desired format clearer by highlighting the seven points above, and we hope that we will be able to follow that in future reports.

In situations where there is perhaps information overload (and 37 programmes is a lot), important details can sometimes slip between the cracks. For instance, in programme 4 we detail our metrics as ‘’Value for money checked when ordering; All relevant events have merchandise available; Usage monitored’’. This was then accounted for in activities conducted: ‘’Stock replenished as required; Review the stock from 2012-13 (ongoing based on usage); Merchandise regularly used at events; Started to maintain detailed records of the stock to improve management’’.

We understand that this is a process in progress and we will be trying a different format for the report in the second quarter. We would like to note that it will take a considerable amount of work to adjust our internal reporting to match the FDC’s expected format, however, the report should be shorter so will be easier for WMUK to deliver in the long run. We would request that any changes to the format are kept to a minimum in future.

We do however feel that the statement “Corresponding metrics were listed in the proposal but were not reported on” is not entirely representative of our report. In some cases metrics were highlighted but not reported on, but the implication in that statement is that this was universally the case. Taking programme 14 as an example, highlighted in Talk:FDC portal/Proposals/2012-2013 round1/Wikimedia UK/Progress report form/Q1#Reporting on metrics, the metrics used to measure the performance of the programme were “Number of institutions, number of people attending related activities, number of people remaining active after 3 months”. In the next two sections we gave details of how many institutions were involved, and the number of people attending where this information was available. However, some information (attendance for some events and activity levels after 3 months) was not available. We should not have included activity levels as a metric because we don’t have that information at the moment, though we expect to have it later in the year. There is a similar situation regarding programme 19, where we identified the metrics as “Number of events that take place, number of people attracted and still engaged; Number of institutions collaborated with.” However, we did not have information on numbers still engaged, though we did report on events delivered and institutions involved.

Overall, metrics need to be reconsidered by Wikimedia UK. We are now ingraining in our staff and volunteers the importance of recording metrics at events - we are currently participating in the metrics conference in Hungary - and we hope that the Foundation will take steps to make it easier for us to track on-line metrics (for example, editor retention tracking) in future.

We note that there are parts of our report which do not provide metrics: however, we did provide them where possible, and we feel that should be reflected in the summary of our report.

Thank you for your thoughts and comments on these issues. I appreciate your thoughtful response and your sharing why you decided to create this progress report as you did. And thanks for your willingness to do it a bit differently the next time round, as I think it will help not only the FDC staff but also the community to more easily review and digest the report. And you are right, this is a process in progress, and I sincerely appreciate that it takes a lot of work to make your internal reporting match up to the FDC formats. I have heard and noted your concern about changing the format in the future. We don't want to make any unnecessary changes, as we realize the cost associated with a changing process.

The metrics question is a big one in the movement right now. I'm glad to hear WMUK attended the conference in Hungary, and I am keen to learn more about what comes out of that work. You're right that we are all learning right now about what impact means, what it looks like in a variety of contexts, and how we know when we're successful. Of course, each entity sets its own metrics, and I'd love to know more in the future about how WMUK is approaching this issue. I look forward to reading more in your reports down the road.

I want to note, too, that this summary of reports is one of many inputs to the FDC for their consideration. Thanks again for taking the time to respond here. Warm regards, KLove (WMF) (talk) 22:24, 27 June 2013 (UTC)