Learning and Evaluation/Evaluation reports/2015/Other Photo Events/Key findings

Other Photo Events

Summary

Limitations

Inputs

Outputs

Outcomes

Key findings

Appendix

What did we learn about Other Photo Events in terms of impact?
We examined the results in terms of the stated priority goals program leaders shared in their reporting as well as outlined critical next steps toward program learning.
Read this page to learn the most important takeaways and recommended next steps.

How does the program deliver against stated goals?	The four most commonly cited priority goals for the photo events were to: increase accuracy and/or detail of information, increase contributions, increase diversity of information coverage, and increase volunteer motivation and commitment. Increasing accuracy and/or detail of information Ten percent of media uploaded was used on over 33,000 Wikimedia project pages. If increasing accuracy continues to be a major priority goal, capturing data on information needed or corrected will be instrumental in tracking the success of this goal. We encourage further investigation into the relationships between style of event, targets for content upload, content use, and participation so that we might learn how to design toward this goal. Increasing contributions The photo events examined in this report generated just over 126,544 media uploads; 12,187 (10%) of which have already been linked to over 33,000 Wikimedia article pages. Media uploads captured in this reporting represents nearly 3% of all media uploaded to Commons by registered users during the reporting period. Media uploaded for these events has been used in articles at more than six times the rate of commons uploads overall during the same time period. Increasing diversity of information coverage Contests reported here were held in 19 different countries and engaged new users at a rate of two new users to each existing user participating. If increasing diversity continues to be a central goal of photo events it will be important to develop and capture additional measures of language and content diversity. Increasing volunteer motivation and commitment Participation in the examined events included nearly 4,000 Wikimedia users (362 existing active users, 866 existing non-active, and 2,741 new users). Of the new users generated through the examined contests, 1.2% survived at three-month follow-up. Still, no data regarding motivation and commitment were reported by program leaders. We anticipate this will change with motivation as one of the newly incorporated Global Metrics for Wikimedia Foundation grantees. While a small fraction of new users were retained, only 6 events selected «Recruiting and/or converting new users» as a priority goal. Of those 6, only 2 had new users participating and they each retained 0.7% and 14.0% of those new users. These 103 photo events demonstrated high retention of existing editors. More investigation into how targeted photo events may motivate existing users and encourage them to contribute more regularly could be very useful for planning and targeting events participants. References
How this information can apply to program planning	Planning for Program Inputs & Outputs Use the information to help you in planning for program inputs and outputs. The numbers evidenced by others can help you to know what the right combination of participants might be for the content outcomes you are hoping to achieve. The data presented in this report suggest that small-scale, thematically-specific events are associated with more existing users which are ‘’active’’ editors. Further, these events higher media use rates, on the other hand, larger events with more new editors have more participating editors and more media uploaded overall. When planning a photo event, it may be useful to try to target one of these groups, or think about how your event can integrate both participant groups in a way that is advantageous for increasing use and ensuring high quality. This information may be especially useful in budgeting and setting reasonable targets Planning event budgets based on the budgets presented here would have many pitfalls: differences in event length and number of participants, local costs, and even event style affect budgets. For example, a small photo event may comprise of lending a Wikimedian photographer a camera so they can go to a cultural festival and upload the photos they take to Commons. If the event organizer already owns the camera, the event may incur no cost, while purchasing a camera would be a significant expense for another leader who does not already have the resource. Reach out and connect to other leaders. To avoid surprises, when using budgets presented here for planning purposes, try to find an event in a location with a similar economy to your area and consider reaching out to successful program leader to discuss potential resource needs (including possible budget or donated resources). Alternatively, you can find an event based on the same template in different location (ie, Wikiexpeditions or WikiTown) and talk to the program leader about the costs incurred before translating those expenses into local prices. Use the distribution statistics as guardrails against costly plans that may not produce scaled results. The boxplots illustrating cost per participant and cost per media uploaded can also be helpful references, if, as with overall budget information, they are taken in the context of each event. If planning a new program, you might expect the costs to fall within middle 50% of costs per output reported (ie, within the green bar on the boxplot.) As programs move down the boxplot they create better outcomes with fewer inputs. We hope, as we continue to evaluate programs and feed the results back into program design, that we can learn more from the programs achieving the most impact using the fewest resources.
How easily can the program be replicated?	Over the years, the global event team was very effective in continuously improving and promoting the concept of Wiki Loves monument, which led to a growing number of countries joining that particular competition, but also in spinning off the model into other national or small local photo events as we have outlined here. Currently the program evaluation and design team members are working to more deeply examine a selection of the most impactful events and reaching out to learn more from program leaders. With the knowledge we are able to gather we will develop a program toolkit specifically for coordinating the stories, resources, and advice for how to plan, run, and evaluate photo contests and events. Look for it later in 2015.
How does the cost of the program compare to its outcomes?	Our cost-benefit-analysis comprises of data from Wiki Loves Monuments and four other photo event implementations reporting non-zero dollar budgets ending between the start of September 2013 and end of September 2014 which had reported budgets. Looking at the cost per uploaded image from the combined reporting, we get to result that for those events with non-zero dollar budgets reported for Wiki Loves Monuments and other photo events . Year Number of countries Average budget (median) Average number of uploaded images (median) Average cost per uploaded image (median) Wiki Loves Monuments 31 $4,618 4,969 $0.90 Other Photo Events 4 $3,757 7,515 $1.17 As photos that are being used in Wikipedia articles reach a much larger audience than those that are not included in articles, we also looked into the number of images that are currently being used in Wikimedia articles (as of February, 2015) to see if this demonstrated increased use. As outlined in the table below, Year Total number of photos uploaded Photos in use on Wikimedia sites Percentage of images in use Cost per used image Wiki Loves Monuments 240,172 240,172 10% $6.54 Other Photo Events 27,788 1,014 4% $30.92 While photo events overall tend to deliver media that gets a high rate of use on Wikimedia articles, for some reason, those four events that reported non-zero dollar budgets did not all see a high rate of use and some one even had especially high costs associated with a more costly media event. Importantly, there were only non-zero dollar budgets for four of the events as most budgets reported were zero dollars. It is critical to note the variety of events and difference in resourcing that points to very different program design models associated with them. Notes
Next steps	Join the conversation! Visit the report talk page to share and discuss: Questions about Evaluation and Impact What, if any, ideas do you have about other ways we should evaluate programs? What questions around program impact or evaluation do you have after reading the reports? What further data investigations would you like to see (or do!) for these programs? Questions about Measures What, if any, measures have you used that are missing from these reports? What, if any, tools/bots/programs/strategies do you use to measure the outcomes of your programs? Questions about Sharing If you ran a program that delivered excellently against goals, speak up! Consider writing a blog or how-to guide highlighting your ideas on why your program was so successful. If your program surmounted a particularly tricky problem in program design, consider writing a learning pattern! If you have run a program and want to report key metrics to the Learning and Evaluation team, our collector is always open. Visit our reporting page to learn about the reporting forms contents and find the link to voluntary reporting. Questions about Connecting If you are considering running a new program or updating an existing one, consider reaching out to experienced program leaders who have organized a similar program. Join the program leader mailing list of weekly updates about program evaluation, tools, etc. Notes