What longer term impacts do we see from the content generated by Wiki Loves Monuments? What do we observe of program participants?
We examined content production and quality improvement, user recruitment and retention, as well as measures of replication and learning as core outcomes across programs.
Read this page to learn how Wiki Loves Monuments delivers in three potential impact areas.
The average national Wiki Loves Monuments contest leads to just over 5,600 photos uploaded to Commons during the month of September. For Wiki Loves Monuments 2012, the average number of photos placed in Wikimedia projects was 885.
We collected photo upload counts for all 27 WLM implementations. Upload counts ranged from 1,572 to 32,987 (Graphs 2 and 3). The average event uploaded 5,635[2] photos during the month of September 2012 or 2013.
Add a callout here:
Small text
One two three
Some shorter thoughts to see if this shrinks.
Regarding image use:
Wiki Loves Monuments 2012 2013 data for image use ranged from 387 to 6,103 unique images used. The average number of images used from a 2012 2013 Wiki Loves Monuments event was 885.[3]2012 also shows us that the more images uploaded, the more images are used (see Graph 4).
Wiki Loves Monuments 2013 2014 data for image use ranged from 10 to 1,687 unique imaged used. The average number of unique images used from a 2013 2014 Wiki Loves Monuments event was 262[4][5] (see Graph 5).
Do note that the numbers reported for 2013 are not final. Since the event ended around the time that we were collecting data, we expect image use in Wikimedia projects to rise.
The average Wiki Loves Monuments photo upload costs almost 50 cents USD. However, more money spent on a national Wiki Loves Monuments event does not necessarily relate to more participants or more image uploads.
Based on reported budgets for Wiki Loves Monuments implementations and the actual photo uploads, the amount of dollars spent for each photo uploaded ranged from $0.09 USD to $9.17 USD each. The average implementation cost $0.48 USD[6] per photo. We also looked at how much it cost in terms of Wiki Loves Monuments photos which were used on Wikimedia projects. For 2012 implementations, the cost per used photo ranged from $0.51 USD to $26.51 USD, averaging $2.43 USD[7] per image. There does not seem to be a direct relationship between budget, participant count, or images uploaded (see Graphs 2-5).
As illustrated in the graph, budget inputs ranging from less than $1,000 USD to almost $17,000 USD. The number of participants ranged from 75 to 2,005, while the number of images added ranged from nearly 2,000 to more than 30,000. The varying sizes of the bubbles — with larger bubbles representing more images uploaded — show that the number of photos increase significantly when events have over 500 participants. There does not seem to be a direct relationship between budget, participant count, or images uploaded. The bubble size doesn't get larger or smaller — meaning when more money is invested in an Wiki Loves Monuments implementation, that doesn't mean the event will have a higher participant count or a higher upload count.
As illustrated in the graph, budget inputs ranged from less than $500 (USD) to nearly $30,000 (USD)[8] and number of participants ranged from as few as 67 and as many as 1485. The number of photos/media added (illustrated by bubble size and label in the graph) ranged from more than 15,00 to nearly 25,000. It appears from the varying sizes of the bubbles on the graph (larger bubbles representing more photos added) that the number of photos added increases along with the number of participants. We also see that event participant counts had a much more stable range of participant numbers for the majority of events, with the median 50% reporting between 156 to 208 participants.Unlike the graph for 2012, we see that the largest bubbles, representing the greatest number of photos added, seem appearing at lower budget levels.
Graph 4: Budget, photos added, and unique photos used for 2012.
This bubble graph compares the number of photos added with the budget, using bubble size to show how many unique photos were added to Wikipedia pages for 2012 competitions. We generally see that as the number of total photos added increased, the number of photos used on Wikipedia pages also increases.
Graph 5: Budget, photos added, unique photos used for 2013.
This bubble graph compares the number of photos added with the budget, using bubble size to show how many photos were added to Wikipedia pages for 2013 competitions. There has not been sufficient time after the end of the competitions to allow photos to be uploaded to Wikipedia pages, showing that the largest number of photos used is only 1,687, even though total photos uploaded is around 25,000.
This boxplot shows the distribution of 2012 and 2013 Wiki Loves Monuments competitions. In comparing between the two years, we see that the dollar cost per photo range was greater in 2012 than in 2013; however, the median cost was lower for 2012 at $0.37 per photo added. This suggests that spending per photo between both years may not be very different. More data would be needed to confirm.
References
↑Note: Although "content production" is a direct product of the program event itself and technically a program output rather than outcome most of the program leaders who participated in the logic modeling session felt this direct product was the target outcome for their programming. To honor this community perspective, we include it as an outcome along with quality improvement and retention of "active" editors.
↑Note: This implementer also had the largest budget in 2012 and did not use the full amount so they applied the remaining budget to their 2013 event budget. It is unknown whether the full budget will be allocated to event expenses.
Photo quality ratings
Only a very small percentage (0.8%) of Wiki Loves Monuments 2012 images are rated as Quality, Valued and/or Featured Images on Wikimedia Commons.
This section focuses only on images uploaded for Wiki Loves Monuments 2012, since the 2013 competition had was just ending as we finished of our data collection; not enough time had passed for ratings to have been awarded at that time.
Add a callout here: Importantly, that the majority of images rated Quality and Valued generating from only one or two contests, indicating that those program leaders are likely doing something that others are not in order to achieve these ratings.
By November 2013, about 13 months after the 2012 contests ended, out of the 12 contests examined, five (42%) had Quality images, three (25%) had Valued images, and most (7 contests, 58%) had Featured Pictures associated with them. Out of the 126,424 photo uploads we examined for Wiki Loves Monuments 2012, 994 (0.8%) were rated as a Quality Image, Valued Image, or Featured Picture (see Graph 8). Out of those 994 rated photos[1]:
932 (94%) were rated as Quality Images. Reports of images rated as Quality ranged from 0 to 484 with the majority (91%) of Quality images generated by only two contests, one which contributed 484 (52%), and another which contributed 364 (39%).
30 (3%) were rated as Valued Images. Reports of images rated as Valued ranged from 0 to 27 with the majority (90%) of Valued images generated by one contest.
32 (3%) were rated as Featured Pictures. Reports of Featured Pictures ranged from 0 to 17 and were associated with several of the contests.
This pie graph compares how many photos were rated to how many were unrated out of all the photos that were uploaded during Wiki Loves Monuments events. The number of rated photos is about 0.08% of the total number of photos uploaded for all Wiki Loves Monuments in 2012.
This pie graph compares only the photos that received a rating of Quality, Valued or Featured image for WLM 2012. Of all the photos that received a quality rating, 94% received a Quality rating, 3% were rated as Valued images, and 3% were rated as Featured images. Due to the method for which ratings are granted to images, each photo may be included in more than one quality category.
References
↑Some photos may have been rated in multiple categories.
Recruitment and Retention of Editors
As of reporting time, we were only able to evaluate recruitment and retention of active users for 20122014?. A few things to remember with this section:
An active retained user is considered one that makes 5 or more edits a month. [1] We examined both the "pages created" (a new page gets created when someone uploads a file) and "edit count" using Wikimetrics to measure user activity on Wikimedia Commons.
A "survived" editor is defined as one that made at least one edit in a given window of time after the event ended.
User retention was measured for both survival and active user retention. Retention was assessed for the first three months after the event (Quarter 1) and the second set of three months (Quarter 2: months four, five, and six) following the event. Here we considered users as surviving if they had made a single contribution, and "active" if they had averaged [2]5 or more edits a month, during the follow-up window.
We couldn't isolate only Wiki Loves Monuments uploads so user contributions and retention numbers are based on any contributions made by participants during September 2012, and the three and six month retention dates.
Three month follow-up represents the first quarter following the event, from October to December 2012.
Six month follow-up represents the second quarter following the event, from January to March 2013.
↑by counting those who had made 15 or more edits for the three month quarter. However, users could make 15 or more edits during any period of time, not five or more each month necessarily. This solution allows us to even much of the month-to-month variation of users to get at those users who are more or less staying active on the project. For example someone who made five edits in the first month, three edits in the second month, and six edits in the third month following the event would be counted as active for the full three month quarter. At the same time, someone who made all their contributions in the first month following the contest, and nothing further, would also be counted as surviving or active for the quarter.
New users
Six months after the end of Wiki Loves Monuments 2012, 0.4% of new users were retained as active contributors to Wikimedia Commons.
Due to Wiki Loves Monuments 2013 ending recently, recruitment and retention data for only 2012 competitions is reported here. Our retention period of 3 months examines the number of uploads over the 3-month period from October to December 2012 while the 6-month retention period lasts through March 2013. The count of uploads per user is pooled; that is, the number of uploads from October through December of 2012 are counted together. Thus, in order to measure 5+ edits per month, we actually use a limit of 15 edits over three months for the 3- and 6-month follow-up point.
For these data, we used Wikimetrics to calculate the number of pages created in namespace 6 for files of Wikimedia Commons. The pages created option in Wikimetrics includes not only photos uploaded, but may also count videos and audio files added. We are unable to isolate just photos uploaded from these other possible media types, so user activity may include these as well. We also could not isolate photos uploaded for Wiki Loves Monuments, versus for other reasons, during the month of September so that the counts associated with the month September are not necessarily photos for the contest. For instance, in 2012, this made for the observation of 399,123 pages created by the full cohort of new and existing users while the actual number of photos uploaded specifically for Wiki Loves Monuments 2012 was 365,322, 8.4% of the pages created by the cohort of contributors were not photos for Wiki Loves Monuments. In addition to pages created, we also examined Wiki Loves Monuments participant behavior in terms of their edit count on Commons. Edit counts may include creation of a new file page, edits to category or other information on an existing file, or new versions of the image that may be uploaded.
13,179 new users[1] participated in Wiki Loves Monuments 2012. 5,399 (41%) of these uploaded five or more files and 5,552 (42%) made five or more edits in September 2012. In regard to retention (see Graph 9):
797 (6%) of new users from Wiki Loves Monuments 2012 had made at least one edit and 627 (4.8%) had uploaded at least one new file to Wikimedia Commons.
Active (Average 5+ contributions a month)
152 (1.2%) of new users from Wiki Loves Monuments 2012 uploaded 15+ files during the first quarter of retention follow-up.
221 (1.7%) of new users from Wiki Loves Monuments 2012 succeeded at making 15+ edits during the first quarter of retention follow-up.
220 (1.7%) of new users from Wiki Loves Monuments 2012 had made at least one edit and 187 (1.4%) had uploaded at least one new file to Wikimedia Commons.
Active (Average 5+ contributions a month)
47 (0.4%) of new users from Wiki Loves Monuments 2012 uploaded an average of 15+ files during the second quarter of retention follow-up.
59 (0.4%) of new users from Wiki Loves Monuments 2012 succeeded at making 15+ edits during the second quarter of retention follow-up.
Although new user activity declined over the three and six month follow-up periods, during the month of Wiki Love Monuments in 2013, there was an increase in activity for the new users from Wiki Loves Monuments 2012. In September 2013, 451 of the previous new user cohort (3.4%) contributed photos/media to Commons. Of those previously new users which were active that month, 306 (68%) added five or more files during the 2013 contest time.
This bar graph shows the number of new users retained for periods of three months at three and six months following the event, as well as during Wiki Loves Monuments 2013. A total of 13,179 new users participated in Wiki Loves Monuments 2012. Interestingly, measuring activity by edit count captured significantly more active users in the initial months following the contest, however, the added value of the edit count metric diminishes over time for new users.
References
↑New users were defined as WLM 2012 uploader usernames which had not contributed to commons in the previous year.
Existing users
Not surprisingly, existing editor retention is higher than new user retention for Wiki Loves Monuments, with 29% actively uploading files and 57% actively editing on Wikimedia Commons six months after the event ended. Half of the existing users who participated in Wiki Loves Monuments 2012 also participated in Wiki Loves Monuments 2013. The number of returning existing contributors seen for Wiki Loves Monuments 2013 was, for the most part, replenished by those Wiki Loves Monuments 2012 contributors who were retained as near the same number of existing users who participated in 2012 did not return for 2013's competition.
1,632 existing users participated in Wiki Loves Monuments 2012 2013. 1,236 (76%) of these uploaded five or more files and 1,307 (80%) made five or more edits during the contest in September. In regards to retention (see Graph 10):
1,073 (66%) of existing users from Wiki Loves Monuments 2012 had made at least one edit and 975 (60%) had uploaded at least one new file to Wikimedia Commons.
Active (Average 5+ contributions a month)
566 (35%) of existing users from Wiki Loves Monuments 2012 uploaded 15+ files during the first quarter of retention follow-up.
759 (47%) of existing users from Wiki Loves Monuments 2012 succeeded at making 15+ edits during the first quarter of retention follow-up.
934 (57%) of existing users from Wiki Loves Monuments 2012 had made at least one edit and 845 (52%) had uploaded at least one new file to Wikimedia Commons.
Active (Average 5+ contributions a month)
477 (29%) of existing users from Wiki Loves Monuments 2012 uploaded an average of 15+ files during the second quarter of retention follow-up.
643 (39%) of existing users from Wiki Loves Monuments 2012 succeeded at making 15+ edits during the second quarter of retention follow-up.
Half of the cohort of existing users (811 users) contributing to Wiki Loves Monuments 2012 also uploaded media during Wiki Loves Monuments 2013. Of those 652 users (80%) uploaded five or more new files to Wikimedia Commons during that time. While new editor retention appears low, compared to existing editor retention, the number of new users that returned for Wiki Loves Monuments 2013 (457 previously new users) helps to replenish the pool of returnees for 2013.
This bar graph shows the number of existing users retained for periods of three and six months as well as during the following Wiki Loves Monuments. There were a total of 1,632 existing users of Wiki Loves Monuments in 2012. Measuring activity by edit count captured significantly more active users than did the "pages created" metric, capturing 4 to 12% more active users at each measurement point.
![Graph 3: Budget, participation, and photos added. As illustrated in the graph, budget inputs ranged from less than $500 (USD) to nearly $30,000 (USD)[8] and number of participants ranged from as few as 67 and as many as 1485. The number of photos/media added (illustrated by bubble size and label in the graph) ranged from more than 15,00 to nearly 25,000. It appears from the varying sizes of the bubbles on the graph (larger bubbles representing more photos added) that the number of photos added increases along with the number of participants. We also see that event participant counts had a much more stable range of participant numbers for the majority of events, with the median 50% reporting between 156 to 208 participants. Unlike the graph for 2012, we see that the largest bubbles, representing the greatest number of photos added, seem appearing at lower budget levels.](/w/index.php?title=Special:Upload&wpDestFile=Budget_Participation_Photos_Added_2013.png){kind=link}
