Fundraising 2011/Local testing/DE

Newest test requests

that´s it!

Completed Tests

Tuesday, 3.1.2012, 10:35 - 13:05 UTC, 100%, anons only

This is it, our last test of the campaign. With this final one, we had been looking for a successful banner for the last two days. We tested the urgent Jimmy appeal and the 5€-banner with and without donation progress meter. To stress out urgency and the forthcoming end of the campaign we tried a completely new banner copy: "Only 2 more days to go. Still € 100.000 needed. Please help!" It worked very well. Together with the 5€-banner which includes the donation meter it was the most successful one. We decided to run the countdown-banner because of the higher donation sum in the end.

The test results clearly show that urgency and the near realization of the goal of the campaign are highly motivating to make a donation. In our first tests of the progress meter we did not have a result as clear as this. That leads to the conclusion that the proximity of the goal is very much important for the efficacy of the donation meter. Another result is similiar to previous tests: Banners with a direct call to donate gain less clicks than the personal appeals but in the end reach more donations. However this is probably very much affected by the the actual phase of the campaign. --TobiasSchumann(WMDE) 12:49, 4 January 2012 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy jacket 5€	Jimmy jacket LP	1238196	3945	187	3168,00	16,94	0,00319	0,0001510	0,04740	0,002559	0,803042	26,28
Jimmy jacket urgent	Jimmy jacket LP	1237881	6979	136	2747,00	20,20	0,00564	0,0001099	0,01949	0,002219	0,393609	30,58
Jimmy jacket countdown	Jimmy jacket LP	1236686	4385	246	5221,00	21,22	0,00355	0,0001989	0,05610	0,004222	1,190650	25,62
Jimmy spacey 5€ thermo	Jimmy spacey LP	1237454	4277	246	4696,45	19,09	0,00346	0,0001988	0,05752	0,003795	1,098071	29,72

Test 23: Re-Testing Landing Page Picture

Wednesday, 28.12. 2011, 11:00 - 13:30 UTC, 100%, anons only

Are our tests only snapshots limited in significance or are the results universally applicable? Can we trust our results and are they valid over a longer period of time? Bearing in mind that we rely on these results when designing and conducting the campaign these questions are of high relevance. We were really curious how this test from Wednesday would end: We simply repeated an old test of November. In test 10 we analyzed the importance of personalization of the landing page. We found out that a picture of the speaker of the appeal improves the efficacy and increases the donation sum. Wikimedia UK afterwards tried that test as well but came to no clear outcome. Moreover, the WMF in 2010 tested a picture in the appeal and had a negative result. Indeed, there were some good reasons to re-test that test. So what is the outcome? We breathe a sigh of relief and now have more certainty about the validity of our test results. Like in the first test of the personalization of the landing page, this time integrating a picture generated more donations. There are now good reasons to believe that our results are generally applicable. --TobiasSchumann(WMDE) 12:33, 29 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy Royal Tannenbaum	without photo/banner	1132856	4964	89	2202,00	24,74	0,00438	0,0000786	0,01793	0,001944	0,443594	25,97
Jimmy Royal Tannenbaum	top banner	1134581	4975	86	2737,00	31,83	0,00438	0,0000758	0,01729	0,002412	0,550151	28,00
Jimmy Royal Tannenbaum	top photo	1132765	4873	107	3183,00	29,75	0,00430	0,0000945	0,02196	0,002810	0,653191	21,98

Tuesday, 27.12. 2011, 14:30 - 17:00 UTC, 100%, anons only

This test was about a fine adjustment of the successful 5€-banner. We wanted to know if we could raise the relatively low click-rate by changing the speaker or simply the picture of him. Thats why we tested the actual banner against an alternative photo of Jimmy and the successful speakers Brandon and Ryan. In fact the new speakers moved more people to click, whereas the alternation of Jimmy's picture made no difference. Unfortunately, the conversion of Brandon and Ryan was pretty low. Very interesting to see once again: not only the number of clicks is important. How successful the 5€-banner in general is, you can realize by comparing the numbers per impressions in all tests of the last months: with the new banner the value of that rate could be increased tenfold. That is a value we had been seeing only during the first two weeks of the campaign and before it started. --TobiasSchumann(WMDE) 12:45, 28 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy Jacket 5€ thermo	Jimmy Jacket LP	843239	2519	186	3703,00	19,91	0,00299	0,0002206	0,07384	0,004391	1,470028	17,11
Ryan 5€ thermo	Ryan LP	836324	2731	145	2139,65	14,76	0,00327	0,0001734	0,05309	0,002558	0,783468	15,97
Brandon 5€ thermo	Brandon LP	837329	2678	165	3185,00	19,30	0,00320	0,0001971	0,06161	0,003804	1,189320	18,60
Jimmy Spacey 5€ thermo	Jimmy Spacey LP	836889	2358	183	3508,00	19,17	0,00282	0,0002187	0,07761	0,004192	1,487701	22,98

Tuesday, 27.12. 2011, 09:00 - 10:45 UTC, 100%, anons only

As in our previous test this time we tried to find out if a succecful banner of WMF from last year's campaign would work as well in Germany: a reminder of the possibility to make a tax-deductible donation in 2011. Although the new banner did quite good the difference to the actual 5€-banner is profound. The 5€-banner another time reached a conversion of 10%, generated twice as much donations and had a much higher donation sum. --TobiasSchumann(WMDE) 12:45, 28 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy tax	Jimmy LP	812486	2345	89	2617,00	29,40	0,00289	0,0001095	0,03795	0,003221	1,115991	15,38
Jimmy 5€ thermo	Jimmy LP	812486	1996	201	4090,12	20,35	0,00246	0,0002474	0,10070	0,005034	2,049158	13,21

Thursday, 22.12.2011, 09:30-11:40 UTC, 100%, anons only

We definitely found the banner for the coming Christmas holidays. It's quite rare that a test has as distinct results as this one. We tested two different aspects of the banner. First, we wanted to know if a very powerful banner text of WMF last year campaign would work in germany as well. We tried the text "If every reader donates 5€ our campaign would end today" and compared it's effect with the "urgent appeal". On the other hand we added the donation progress meter as a variation to each of both banners to find out if the thermometer would function as a strengthener. All banners had an identical landing page.

The results are exciting: Although less people clicked the 5€-banner - the click-rate was the lowest we have ever measured in a test - these people were very much convinced to make a donation. 10-12% of the visitors on the landing page made a donation, a by far unparalleled conversion rate! It's no wonder that the average donation is much smaller, but all in all the new banner text leads to an incredible augmentation of donation numbers of 100%. Less clear is the effect of the progress meter. In one case it could improve the efficacy, in the other not. What is clear and interesting is that in both cases less people clicked the banner. But these are more likely to donate as the higher conversion rate shows. --TobiasSchumann(WMDE) 11:32, 23 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
if everyone 5€	Jimmy LP	700640	2181	226	3929,56	17,39	0,00311	0,0003226	0,10362	0,005609	1,801724	22,56
if everyone 5€ thermo	Jimmy LP	700900	1788	222	3949,00	17,79	0,00255	0,0003167	0,12416	0,005634	2,208613	13,14
urgent	Jimmy LP	700858	2940	96	2238,00	23,31	0,00419	0,0001370	0,03265	0,003193	0,761224	15,48
urgent thermo	Jimmy LP	700161	2259	107	2174,00	20,32	0,00323	0,0001528	0,04737	0,003105	0,962373	21,74

Test 19: test of various appeals

Wednesday, 21.12.2011, 10:30 - 15:30 UTC, 100%, anons only

Great! This test has a really terrific result as we think. For the first time an appeal out of the german community was the winner of the test, even though with a small advance. Katrin, a donor and commited in the wikimedia-universe, wrote an appeal about the importance of Wikipedia which we already tested a couple of weeks ago. Her appeal worked quite good, but her banner only moved a small amount of people to click. This proved the weakness of the old-type banner style with people in front of a white background. So this time we tried it with a new picture, shooted in the botanical garden, of course. Not suprising, the new banner had a much higher click-rate and outperformed the other banners so that Katrin's appeal got the most donations and the highest donation sum. For another time we remarkably can see how good the new concept of photographs works. However Katrin's appeal did not have a good conversion. The appeal of Ryan Kaldari, which already outperformed Jimmy' text in our tests, once again proved his power and had the best conversion rate. Finally the result is not very distinctive so that strictly speaking Katrin and Ryan are on the same level. We decided to put Katrin's appeal live to maintain the campaign diverse and exciting. Unfortunately the appeal of Ward Cunningham - we think it's amazing he wrote one - was not as good as Katrin and Ryan even though all appeals worked quite good. --TobiasSchumann(WMDE) 16:49, 22 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Alan green	LP Alan	1856858	6598	53	1938,00	36,57	0,00355	0,0000285	0,00803	0,001044	0,293725	20,00
Ward green	LP Ward	1861560	7275	70	2750,00	39,29	0,00391	0,0000376	0,00962	0,001477	0,378007	20,31
Katrin green	LP Katrin	1856880	12284	88	2888,00	32,82	0,00662	0,0000474	0,00716	0,001555	0,235103	17,57
Ryan green	LP Ryan	1859754	7565	84	2205,00	26,25	0,00407	0,0000452	0,01110	0,001186	0,291474	20,55

Test 18: Jimmy's urgent appeal

Tuesday, 20.12.2011, 11:00 - 15:00 UTC; 100%, anons only

Since we know that the days around Christmas and New Year are very important for the fundraiser we wanted to see, if a small strengthener can improve the effect of the banner. Would an "urgent appeal" bring an increase in click rate compared to the "personal appeal"? The answer is yes. In fact, this small modification in the banner text lead to a real difference. Considerably more people clicked on the banner and made a donation. --TobiasSchumann(WMDE) 16:49, 22 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy Jacket urgent	LP Jimmy Jacket urgent	3180155	15803	228	4965,24	21,78	0,00497	0,0000717	0,01443	0,001561	0,314196	30,26
Jimmy Jacket	LP Jimmy Jacket	3184272	12663	166	3011,00	18,14	0,00398	0,0000521	0,01311	0,000946	0,237779	26,67

Test 17: German editor and donor appeals Round 2 + Ryan Kaldari

Thursday, 15.12, 12:45 - 16:30 UTC, 100%, anons only

After we have not found a new editor appeal to beat Jimmy the day before we thought to do another round of testing. Especially, since we had a new appeal of ED Pavel Richter and a new picture of Ryan (finally in green). So we were excited, if we would end up with a new banner after the test. And so we did. The results are: Pavel and Attila had good clicks but very low conversion rates. Ryan did extremely good. His appeal already worked pretty good during his first test, but with the new banner he certainly reached Jimmy level. He had a similiar conversion rate and moreover outperformed Jimmy in clicks and donation numbers. Of course we decided to go on with Ryan and leave him up for the coming days. But around christmas we will have another round of Jimmy banners (probably) to head towards a good years end.

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Pavel	LP Pavel	1583377	9340	70	1799,00	25,70	0,00590	0,0000442	0,00749	0,001136	0,192612	19,67
Ryan	LP Kaldari	1582892	12253	123	3317,50	26,97	0,00774	0,0000777	0,01004	0,002096	0,270750	29,81
Attila	LP Attila	1583427	10459	51	1595,00	31,27	0,00661	0,0000322	0,00488	0,001007	0,152500	15,91
Jimmy Jacket	LP Jimmy Jacket	1585317	8158	78	2148,00	27,54	0,00515	0,0000492	0,00956	0,001355	0,263300	30,43

Till Mletzko (WMDE) 14:48, 16 December 2011 (UTC)[reply]

Test 16: German editor and donor appeals

Wednesday, 14.12., 10:00-14:00 UTC, 100 % for anons

For the first time this year we were happy to be able to test author- and donor appeals from Germany. With Kurt a long time wikipedian and former president of the board of WMDE asks for a donation and talks about what fascinates him about Wikipedia. Corinna and Katrin speak as donors about their personal relationship to the online encyclopedia and the importance it has to them. The appeals were quite diverse in content, equally their results were different. Unfortunately none of the appeals reached the level of Jimmy’s appeal. What the test reveals once again is that new banners receive a high attention and therefore generate a much higher click-rate compared to the banner of Jimmy. The exception is the banner of Katrin which is in an old banner layout showing the speaker cut out on a white background. This remarkably goes along with the findings about the importance of the picture and the efficiency of the new type of photographs in natural surroundings. We hope to improve the click-rate of Katrins appeal significantly with a new picture and by that find an alternative to Jimmy’s appeal since her conversion is quite solid. --TobiasSchumann(WMDE) 09:59, 16 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Katrin	appeal Katrin	1647441	7712	61	1798,00	29,48	0,00468	0,0000370	0,00791	0,001091	0,233143	26,09
Corinna	appeal Corinna	1643034	10369	63	2090,00	33,17	0,00631	0,0000383	0,00608	0,001272	0,201562	24,56
Kurt	appeal Kurt	1646670	11034	53	1492,76	28,17	0,00670	0,0000322	0,00480	0,000907	0,135287	18,18
Jimmy Jacket	appeal Jimmy	1646024	7379	95	3172,00	33,39	0,00448	0,0000577	0,01287	0,001927	0,429869	22,22

Test 15: Text vs Video

Tuesday, 13.12.2011, 12:10-16:00, 100 % for anons

Video or text - what works better in persuading people to make a donation fo free knowledge? An exciting question that we tried to answer last summer by testing on the WMDE-run search portal wikipedia.de. Due to low traffic numbers of wikipedia.de the amount of donations was very low. Thats why our results were not very significant. But taken as mere indications the results were instructive. It didn't make a difference if video or text were displayed on the landing page. On the contrary, a video instead of a banner on wikipedia.de convinced a lot more readers to donate - we had a converson rate of 6%!

On Tuesday we tested that video again, thanks to the high traffic with a larger sample. How did we approach? We used three Jimmy banners, while two different banners linked to the video landing page. Here are the results:

Unfortunately the video didn't work on the landing page. The conversion rate is considerably lower than the text landing page.

This can have different reasons. Either videos don't work in general or particularly this video was not effective. Contrary to the old tests the content of video and text are not identical, that is comparability is limited. We know the current text appeal is a very effective one.

The click-rate show clearly the potential of videos

Obviously the inpact of the banners was very different. One of the two Jimmy banners linked to the video-page had a play button on it. By klicking on it no video started but the reader was directed to the landing page. This banner received by far most clicks. Reflecting on the low conversion rate we assume that the redirection to the landing page instead of running the video produced disappointment. But the click-rates remarkably show the attention a video directly on Wikipedia would receive.

The line "Please read", which we cut of in the video banners, is very important to generate klicks.

A comparison of the two banners without playbutton is very interesting. The huge differences in click-rates underline the results of former WMF-tests about the importance of the text line on the banner.

--TobiasSchumann(WMDE) 13:56, 14 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy jacket	Jimmy text	2310573	11609	124	3008,00	24,26	0,00502	0,0000537	0,01068	0,001302	0,259109	35,45
Jimmy playbutton	Jimmy video	2301875	19394	81	1941,00	23,96	0,00843	0,0000352	0,00418	0,000843	0,100082	27,54
Jimmy no playbutton	Jimmy video	2305326	9625	50	1420,00	28,40	0,00418	0,0000217	0,00519	0,000616	0,147532	33,33

Test 14: Author Appeals

Friday, 09.12.2011, 12:30-16:30 Uhr, 100 % for anons

The third test in one week - more than ever. This friday we tested two new editor appeals, compared to Jimmy and Brandon. It is very obvious: the new banners have higher click-rates and therefore much more visitors on the landing page. That shows clearly that new banners receive high attention and that people are curious and want to read the fresh appeals. However, a much smaller amount of landing page visitors actually donates compared to the appeal of Jimmy. Given his high conversion rate it is clear that his texts works much better than the new appeals. But fortunately the appeal of Susan generated a very high average sum. Together with the high click rate of her appeal she is on par with Jimmy. An effect of high attention that runs out fast. Thats why we decided to put her banner live, but only for a short period of time. Today we already switched banners again to run the Jimmy appeal. --TobiasSchumann(WMDE) 16:50, 12 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Susan	Susan appeal	1403390	12132	95	3085,00	32,47	0,00864	0,0000677	0,00783	0,002198	0,254286	28,24
Brandon woods	Brandon appeal	1403021	8739	77	2505,00	32,53	0,00623	0,0000549	0,00881	0,001785	0,286646	17,19
Karthik	Karthik appeal	1403799	12589	63	1988,00	31,56	0,00897	0,0000449	0,00500	0,001416	0,157916	19,23
Jimmy Jacket	Jimmy appeal	1404663	8255	120	2951,00	24,59	0,00588	0,0000854	0,01454	0,002101	0,357480	34,34

Test 13: Jimmy Banner

Thursday, 08.12.2011, 14:50-18:50 UTC, 100 % for anons.

Since our German editor appeals were not ready yet, we decided to have a Jimmy-Banner-test. We wanted to see which one turns out to be best in order to have the right Jimmy banner to kick off when everything else drops. The results are a little bit confusing. More clicks on the new banner, but no big differences in the donation numbers. Royal Tannenbaum and Jacket had the best conversion rate, while the CtR-winner was Kevin Spacey. Please notice that Jacket had some high donations - which we did not exclude -, so the donation sum in that one should be read carefuly. What we can learn here is that we are able to turn on new Jimmy-banners without risking loosing donations. But see for yourself:

Banner	Landingpage	Impressions	Visitors	Numbers	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per Visitor	Sum per Imp.	Sum per Visitor	% Women
Royal Tannenbaum	current LP	1936389	12892	153	3007,00	19,65	0,00666	0,0000790	0,01187	0,001553	0,233245	25,74
Kevin Spacey	current LP	2008481	14980	153	3122,00	20,41	0,00746	0,0000762	0,01021	0,001554	0,208411	29,10
Jacket	current LP	2009227	14554	169	4217,13	24,95	0,00724	0,0000841	0,01161	0,002099	0,289757	30,46

Till Mletzko (WMDE) 10:20, 9 December 2011 (UTC)[reply]

Test 12: layout donation form

Wednesday, 07.12.2011, 10:00-16:00 UTC; on 100% for anons

It's done! A test we had on our list for quite a while - it' about colors. Before we conduct more tests on banners and appeals already this week we tried to improve the layout of our landing page again. We analysed different colors of the donation form. Amazingly it had an effect on the amount of donations coming in. Compared to the variants in red (our default), blue and orange the donation form in green worked best. --TobiasSchumann(WMDE) 17:34, 8 December 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Brandon woods	red form	2634236	15822	132	3323,12	25,18	0,00601	0,0000501	0,00834	0,001262	0,210032	24,14
Brandon woods	orange form	2638177	16844	143	4285,50	29,97	0,00638	0,0000542	0,00849	0,001624	0,254423	26,67
Brandon woods	blue form	2633606	16539	136	4530,00	33,31	0,00628	0,0000516	0,00822	0,001720	0,273898	19,49
Brandon woods	green form	2631397	16492	179	5653,00	31,58	0,00627	0,0000680	0,01085	0,002148	0,342772	22,50

Test 11: Jimmy vs. non-Jimmy

30.11.2011, 12:00 UTC - 18:15 UTC, 100 % for anons

We have had Jimmy up for nearly the first two weeks. Since we saw a decline in donations we decided to run a non-Jimmy-appeal. Unfortunately, our German specific appeals are not ready yet. Fortunately, there are already great appeals out there we can adapt. So we did. We ran Jimmy against Ryan Kaldari, Brandon Harris and Alan Sohn. The result are interesting: Brandon did the best in CtR but the worst in conversion (but had the best donation sum). Alan, who had the least klicks, had the best conversion rate. So we decided to run Brandon 100 % and we actually made the best day within the last week. But see for yourself:

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy royal tannenbaum	Aufruf Jimmy	2142694	15294	184	5376,00	29,22	0,00714	0,0000859	0,01203	0,002509	0,351510	27,78
Kaldari	Aufruf Kaldari	2141460	15718	203	5402,00	26,61	0,00734	0,0000948	0,01292	0,002523	0,343682	20,47
Brandon woods	Aufruf Brandon	2141181	27522	315	8416,29	26,72	0,01285	0,0001471	0,01145	0,003931	0,305802	20,22
Alan	Aufruf Alan	2143790	13975	189	5208,00	27,56	0,00652	0,0000882	0,01352	0,002429	0,372665	20,13

I would like to emphasize three points to that test. 1. We were able to revert a declining donation income by changing the appeal. 2. A non-Jimmy-appeal works great in Germany aswell and 3. Brandon did not have the best conversion rate. This is why we will have to make the next step to more complex testing.

Till Mletzko (WMDE) 09:16, 2 December 2011 (UTC)[reply]

Test 10: landing page layout - top banner II

Thursday, 24.11.2011, 14:30-17:30 UTC, 3 banners together on 100%

This time testing was again about the layout of the landing page. In a previous test we analysed wether a banner on the landing page affects the donation process negatively or not. The result was not distinctive. But it looked like the banner increases donations slightly. That was contradictory to our assumptions because appeal and donation form are moved downwards by the use of the banner. This can be a problem on small laptop screens. Our conclusion was: personalization of the landing page could be very successful. This time we wanted to test exactly that. We tested our default layout against a landing page with banner and a landing page with a picture of Jimmy added to the appeal. It worked really well! The personalised landing page with the photo really improved the conversion rate and by that the number and amount of donations. Due to this test we were able to change our landing page for better again. --TobiasSchumann(WMDE) 12:16, 25 November 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Visitors	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
royal tannenbaum	top photo	1699056	13053	194	5054,01	26,05	0,00768	0,0001142	0,01486	0,002975	0,387191	25,99
royal tannenbaum	top banner	1696215	13011	136	3036,00	22,32	0,00767	0,0000802	0,01045	0,001790	0,233341	23,26
royal tannenbaum	without photo/banner	1694102	12961	149	3788,00	25,42	0,00765	0,0000880	0,01150	0,002236	0,292261	23,19

Test 9: landing page layouts

Friday, 18.11.2011; 16:15-19:15 UTC ; both banners together on 100%

After several test on Banners and donation forms this test focussed on the layout of the whole landing page. The goal was to find out, if a more simple design brings more clarity to the site and by that a higher donation rate. On the right side of the landing page we have several infoboxes (the so called tower) in our default layout. In the test we let them out and brought the information to other pages, accessible by the modified navigation. We tested the new layout against our current landing page. The winner is: the new layout. But the result is not very distinct and should be treated with caution. --TobiasSchumann(WMDE) 15:21, 24 November 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Besucher	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
royal tannenbaum	with tower	1892034	24612	361	7642,75	21,17	0,01301	0,0001908	0,01467	0,004039	0,310529	22,39
royal tannenbaum	no tower	1890619	24557	391	8562,95	21,90	0,01299	0,0002068	0,01592	0,004529	0,348697	21,45

Test 8: banner test

Tuesday, 15.11.2011, 17:30-19:30 UTC; both banner running 100%

Just having started the campaign the testing continues. When we heard from the success of the new Jimmy-Banner in tests of WMF and WMFR we decided to check immediately if it has the same impact on german Wikipedia. The new banner seemed promising with its increase in click-rate and donation amount. The test results are exciting. The new banner lead to a one hundred percent increase in donation amount compared to the old Jimmy-Banner. Obviously, the reason for this lies in the dramatic augmentation of the click rate, which is twice as high. With this outcome we of cource decided to run the new banner as our current campaign banner. --TobiasSchumann(WMDE) 12:36, 16 November 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Besucher	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
crossed arms	default	1.594.843	17.955	277	5859,10	21,15	0,01126	0,0001737	0,01543	0,003674	0,326321	20,15
royal tannenbaum	default	1.595.965	36.019	605	12.180,51	20,13	0,02257	0,0003791	0,01680	0,007632	0,338169	19,40

Test 7: appeal test

11.11.2011, 10-12 UTC, Two Jimmy-appeals running 100%

This was the last test before the beginning of the Fundraiser. As a last preperation we wanted to know which of two different Jimmy-appeals works better. The banners were visible for 100% of anons. The new appeal stretches out the difference of Wikipedia compared to other succsessful internet sites like google. In tests of the WMF this appeal was very effective. After the test it is clear now that our old appeal works as successful as the new one. Both appeals have virtually the same results. So now we know that we have two promising appeals to work with in the next weeks. --TobiasSchumann(WMDE) 11:45, 15 November 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Besucher	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
crossed arms	Aufruf new	1.502.516	13.447	319	6680,61	20,94	0,00895	0,0002123	0,02372	0,004446	0,496810	21,32
crossed arms	Aufruf old	1.505.269	13.589	329	6870,60	20,88	0,00903	0,0002186	0,02421	0,004564	0,505600	25,84

Test 6: donation ask string

07.11.2011, 14:30-16:30 UTC - Jimmy 100 %

Last monday we had our sixth test with Jimmy running 100 % for two hours for anons. We had three donation forms with different ask strings. We wanted to see if the amount and numbers of donation suggestions have an impact on the outcome. And it does! Our current ask string with five options beat out the other ones. We can now confirm the test conclusion from WMF last year that the more options you offer, the more people donate. In another test we will see if (more) higher amounts lead to an higher average donation.

Till Mletzko (WMDE) 15:15, 9 November 2011 (UTC)[reply]

Banner	Landingpage	Impressions	Besucher	Sum	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
crossed arms	3 amounts	1.345.338	16.924	165	4198,15	25,44	0,01258	0,0001226	0,00965	0,003121	0,248059	23,03
crossed arms	5 amounts	1.346.105	16.749	253	6313,10	24,95	0,01244	0,0001879	0,01511	0,004690	0,376924	22,53
crossed arms	4 amounts	1.349.501	16.894	207	5097,72	24,63	0,01252	0,0001534	0,01225	0,003777	0,301747	18,36

Test 5: radiobutton vs. dropdown

31.10.2011, 13:00 - 15:00 UTC - Jimmy 100%

We ran our fifth test on Monday 31st. For two hours we had the crossed-arm-Jimmy-Banner up leading to two different landingpages. One LP had our current dropdown-donationform while the other had a new radiobutton-donationform. With the new form we optimized the usability and the donation process. Unfortunatley, the results are not very enlightening. We had more donations with the radiobutton-form but a higher donation sum with the dropdown-form. Next week, we want to see in another test if changing the donation ask string in the radiobutton-form has an effect on the donation sum.

Banner	Landingpage	Impressions	Vistors	Numbers	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
crossed arms	Radio-Button	1.527.107	19.145	321	6.105,40	19,02	0,01254	0,0002102	0,01677	0,003998	0,318903	18,69
crossed arms	Drop-Down	1.526.563	19.218	299	6.868,22	22,97	0,01259	0,0001959	0,01259	0,004499	0,357385	21,07

Test 4: Banner-Layout

26.10.2011, Tuesday 10:15 - 16:15 UTC, all Banners with 25%

Banner	Landingpage	Impressions	Visitors	Numbers	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
crossed arms	default layout	2.442.011	30272	576	11.611,76	20,16	0,01240	0,0002359	0,01903	0,004755	0,383581	21,35
white banner	default layout	2.442.512	27752	436	8324	19,09	0,01136	0,0001785	0,01571	0,003408	0,299942	24,54
black banner	default layout	2.444.291	17192	306	6500,88	21,24	0,00703	0,0001252	0,01780	0,002660	0,378134	23,53
green banner	default layout	2.444.670	17429	305	6750	22,13	0,00713	0,0001248	0,01750	0,002761	0,387286	22,62

On Oct. 26th we tested the fourth time. This one was a performance test in order to check our server-infrastructure. The test lasted for six hours on 100% during daytime. The results show us that we are well prepared for the coming fundraiser from a tech perspective while the donation side also looks pretty good. We used the performace test for Jimmy-banners aswell. We were searching for an opener and we might have found it. The crossed-arms Jimmy performed the best (in generell the white banners did better than the other colours). But take a look at the stats yourself. If you have questions, go ahead: till.mletzko[at]wikimedia.de

--TobiasSchumann(WMDE) 17:09, 31 October 2011 (UTC)[reply]

Test 3: Top-Banner vs. without Banner

18.10.2011, Tuesday 12:00 - 14:00 UTC, both Banners with 50%

Banner	Landingpage	Impressions	Visitors	Numbers	Amount	Average	Click- Through- Rate	Numbers per Imp.	Numbers per View	Sum per Imp.	Sum per View	% Women
Jimmy Black	Landingpage A without Top-Banner	1.475.728	14.967	203	4174,77	20,43	0,01014	0,0001376	0,01356	0,002811	0,277128	21,67%
Jimmy Black	Landingpage B with Top-Banner	1.481.728	14.970	215	4582,76	21,32	0,01010	0,0001451	0,01436	0,003093	0,306130	17,21%

We had the third test on Oct. 18th. We had the Jimmy Banner up for two hours going to two different LP 100%. One of the LP had the Jimmy Top-Banner, the other one had none. In addition, we activated the CC-donation-option. The test was very successful from a techical and content perspective. Even though the differences in the absolut numbers are not that high, we can say that not using the top-banner on the LP has no big influence. Therefore we will not have it for the coming fundraiser to imcrease usability for smaller screens. On top of that, we added another testing variable: gender! We know that only a small part of our donors are female despite the fact that a lot more women using Wikipedia. We are convinced that raising the number of female donors is one of the major reasons for the success of the coming fundraiser. This is why the gender category will be one more thing to analyse and test.

TobiasSchumann(WMDE) 07:30, 21 October 2011 (UTC)[reply]

Impressions and rates added TobiasSchumann(WMDE) 09:36, 26 October 2011 (UTC)[reply]

Test 2: Top-Banner vs. without Banner

10.10.2011; Monday between 12:05-13:05 UTC, beginning with 25% each, going up to 50% each after 20 minutes

Banner	Landingpage	Impressions	View	Click-Through-Rate	Numbers Direct Debit	Sum Direct Debit	Numbers Money Transfer	Sum Money Transfer	Numbers PayPal	Sum PayPal	Numbers Total	Sum Total Euro	Average	Amount per View	Amount per Impression
Jimmy Black	Landingpage A without Top-Banner	362.507	5203	0,0143	13	285	28	740
Jimmy Black	Landingpage B with Top-Banner	361.612	5142	0,0142	14	320	18	465
									40	605	113	2415	21,37	0,2334	0,0033

Results: this test was not really successful. We had a major bug in the donation form, which we discovered after the test was over. Due to the bug we had a lot of direct-debit-donors that jumped off right before they finished the donation and (even worse) some finished donations containing false adress information. We fixed that bug a couple of hours later (in the meantime we had to put down the donation site). In addition to that PayPal recently changed the information that they used to send us per default. Because of that we are currently not able to track the PayPal-donations to one or the other Landingpage. This is why the numbers above are not reliable and are not linked entirely to the tested Landingpages. I think we will have to repeat that test configuration in the coming weeks. The good thing about PayPal is that we will have a PayPal-iFrame set up in the coming days with which we can gather adress and tracking information on our side.

Till Mletzko (WMDE) 11:55, 12 October 2011 (UTC)[reply]

Test 1: Performance Test for new wiki-donationform

30.09.2011; Friday between 12:05-13:05 UTC

Banner	LandingPage	Impressions	Views	Numbers Direct Debit	Sum Direct Debit	Numbers Money Transfer	Sum Money Transfer	Numbers PayPal	Sum PayPal	Numbers Total	Sum Total Euro	Average	Click-Through-Rate	Amount per View	Amount per Impression
Jimmy Black	Landingpage with new wiki-based donation form	97.311	2.332	13	265	9	350	12	275	34	890	26,18	0,0240	0,3816	0,0091

WMDE had its first test on de.wikipedia.org last week. We ran the black Jimmy banner for one hour between 12:05-13:05 UTC. We started with 5%, moved quickly to 10% and ended up with 20% for the second 30 minutes for anonymous users only. We tested our new wiki-based donation form with some alterations regarding the payment method money transfer. The test was quite successful despite some technical (e.g. Piwik) and minor functional deficits that we will correct for next time. The new donation form worked pretty well and now we are able to test different donation forms a lot easier. In addition to that we are now able to gather adress information from money transfer donation before the possible donor leaves our landingpage. Donations via money transfer usually does not contain adress information, so in the past we were neither able to send them a donation receipt nor cultivate them. But one has to keep in mind that we only gather adress information with our new donation form. Therefore those "donations" are in fact donation intentions. We can only say after some time if the donation intention became a real donation on our account. Till Mletzko (WMDE) 09:16, 5 October 2011 (UTC)[reply]