Fundraising 2012/We Need A Breakthrough

Contents

August 24, 2012 "If everyone donated $5.." in Facts banner. Hint: BIG FIND!

Pop quiz from Fundraiser 2011: What banner text did we finally find that actually beat the "Jimmy appeal" banner at some point during the fundraiser? Can't remember? I'll tell you!

Answer: "If everyone reading this donated $5, we would only have to fundraise for one day a year. Please donate to keep Wikipedia free. "

We know this line is really important in the appeal too. We've met with donors and listened to them tell us how important that line is in the Jimmy message. So we're way past due to test this line in our new facts banner.

Here's the test setup:

• Control facts banner: http://en.wikipedia.org/wiki/Control?banner=B12_082412_NofaceTopGreyFactsask
• Facts banner with $5 line: http://en.wikipedia.org/wiki/Five?banner=B12_082412_NofaceTopGreyFactsaskFive

AND THE VERY EXCITING RESULTS THAT MAKE MEGAN SMILE:

+--------------------------------------+-----------+----------+----------+-----------+-----------+
| banner                               | donations | amount   | amount18 | avg       | avg18     |
+--------------------------------------+-----------+----------+----------+-----------+-----------+
| B12_082412_NofaceTopGreyFactsask     |       579 | 11500.00 |  7048.00 | 19.861831 | 12.172712 |
| B12_082412_NofaceTopGreyFactsaskFive |      1041 | 14700.00 | 10283.00 | 14.121037 |  9.878002 |
+--------------------------------------+-----------+----------+----------+-----------+-----------+ 

Here's the confidence on the number of donations:

Confidence on number of donations

I'm awake! Great test. Need to find a few more improvements like this before we launch in November.

4 September, 2012 Reality Checking Top facts vs Inline Jimmy

In June, we found a top facts banner that actually beat our Jimmy banner from last year. Big find! Over the past few months, we've gone through a bunch of variations of that top facts banner so we wanted to run a reality check on how well we're doing with our current best facts banner against where we were last year.

Today we're running:

• Inline Jimmy: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_JW
• Top facts $5: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFive

Here are the half-time results after 30 minutes of testing:

 +--------------------------------------+-----------+---------+----------+--------+
| banner                               | donations | amount  | amount18 | max    |
+--------------------------------------+-----------+---------+----------+--------+
| B12_082412_JW                        |       180 | 3516.00 |  2110.00 | 100.00 |
| B12_082412_NofaceTopGreyFactsaskFive |       304 | 5037.50 |  3150.50 | 300.00 |
+--------------------------------------+-----------+---------+----------+--------+

And here are the results from the full hour of testing:

 +--------------------------------------+-----------+---------+----------+--------+
| banner                               | donations | amount  | amount18 | max    |
+--------------------------------------+-----------+---------+----------+--------+
| B12_082412_JW                        |       351 | 6993.00 |  4082.00 | 250.00 |
| B12_082412_NofaceTopGreyFactsaskFive |       566 | 9322.50 |  5678.50 | 300.00 |
+--------------------------------------+-----------+---------+----------+--------+ 

Confidence on number of donations

It's very encouraging to see this new banner we've been working on is coming out confidently ahead of where we were last year. And we still have a few months of testing to go before the campaign launches! Meganhernandez (talk) 15:04, 4 September 2012 (UTC)

6 September, 2012 Quick 10 minute Test without Google/Yahoo line

We're trying out some alternatives to the Google/Yahoo line in the facts banner. ("Google might have close to a million servers. Yahoo has something like 12,000 staff. We have 679 servers and 138 employees.") A brief background: We added this comparison with Google/Yahoo in 2011 and found that it was a huge gain for both donations and total amount raised. We're testing a couple new variations of the same concept.

Today we tested (very quickly- just 10 minutes):

• Google/yahoo Facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFive
• No google/yahoo line Facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFiveNoG

Here are the results (remember, after just 10 minutes - not long enough to be totally confident in these results):

+-----------------------------------------+-----------+---------+----------+
| banner                                  | donations | amount  | amount18 |
+-----------------------------------------+-----------+---------+----------+
| B12_082412_NofaceTopGreyFactsaskFive    |       293 | 4098.00 |  2957.00 |
| B12_082412_NofaceTopGreyFactsaskFiveNoG |       218 | 3074.00 |  2073.00 |
+-----------------------------------------+-----------+---------+----------+

6 September, 2012 Another quick 15-min Test: Trying to fix the Google/Yahoo line!

Potential good news here. We're trying to get away from the Google/Yahoo line that has caused some unhappiness/concern over the past year. We just ran a quick 15-min test (so, this is a test we'll want to run again to confirm the results), but it looks like we found a new variation of that line that might work just as well. Great potential news! (Numbers below.) We've been testing a bunch of different lines over the last few weeks, but this is the closest we've come so far.

We tested:

• Old text with the Google/Yahoo numbers: http://en.wikipedia.org/wiki/International_Olympic_Committee?banner=B12_082412_NofaceTopGreyFactsaskFive
• New text: http://en.wikipedia.org/wiki/International_Olympic_Committee?banner=B12_082412_NofaceTopGreyFactsaskFiveThous

And the numbers:

+-------------------------------------------+-----------+---------+----------+-----------+----------+
| banner                                    | donations | amount  | amount18 | avg       | avg18    |
+-------------------------------------------+-----------+---------+----------+-----------+----------+
| B12_082412_NofaceTopGreyFactsaskFive      |       407 | 6103.31 |  4012.81 | 14.995848 | 9.859484 |
| B12_082412_NofaceTopGreyFactsaskFiveThous |       409 | 5311.00 |  3816.00 | 12.985330 | 9.330073 |
+-------------------------------------------+-----------+---------+----------+-----------+----------+

So really exciting result. It was a short test, so we'll re-run it to confirm. More later, running to meet a donor!

7 September, 2012 Repeating the Google line test from yesterday

Here are the results combined from yesterday and today:

+-------------------------------------------+-----------+----------+----------+-----------+----------+
| banner                                    | donations | amount   | amount18 | avg       | avg18    |
+-------------------------------------------+-----------+----------+----------+-----------+----------+
| B12_082412_NofaceTopGreyFactsaskFive      |      1114 | 15928.31 | 10981.81 | 14.298303 | 9.857998 |
| B12_082412_NofaceTopGreyFactsaskFiveThous |      1022 | 13860.00 |  9684.00 | 13.561644 | 9.475538 |
+-------------------------------------------+-----------+----------+----------+-----------+----------+ 

September 12, 2012 Retesting the Google/Yahoo line

We just reran the test from last week:

• Google millions facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFive
• Google thousands facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFiveThous

Here are the results from the one-hour test this morning:

+-------------------------------------------+-----------+----------+----------+-----------+-----------+
| banner                                    | donations | amount   | amount18 | avg       | avg18     |
+-------------------------------------------+-----------+----------+----------+-----------+-----------+
| B12_082412_NofaceTopGreyFactsaskFive      |       701 | 10754.00 |  7206.00 | 15.340942 | 10.279601 |
| B12_082412_NofaceTopGreyFactsaskFiveThous |       636 | 10426.00 |  6239.00 | 16.393082 |  9.809748 |
+-------------------------------------------+-----------+----------+----------+-----------+-----------+
 

Confidence testing on number of donations:

Confidence on number of donations

And here are the combined results from last week and today:

+-------------------------------------------+-----------+----------+----------+-----------+-----------+
| banner                                    | donations | amount   | amount18 | avg       | avg18     |
+-------------------------------------------+-----------+----------+----------+-----------+-----------+
| B12_082412_NofaceTopGreyFactsaskFive      |      1857 | 27607.31 | 18626.81 | 14.866618 | 10.030592 |
| B12_082412_NofaceTopGreyFactsaskFiveThous |      1695 | 24939.00 | 16302.00 | 14.713274 |  9.617699 |
+-------------------------------------------+-----------+----------+----------+-----------+-----------+

Confidence on number of donations from the combined results:

Confidence on number of donations

Sept. 14, 2012 Expander Banner

Wait! After this test ran (and I posted the update), we saw there was a mistake in the ask amounts on the new expanding/dropdown form. We somehow accidentally swapped out the $20 option and the ask amounts were different on the two forms. Skip down to the Sept. 17 update to see the results from this test once we fixed the ask button. Meganhernandez (talk) 18:17, 18 September 2012 (UTC)

We're testing a new type of banner today. The banner stays in place when you click on it and the donation form drops down from the banner instead of taking you to a new page.

Take a look:

• Control facts banner: http://en.wikipedia.org/wiki/2012?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
• New expanding/dropdown banner: http://en.wikipedia.org/wiki/September_14?banner=B12_Expandable_Facts_PlainButtons_Image

The results:

+-------------------------------------------+-----------+----------+----------+-----------+----------+
| banner                                    | donations | amount   | amount18 | avg       | avg18    |
+-------------------------------------------+-----------+----------+----------+-----------+----------+
| B12_082412_NofaceTopGreyFactsaskFiveThous |       941 | 13245.11 |  8987.11 | 14.075569 | 9.550595 |
| B12_Expandable_Facts_PlainButtons_Image   |      1054 | 13821.59 |  9751.59 | 13.113463 | 9.251983 |
+-------------------------------------------+-----------+----------+----------+-----------+----------+

Confidence:

Confidence on number of donations

September 17, 2012 Expanding/dropdown banner test

We're re-testing the dropdown banner.

The banners:

• Control facts banner: http://en.wikipedia.org/wiki/Fact?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
• Dropdown facts banner: http://en.wikipedia.org/wiki/Dropkick?banner=B12_Expandable_Facts_PlainButtons_Image_Amts

The results:

+----------------------------------------------+-----------+----------+----------+-----------+----------+
| banner                                       | donations | amount   | amount18 | avg       | avg18    |
+----------------------------------------------+-----------+----------+----------+-----------+----------+
| B12_082412_NofaceTopGreyFactsaskFiveThous    |       673 | 10465.25 |  6392.65 | 15.550149 | 9.498737 |
| B12_Expandable_Facts_PlainButtons_Image_Amts |       814 | 10438.77 |  7647.77 | 12.824042 | 9.395295 |
+----------------------------------------------+-----------+----------+----------+-----------+----------+ 

Confidence on number of donations

September 20, 2012 Less wordy facts & Amazon test

We ran a multivariate test in the US today. I'll report on the landing page test first, since the results from that test were more inspirational than the banner test results.

Amazon landing page test

We tested a form offering Amazon Payments as a new payment method.

Our control form allows users to donate by:

• Credit/Debit card
• PayPal

The new form included:

• Credit/Debit card
• PayPal
• Amazon

Here are the pages:

• Control credit/PayPal: https://donate.wikimedia.org/wiki/L12_0628_norec_JW/en
• New form with Amazon: https://donate.wikimedia.org/wiki/L12_0917_amazon_JW/en

Results:

Results by landing page:

Results broken out by payment method:

+--------------------+----------------+-----------+---------+----------+-----------+-----------+
| landing_page       | payment_method | donations | amount  | amount18 | avg       | avg18     |
+--------------------+----------------+-----------+---------+----------+-----------+-----------+
| L12_0628_norec_JW  | cc             |       266 | 4825.03 |  2930.47 | 18.139211 | 11.016805 |
| L12_0628_norec_JW  | pp             |       295 | 3430.99 |  2669.99 | 11.630475 |  9.050814 |
| L12_0917_amazon_JW | amazon         |       129 | 1552.00 |  1196.00 | 12.031008 |  9.271318 |
| L12_0917_amazon_JW | cc             |       231 | 4069.10 |  2518.10 | 17.615152 | 10.900866 |
| L12_0917_amazon_JW | pp             |       261 | 2836.00 |  2256.00 | 10.865900 |  8.643678 |
+--------------------+----------------+-----------+---------+----------+-----------+-----------+

So, it looks like offering Amazon may be increasing the number of donations coming in, and could possibly also affect the amount raised (look at amt18). We'll run some confidence tests on those results and post back. We will probably want to re-run this test for longer at some point since the results look pretty close. It's really encouraging if we're making it easier for more readers to donate.

Banner Test

We tested our control facts banner against a new banner with fewer words and with vertical lines separating each fact.

• Control facts banner: http://en.wikipedia.org/wiki/Words?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
• New banner with fewer words & vertical lines: http://en.wikipedia.org/wiki/Vertical_direction?banner=B12_091912_Facts_LessText_LinesBold

And the banner results which really bum Megan out:

+-------------------------------------------+-----------+---------+----------+-----------+----------+
| banner                                    | donations | amount  | amount18 | avg       | avg18    |
+-------------------------------------------+-----------+---------+----------+-----------+----------+
| B12_082412_NofaceTopGreyFactsaskFiveThous |       699 | 9688.24 |  6788.24 | 13.860143 | 9.711359 |
| B12_091912_Facts_LessText_LinesBold       |       483 | 7024.88 |  4782.32 | 14.544265 | 9.901284 |
+-------------------------------------------+-----------+---------+----------+-----------+----------+

I thought we could make the banner easier for people to read if we made it less wordy, made the font size bigger and added some lines separating each fact. Clearly, I was wrong. We still have a ton of variations of that facts banner to test though. Improvements on the way!

Meganhernandez (talk) 20:34, 20 September 2012 (UTC)

September 24, 2012 Retesting a 2011 campaign

Results from Sept. 24, 2012:

+------------------+-----------+----------+----------+-----------+-----------+
| banner           | donations | amount   | amount18 | avg       | avg18     |
+------------------+-----------+----------+----------+-----------+-----------+
| B12_0924_btPf_US |       511 | 11548.00 |  6718.00 | 22.598826 | 13.146771 |
+------------------+-----------+----------+----------+-----------+-----------+

Results from Nov. 15, 2011:

+------------------+-----------+----------+----------+-----------+-----------+
| banner           | donations | amount   | amount18 | avg       | avg18     |
+------------------+-----------+----------+----------+-----------+-----------+
| B11_1115_btPf_US |       543 | 15105.00 |  8058.00 | 27.817680 | 14.839779 |
+------------------+-----------+----------+----------+-----------+-----------+

September 26, 2012 More Dropdown banner testing

Last week (Sept. 17), we tested a new dropdown/expander banner and it looked like we saw a big increase from the standard facts banner. We showed this new banner around to some donors and got some great feedback that it wasn't necessarily clear to everyone that they were supposed to click on the banner.

So today we tested a couple variations of dropdown banners (with arrows & butons) and tweaked the messaging a bit.

The banners:

• Arrow & control text : http://en.wikipedia.org/wiki/Arrow?banner=B12_ExfactsTabarrowSeptext

• Button & control text: http://en.wikipedia.org/wiki/Button?banner=B12_ExfactsTabbuttonSeptext

• Arrow & new "long way" text: http://en.wikipedia.org/wiki/L%C3%B3ng?banner=B12_ExfactsTabarrowLongway

• Button & new "long way" text: http://en.wikipedia.org/wiki/The_Way_(film)?banner=B12_ExfactsTabbuttonLongway

The results:

+-----------------------------+-----------+---------+----------+-----------+-----------+
| banner                      | donations | amount  | amount18 | avg       | avg18     |
+-----------------------------+-----------+---------+----------+-----------+-----------+
| B12_ExfactsTabarrowLongway  |       376 | 5998.00 |  3927.00 | 15.952128 | 10.444149 |
| B12_ExfactsTabarrowSeptext  |       454 | 6719.00 |  4422.00 | 14.799559 |  9.740088 |
| B12_ExfactsTabbuttonLongway |       329 | 4879.37 |  3358.37 | 14.830912 | 10.207812 |
| B12_ExfactsTabbuttonSeptext |       418 | 6018.00 |  4075.00 | 14.397129 |  9.748804 |
+-----------------------------+-----------+---------+----------+-----------+-----------+

Arrow vs. Button:

It looks like Arrow is a little ahead of Button, but not with a ton of confidence .

Control vs. new text:

We have more confidence that our control text comes brings in more donations than the new "donations go a long way" text.

Meganhernandez (talk) 16:48, 27 September 2012 (UTC)

October 1 "Forget fundraising & get back to Wikipedia" and Amazon

Zack came up with a new fact for the banner and it looks like it's helping big time!

• Control facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1001_Facts

• New "back to Wikipedia" facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1001_FactsBackto

• Half of 1%: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1001_Half1Cols

Results

+----------------------+-----------+----------+----------+-----------+-----------+
| banner               | donations | amount   | amount18 | avg       | avg18     |
+----------------------+-----------+----------+----------+-----------+-----------+
| B12_1001_Facts       |       848 | 12163.85 |  8069.85 | 14.344163 |  9.516333 |
| B12_1001_FactsBackto |      1123 | 15620.59 | 10474.65 | 13.909697 |  9.327382 |
| B12_1001_Half1Cols   |       853 | 11620.75 |  8701.75 | 13.623388 | 10.201348 |
+----------------------+-----------+----------+----------+-----------+-----------+

The last fact on the "back to Wikipedia" banner helps a lot. We'll test more to see if the "one hour" and "back to Wikipedia" texts are both helping.

The "half of 1%" is also doing about as well as our control facts banner. Which is great since it's missing our really strong fact comparing Wikipedia to Google and Yahoo. There must be parts of this banner that are helping and hurting donations. We'll test more to isolate the parts that are helping.

Amazon retest

We also re-tested offering Amazon as a payment option (first test was Sept. 20).

• Credit card & Paypal donation page: https://donate.wikimedia.org/wiki/L12_0628_norec_JW/en

• Credit card, Paypal & Amazon LP: https://donate.wikimedia.org/wiki/L12_0917_amazon_JW/en

Results by landing page

Results by payment method

 +--------------------+----------------+-----------+----------+----------+-----------+-----------+
| L12_0628_norec_JW  | cc             |       619 |  9945.17 |  6480.23 | 16.066511 | 10.468869 |
| L12_0628_norec_JW  | pp             |       723 | 10206.15 |  6695.15 | 14.116390 |  9.260235 |
| L12_0917_amazon_JW | amazon         |       327 |  4327.15 |  2953.15 | 13.232875 |  9.031040 |
| L12_0917_amazon_JW | cc             |       489 |  7248.78 |  5096.78 | 14.823681 | 10.422863 |
| L12_0917_amazon_JW | pp             |       666 |  7677.94 |  6020.94 | 11.528438 |  9.040450 |
+--------------------+----------------+-----------+----------+----------+-----------+-----------+

Our two Amazon tests (Sept. 20 & Oct. 1) show the Amazon page coming slightly ahead. We'll run the confidence test on the amount raised later, but both tests look like offering Amazon along with credit card and Paypal increases the amount we raise. (Plus hopefully makes some people happier by having their preferred payment method available :)

Meganhernandez (talk) 19:09, 2 October 2012 (UTC)

ITALIAN TEST SEPTEMBER 30 - OCTOBER 7, 2012

Ask amount Test €5 vs €10

• LP1 (5$): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=it&country=IT&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_InPaJwJkRn_IT_FR&utm_campaign=none_FR

• LP2: (10$): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=it&country=IT&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-variable1&utm_medium=sitenotice&utm_source=B12_InPaJwJoRn_IT_IT&utm_campaign=none_IT

+-------------------------------------------+-----------+----------+----------+-----------+-----------+
| landing_page                              | donations | amount   | amount18 | avg       | avg18     |
+-------------------------------------------+-----------+----------+----------+-----------+-----------+
| default~default~default~default~control   |      1269 | 19764.79 | 13191.77 | 15.575091 | 10.395406 |
| default~default~default~default~variable1 |      1026 | 17323.41 | 12624.41 | 16.884415 | 12.304493 |
+-------------------------------------------+-----------+----------+----------+-----------+-----------+

-s 20120930174200 -e 20121002132400 -c C12_iwpitWW_FR_IT -l -t

Jimmy photo-at-thon

Banners:

• http://it.wikipedia.org/wiki/The_Jacket?banner=B12_InPaJwJkRn_IT
• http://it.wikipedia.org/wiki/Face?banner=B12_InPaJwPfRn_IT
• http://it.wikipedia.org/wiki/Space_Jam?banner=B12_InPaJwKsRn_IT
• http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_InPaJwJoRn_IT

 +----------------------+-----------+---------+----------+------------+-----------+
| banner               | donations | amount  | amount18 | avg        | avg18     |
+----------------------+-----------+---------+----------+------------+-----------+
| B12_InPaJwJkRn_IT_IT |       316 | 4808.21 |  3580.76 |  15.215854 | 11.331519 |
| B12_InPaJwJoRn_IT_IT |       315 | 4544.40 |  3503.32 |  14.426667 | 11.121651 |
| B12_InPaJwKsRn_IT_IT |       294 | 4382.08 |  3344.86 |  14.905034 | 11.377075 |
| B12_InPaJwPfRn_IT_IT |       346 | 5508.44 |  3967.21 |  15.920347 | 11.465925 |
+----------------------+-----------+---------+----------+------------+-----------+

ecom -s 20120930174200 -e 20121001095700 -c C12_iwpitWW_FR -b -t

Green background Jimmy vs White background Jimmy

• Green: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_InPaJwG2Rn_IT
• White: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_InPaJwJoRn_IT

+----------------------+-----------+----------+----------+-----------+-----------+
| banner               | donations | amount   | amount18 | avg       | avg18     |
+----------------------+-----------+----------+----------+-----------+-----------+
| B12_InPaJwG2Rn_IT_IT |      1139 | 15783.10 | 11688.99 | 13.856980 | 10.262502 |
| B12_InPaJwJoRn_IT_IT |      1013 | 14156.89 | 10358.20 | 13.975212 | 10.225271 |
+----------------------+-----------+----------+----------+-----------+-----------+

-s 20121002132400 -e 20121003191100 -c C12_iwpitWW_FR_IT -b -t

Jimmy vs dropdown

• Jimmy banner: http://it.wikipedia.org/wiki/Comune_italiano?banner=B12_InPaJwG3Rn_IT
• Dropdown banner: http://it.wikipedia.org/wiki/Palazzi_di_Cesena?banner=B12_ExpandFactsImage_IT

+-------------------------+-----------+---------+----------+-----------+-----------+
| banner                  | donations | amount  | amount18 | avg       | avg18     |
+-------------------------+-----------+---------+----------+-----------+-----------+
| B12_ExpandFactsImage_IT |       755 | 7482.60 |  6447.42 |  9.910728 |  8.539629 |
| B12_InPaJwG3Rn_IT_IT    |       702 | 8955.38 |  6943.37 | 12.756952 |  9.890840 |
+-------------------------+-----------+---------+----------+-----------+-----------+

ecom -s 20121003191100 -e 20121004113100 -c C12_iwpitWW_FR -b -t

Italy Grey dropdown facts vs blue modified facts

• grey facts:http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandFactsImage_IT
• blue (fact 3: focus on wikipedia , 20 average amount highlighted, $5/1 day on right side of form): http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandHm

+-------------------------+-----------+----------+----------+-----------+-----------+
| banner                  | donations | amount   | amount18 | avg       | avg18     |
+-------------------------+-----------+----------+----------+-----------+-----------+
| B12_ExpandFactsImage_IT |       954 | 10286.56 |  8348.97 | 10.782558 |  8.751541 |
| B12_ExpandHm            |      1452 | 16851.80 | 13411.37 | 11.605923 |  9.236481 |
+-------------------------+-----------+----------+----------+-----------+-----------+

ecom -s 20121004113100 -e 20121005120000 -c C12_iwpitWW_FR -b -t

Italy blue modified facts vs New fact 3

• Blue modified: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandHm2
• New 3rd fact translation: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandHmTrans

+-------------------------+-----------+---------+----------+-----------+-----------+
| banner                  | donations | amount  | amount18 | avg       | avg18     |
+-------------------------+-----------+---------+----------+-----------+-----------+
| B12_ExpandHm2           |       552 | 6173.19 |  5110.62 | 11.183315 |  9.258370 |
| B12_ExpandHmTrans       |       491 | 5724.97 |  4704.82 | 11.659817 |  9.582118 |
+-------------------------+-----------+---------+----------+-----------+-----------+ 

ecom -s 20121005134500 -e 2012101500000000 -c C12_iwpitWW_FR -b -t

Reports left to run

• Second step address test

US Long & Short appeals from Dropdown - Friday October 5, 2012

Our "facts banner" provides much more info directly from the banner than last year's "Please read: a personal appeal..." banner. Now that our readers are already learning the reasons to donate directly from the banner, we experimented today with showing a much shorter appeal than last year's appeal once you click on the banner. We're trying to limit repetition and make it quick and easy for people to donate.

• Long appeal: http://en.wikipedia.org/wiki/Main_Page?banner=B12_DropdownL
• Short appeal: http://en.wikipedia.org/wiki/Main_Page?banner=B12_DropdownS

+---------------+-----------+----------+----------+-----------+----------+
| banner        | donations | amount   | amount18 | avg       | avg18    |
+---------------+-----------+----------+----------+-----------+----------+
| B12_DropdownL |      1374 | 19980.58 | 13213.33 | 14.541907 | 9.616689 |
| B12_DropdownS |      1478 | 20837.64 | 14281.64 | 14.098539 | 9.662815 |
+---------------+-----------+----------+----------+-----------+----------+

ecom -s 20121005131000 -e 20121005171000 -c C12_1005_LvS -t

Confidence on number of donations

Monday October 8, 2012 EN T1 Jimmy vs Blue Facts

Hour 1:

• jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal

• Bluetop facts: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts

LP (CA): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=CA&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1008_Backtobluefacts_undefined&utm_campaign=C12_iwpenT1_FR_undefined

Hour 2:

• Jimmy: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal2

• Bluetop: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts

+------+-----------------------------+-----------+---------+----------+-----------+-----------+
| lpi  | banner                      | donations | amount  | amount18 | avg       | avg18     |
+------+-----------------------------+-----------+---------+----------+-----------+-----------+
| 4250 | B12_1008_Jimmyinlineappeal2 |       108 | 1997.65 |  1281.56 | 18.496759 | 11.866296 |
| 3955 | B12_1008_Jimmyinlineappeal  |       106 | 2884.08 |  1321.77 | 27.208302 | 12.469528 |
| 3209 | B12_1008_Backtobluefacts    |       375 | 5631.31 |  3684.99 | 15.016827 |  9.826640 |
| 3091 | B12_1008_Backtobluefacts2   |       320 | 4198.77 |  2939.87 | 13.121156 |  9.187094 |
+------+-----------------------------+-----------+---------+----------+-----------+-----------+

http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=108%2C4250&B12_1008_Jimmyinlineappeal=106%2C3955&B12_1008_Backtobluefacts=375%2C3209&B12_1008_Backtobluefacts2=320%2C3091&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

EN T2 (including GB) Jimmy vs. Blue Facts Tuesday October 9, 2012

First 38 minutes when GB was included in EN T2:

• Jimmy banner; http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Gd

• Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Bd

• Blue banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts3

+-------------------------------+-----------+---------+----------+-----------+-----------+
| banner                        | donations | amount  | amount18 | avg       | avg18     |
+-------------------------------+-----------+---------+----------+-----------+-----------+
| B12_1008_Backtobluefacts3     |       427 | 5526.81 |  4613.39 | 12.943349 | 10.804192 |
| B12_1008_Jimmyinlineappeal_Bd |        51 |  923.62 |   681.31 | 18.110196 | 13.359020 |
| B12_1008_Jimmyinlineappeal_Gd |        73 | 1454.45 |   928.01 | 19.923973 | 12.712466 |
+-------------------------------+-----------+---------+----------+-----------+-----------+

ecom -s 20121009081500 -e 20121009084800 -c C12_iwpenT2EurAf_FR -b --sql -m | db -t

GB-only Jimmy vs Blue facts banner Tuesday October 9, 2012

• Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Bd_GB

• Jimmy twin: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Gd_GB

• Blue facts: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts3_GB

• LP: https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=GB&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1008_Jimmyinlineappeal_Gd_GB_FR&utm_campaign=none_FR

+----------------------------------+-----------+---------+----------+-----------+-----------+
| banner                           | donations | amount  | amount18 | avg       | avg18     |
+----------------------------------+-----------+---------+----------+-----------+-----------+
| B12_1008_Backtobluefacts3_GB     |       474 | 8285.70 |  6350.45 | 17.480380 | 13.397574 |
| B12_1008_Jimmyinlineappeal_Bd_GB |       145 | 2719.86 |  2064.25 | 18.757655 | 14.236207 |
| B12_1008_Jimmyinlineappeal_Gd_GB |       142 | 3018.45 |  2095.70 | 21.256690 | 14.758451 |
+----------------------------------+-----------+---------+----------+-----------+-----------+

ecom -s 20121009090100 -e 20121009110900 -c C12_iwpenGB_FR -b --sql -m | db -t

EN T2 (excluding GB) Jimmy vs Blue facts Tuesday October 9, 2012

note: removed GB from this T2 EN campaign and restarted this EN test from earlier in the morning.

campaign: http://meta.wikimedia.org/w/index.php?title=Special:CentralNotice&method=listNoticeDetail&notice=C12_iwpenT2EurAf_FR

Jimmy banner; http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Gd

Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Bd

blue banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts3

• • NOTE: Blue banner was running at 50%, each Jimmy banner was running at 25% **

+-------------------------------+-----------+---------+----------+-----------+-----------+
| banner                        | donations | amount  | amount18 | avg       | avg18     |
+-------------------------------+-----------+---------+----------+-----------+-----------+
| B12_1008_Backtobluefacts3     |       682 | 9888.51 |  7075.81 | 14.499282 | 10.375088 |
| B12_1008_Jimmyinlineappeal_Bd |       103 | 2175.41 |  1214.84 | 21.120485 | 11.794563 |
| B12_1008_Jimmyinlineappeal_Gd |       117 | 1860.64 |  1368.90 | 15.902906 | 11.700000 |
+-------------------------------+-----------+---------+----------+-----------+-----------+

ecom -s 20121009091200 -e 20121009102600 -c C12_iwpenT2EurAf_FR -b --sql -m | db -t

Japanese Jimmy Test Tuesday October 9, 2012

Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1009_JA2HR

LP (JP) :https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=ja&country=JP&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1009_JA2HR_FR&utm_campaign=none_FR

+----------------+-----------+---------+----------+-----------+-----------+
| banner         | donations | amount  | amount18 | avg       | avg18     |
+----------------+-----------+---------+----------+-----------+-----------+
| B12_1009_JA2HR |       152 | 4373.23 |  2221.48 | 28.771250 | 14.615000 |
+----------------+-----------+---------+----------+-----------+-----------+

ecom -s 20121009071500 -e 20121009120000 -c C12_iwpjaWW_FR -b --sql -m | db -t+

2011 / 2012 Comparison

+----------+---------+----------+----------+---------+----------+
| 11_count | 11_amt  | 11_amt18 | 12_count | 12_amt  | 12_amt18 |
+----------+---------+----------+----------+---------+----------+
|      195 | 6386.34 |  2882.21 |      139 | 4036.25 |  2031.64 |
+----------+---------+----------+----------+---------+----------+

October 09, 2012 Tuesday US Dropdown tab test ("Please help" vs "Personal appeal") & highlighting the $20 average

We ran a multivariate test in the US today. We're testing a new tab for our dropdown banner with the text "Please read a personal appeal from Wikipedia's founder" against our control "Please help" tab.

We also tested two versions of the dropdown landing page: one with the $20 average amount highlighted, and one without $20 highlighted.

The banners:

• Personal appeal tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_PersonalAppeal_Tab
• Personal appeal tab / average donation: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_PersonalAppeal_TabAve
• Please help tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1009_Plaintab2
• Please help tab // average donation: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1009_PlaintabAvg2

Results

+---------------------------+-----------+----------+----------+-----------+-----------+
| banner                    | donations | amount   | amount18 | avg       | avg18     |
+---------------------------+-----------+----------+----------+-----------+-----------+
| B12_1009_Plaintab2        |       874 | 10326.16 |  8034.16 | 11.814828 |  9.192403 |
| B12_1009_PlaintabAvg2     |       910 | 13626.00 |  9458.00 | 14.973626 | 10.393407 |
| B12_PersonalAppeal_Tab    |       856 | 10719.80 |  7923.91 | 12.523131 |  9.256904 |
| B12_PersonalAppeal_TabAve |       790 | 10334.86 |  7951.43 | 13.082101 | 10.065101 |
+---------------------------+-----------+----------+----------+-----------+-----------+

ecom -c C12_US_1009_Avg2 -s 20121009142300 -e 201210092000 --sql | db -t

Combined results

Commentary: SO! What a great day (see also below). Above you see one interesting discovery and one exciting discovery. First, we have two great banners that work well for us: "Please read a personal appeal from Wikipedia founder" and a long "Facts" banner that makes the big points from our traditional appeal right in-your-face in the banner. Both work great. So wouldn't they work even better if we combined them in the same banner? We combined them today. But the combo didn't do well. It didn't do horribly, but significant drop vs the plain facts. (These Facts are out-performing the appeal banner in the US btw).

Second, we have been looking for a way to bring up the average donation. Last year we basically doubled our number of donations while not-quite halving our average. We're talking people to make a lower donation than they might actually favor doing with the $5 line. So we tried indicating that our average donation is about $20 on the landing page. It worked! The normalized average went up about $1.

EN T1 $5 vs "price of a sandwich" Tuesday October 9. 2012

We are really excited about this test for two reasons:

1) The $5 line that we've been using is great for bringing in a ton of donations -- like tons of $5 donations! But that brings the average donation down too far. We're talking a lot of people who would probably like to give $20 into giving $5. By saying, "price of a sandwich" instead of $5, it makes it a sliding scale: how much does your typical sandwich cost? For some people $5, for others $10, and it wasn't that long ago that I had a sandwich for less than a dollar while traveling in a low-income country. And lo and behold, sandwich brought up the average by a whole dollar (looking at avg18) and two looking at plain avg (which can be thrown off by one large donation.)

2) The $5 line has another problem: We have to convert it into different currencies and amounts in different countries. In fact, testing late last night (for me) and early in the morning (for Megan) we forgot about this and subjected millions in Africa and Europe to "$5".

Next to try: "A couple cups of coffee", "A movie ticket", what else?

Banners:

• $5 facts: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Backtobluefacts2

• sandwich facts: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_BacktobluefactsSand

+------------------------------+-----------+---------+----------+-----------+-----------+
| banner                       | donations | amount  | amount18 | avg       | avg18     |
+------------------------------+-----------+---------+----------+-----------+-----------+
| B12_1008_Backtobluefacts2    |       509 | 7459.98 |  4961.57 | 14.656149 |  9.747682 |
| B12_1008_BacktobluefactsSand |       506 | 8360.60 |  5441.11 | 16.522925 | 10.753182 |
+------------------------------+-----------+---------+----------+-----------+-----------+>

ecom -c C12_iwpenT1_FR -s 20121009150000 -e 20121009200000 --sql -m | db -t

October 11, 2012 Thursday US Sandwich & Average Test

We're working to increase our average donation in our testing this week. Over the last couple of testing months, we found a line that brings in a lot of donations, but lowers our average donation ("If everyone reading this donated $5, the fundraiser would be done in an hour.") We saw some promising results from the "price of a sandwich" test (update above) that ran outside of the US, so we wanted to confirm those results with a US test.

We also re-tested highlighting the $20 average on our donation page.

The banners:

• $5 / no average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_FiveNoavg
• $5 / average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_FiveAvg
• Sandwich / no average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_SandNoavg
• Sandwhich / average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_SandNoavg

The Results

+--------------------+-----------+---------+----------+-----------+-----------+
| banner             | donations | amount  | amount18 | avg       | avg18     |
+--------------------+-----------+---------+----------+-----------+-----------+
| B12_1010_FiveAvg   |       630 | 9558.00 |  6648.00 | 15.171429 | 10.552381 |
| B12_1010_FiveNoavg |       624 | 8010.00 |  5783.00 | 12.836538 |  9.267628 |
| B12_1010_SandAvg   |       550 | 9841.73 |  6507.73 | 17.894055 | 11.832236 |
| B12_1010_SandNoavg |       633 | 9310.00 |  6272.00 | 14.707741 |  9.908373 |
+--------------------+-----------+---------+----------+-----------+-----------+

Combined Results

October 12, Friday US No Google line & Average Test

Today we tested a banner without the Google/Yahoo comparison. We also tested the new "average donation" page against our control landing page. (When we tested "average donation" page earlier this week, we eliminated the $250 radio button option.)

Banners

• No Google // no average: http://en.wikipedia.org/wiki/Extraterrestrial_life?banner=B12_1011_FactsNoGNAve

• No Google // average: http://en.wikipedia.org/wiki/Extraterrestrial_life?banner=B12_1011_FactsNoGAve

• Google // no average: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1012_FactsGNAve

• Google // average: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1012_FactsGAve

Results

+-----------------------+-----------+---------+----------+-----------+-----------+
| banner                | donations | amount  | amount18 | avg       | avg18     |
+-----------------------+-----------+---------+----------+-----------+-----------+
| B12_1011_FactsNoGAve  |       332 | 5259.50 |  3737.50 | 15.841867 | 11.257530 |
| B12_1011_FactsNoGNAve |       397 | 6530.00 |  4051.00 | 16.448363 | 10.204030 |
| B12_1012_FactsGAve    |       469 | 8258.61 |  5504.61 | 17.608977 | 11.736908 |
| B12_1012_FactsGNAve   |       468 | 6738.00 |  4731.00 | 14.397436 | 10.108974 |
+-----------------------+-----------+---------+----------+-----------+-----------+

Combined Results

October 16, 2012 Tuesday US "No government funds"

We found a new line for our facts banner that actually does a bit better than the Google/Yahoo comparison that has performed very well over the past year.

The banners

• Control Google/Yahoo banner: http://en.wikipedia.org/wiki/Google?banner=B12_1016_FactsGoog
• No government funds banner: http://en.wikipedia.org/wiki/Government?banner=B12_1016_FactsGovmore

The results

+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+
|         banner        | donations | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+
|   B12_1016_FactsGoog  |    1520   |  79%  |  62%  |  79%  | 19336.82 | 14235.19 | 12.721592 |  9.365257 |
| B12_1016_FactsGovmore |    1680   |  84%  |  61%  |  77%  | 23991.12 | 17488.00 | 14.280429 | 10.409524 |
+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+ 

ecom -c C12_US_1016_GgvGv -s 20121016133800 -e 20121016173800 http://www.thumbtack.com/labs/abba/#B12_1016_FactsGoog=1520%2C1000000&B12_1016_FactsGovmore=1680%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

Great news! Not only could this allow us to retire the Google line in the banner, but it also looks like this new text even performs a little better.

We're brainstorming some new versions to try to get our average back up. (Since we've been using the $5 line in the banner, we've increased the number of donations but decreased the average amount.)

More testing to come later this week.

October 19, 2012 US First Editor Test!

We've tested our first editor appeal of 2012. Last year, the messages from editors gave the campaign personality and introduced our volunteer community to our readers. It was the fun part of the fundraiser and the comments we got from readers were really inspiring.

Our first editor this year is a retired math teacher from India. Poongothai writes articles on quadratic functions, probability, charts, and graphs.

We tested Poongothai's appeal coming from our facts dropdown banner. Next step is to test a new version of the facts banner that includes her photo.

The banners

• Control facts banner: http://en.wikipedia.org/wiki/Fact?banner=B12_1018_FactsGovmore
• Editor banner: http://en.wikipedia.org/wiki/Editor?banner=B12_1018_FactsEditor

The Results

+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+
|         banner        | donations | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+
|  B12_1018_FactsEditor |    768    |  42%  |  62%  |  79%  | 12558.00 | 8228.00  | 16.351563 | 10.713542 |
| B12_1018_FactsGovmore |    784    |  45%  |  66%  |  78%  | 12164.08 | 8160.08  | 15.515408 | 10.408265 |
+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+

ecom -c C12_US_1015_Editor -s 20121019130500 -e 20121019160500

http://www.thumbtack.com/labs/abba/#B12_1018_FactsEditor=768%2C1000000&B12_1018_FactsGovmore=784%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

Those results are really encouraging! In previous years, our editor banners always received great feedback from our readers, but they always brought in fewer donations than our standard founder appeal. In our test this morning, the Jimmy appeal and the editor appeal performed just about the same. Both appeals had the exact same banner, so it looks like that facts banner is really doing the work of compelling readers to donate, and that the message we put on the appeal page has a lesser impact than it did when we used the "personal appeal" banners last year. Great first editor test!

Next week, we'll test a new editor banner that incorporates the facts along with the photo and an invitation to read the editor message. Can't wait.

October 22, 2012 US Comparing where we're at to 2011

We duplicated a campaign setup from 2011 to see how our banner impressions, click rate, and donation rates are comparing to last year when we hold the creative content constant.

Here's the setup from 2011:

3 jimmy banners:

• http://en.wikipedia.org/wiki/Main_Page?banner=B11_1114_JKwfCf_US
• http://en.wikipedia.org/wiki/Main_Page?banner=B11_1114_JKwfLf_US
• http://en.wikipedia.org/wiki/Main_Page?banner=B11_Foundation_Jimmy_KevinSpaceywhite_US

1 landing page: http://wikimediafoundation.org/wiki/L11_1112_JBT/en/US

The setup from 2012:

3 Jimmy banners:

• http://en.wikipedia.org/wiki/Main_Page?banner=B12_1022_JKwfCf_US
• http://en.wikipedia.org/wiki/Main_Page?banner=B12_1022_JKwfLf_US
• http://en.wikipedia.org/wiki/Main_Page?banner=B12_Foundation_Jimmy_KevinSpaceywhite_US

1 landing page: https://donate.wikimedia.org/wiki/L11_1112_JBT/en/US

2011 US Donation Results

+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|                  banner                  | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|            B11_1114_JKwfCf_US            |    497    |  722   |  41%  |  59%  |  78%  | 11600.00 | 7449.00  | 23.340040 | 14.987928 |
|            B11_1114_JKwfLf_US            |    513    |  772   |  44%  |  55%  |  79%  | 13978.00 | 7655.00  | 27.247563 | 14.922027 |
| B11_Foundation_Jimmy_KevinSpaceywhite_US |    536    |  834   |  49%  |  58%  |  71%  | 12618.00 | 7965.00  | 23.541045 | 14.860075 |
+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+

ecom -c C11_1114_CFLFKS_US -s 20111114182500 -e 20111114192700 -g b

2012 US Donation Results

 +------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|                  banner                  | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|            B12_1022_JKwfCf_US            |    323    |  533   |  45%  |  54%  |  68%  | 8594.30  | 4922.30  | 26.607740 | 15.239319 |
|            B12_1022_JKwfLf_US            |    344    |  541   |  52%  |  59%  |  69%  | 9002.54  | 5076.54  | 26.170174 | 14.757384 |
| B12_Foundation_Jimmy_KevinSpaceywhite_US |    314    |  522   |  51%  |  55%  |  66%  | 10153.35 | 4807.00  | 32.335510 | 15.308917 |
+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+

ecom -c C12_US_1022_RepTst -s 20121022172500 -e 20121022182700 -g b

2011 Banner Impressions

 | utm_source                               | sum(counts) |
+------------------------------------------+-------------+
| B11_1114_JKwfCf_US                       |     1981114 |
| B11_1114_JKwfLf_US                       |     1982674 |
| B11_Foundation_Jimmy_KevinSpaceywhite_US |     1979817 | 

2012 Banner Impressions

+------------------------------------------+------------+
| banner                                   | sum(count) |
+------------------------------------------+------------+
| B12_1022_JKwfCf_US                       |    1725400 |
| B12_1022_JKwfLf_US                       |    1694200 |
| B12_Foundation_Jimmy_KevinSpaceywhite_US |    1717600 |
+------------------------------------------+------------+

2011 Landing page impressions

 +-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|  lpi  |                  banner                  | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
| 31199 | B11_Foundation_Jimmy_KevinSpaceywhite_US |    536    |  834   |   0%  |   0%  |  71%  | 12618.00 | 7965.00  | 23.541045 | 14.860075 |
| 30669 |            B11_1114_JKwfLf_US            |    513    |  772   |   0%  |   0%  |  79%  | 13978.00 | 7655.00  | 27.247563 | 14.922027 |
| 30279 |            B11_1114_JKwfCf_US            |    497    |  722   |   0%  |   0%  |  78%  | 11600.00 | 7449.00  | 23.340040 | 14.987928 |
+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+

ecom -c C11_1114_CFLFKS_US -s 20111114182500 -e 20111114192700 -g b -i

2012 Landing page impressions

+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|  lpi  |                  banner                  | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
| 20642 | B12_Foundation_Jimmy_KevinSpaceywhite_US |    314    |  522   |  51%  |  55%  |  66%  | 10153.35 | 4807.00  | 32.335510 | 15.308917 |
| 20355 |            B12_1022_JKwfLf_US            |    344    |  541   |  52%  |  59%  |  69%  | 9002.54  | 5076.54  | 26.170174 | 14.757384 |
| 20077 |            B12_1022_JKwfCf_US            |    323    |  533   |  45%  |  54%  |  68%  | 8594.30  | 4922.30  | 26.607740 | 15.239319 |
+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+

ecom -c C12_US_1022_RepTst -s 20121022172500 -e 20121022182700 -g b -i

October 22, 2012 EN T1 Comparison from Oct. 8, 2012 Test

Test setup from Oct. 8, 2012 and Oct. 22, 2012:

• Jimmy banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Jimmyinlineappeal2
• Blue facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Backtobluefacts2

Landing page (CA): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=CA&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1008_Backtobluefacts_undefined&utm_campaign=C12_iwpenT1_FR_undefined

Facts landing page (CA running from Jimmy banner): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=CA&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-facts&infobox=Appeal-default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1008_Jimmyinlineappeal2_CA&utm_campaign=C12_iwpenT1_FR_CA

October 8, 2012 Results:

+------+---------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |              banner             | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+---------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 4742 |   B12_1008_Jimmyinlineappeal2   |    104    |  196   |  44%  |  45%  |  30%  | 1956.78 | 1248.23  | 18.815192 | 12.002212 |
| 3537 |    B12_1008_Backtobluefacts2    |    309    |  568   |  34%  |  38%  |  69%  | 4046.52 | 2855.49  | 13.095534 |  9.241068 |
+------+---------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -c C12_iwpenT1_FR -s 20121008180100 -e 20121008190000 -g b -m -i

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=104%2C1000000&B12_1008_Backtobluefacts2=309%2C1000000&B12_1008_Backtobluefacts=9%2C1000000&B12_1008_Jimmyinlineappeal=3%2C1000000&B12_1008_Backtobluefacts_undefi=0%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true d/lpi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=104%2C4742&B12_1008_Backtobluefacts2=309%2C3537&B12_1008_Backtobluefacts=9%2C167&B12_1008_Jimmyinlineappeal=3%2C81&B12_1008_Backtobluefacts_undefi=0%2C2&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true lpi/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=4742%2C1000000&B12_1008_Backtobluefacts2=3537%2C1000000&B12_1008_Backtobluefacts=167%2C1000000&B12_1008_Jimmyinlineappeal=81%2C1000000&B12_1008_Backtobluefacts_undefi=2%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |               landingpage               | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 6100 | default~default~default~default~control |    377    |  706   |  35%  |  39%  |  58%  | 5242.07 | 3615.28  | 13.904695 |  9.589602 |
| 2429 |  default~default~facts~default~control  |     48    |   88   |  46%  |  45%  |  30%  |  880.13 |  597.37  | 18.336042 | 12.445208 |
+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -c C12_iwpenT1_FR -s 20121008180100 -e 20121008190000 -g l -m -i

Oct. 22 Results:

+------+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |            banner           | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 5722 | B12_1008_Jimmyinlineappeal2 |    103    |  228   |  48%  |  38%  |  24%  | 2086.04 | 1133.73  | 20.252816 | 11.007087 |
| 3555 |  B12_1008_Backtobluefacts2  |    308    |  539   |  37%  |  43%  |  70%  | 4296.99 | 3008.93  | 13.951266 |  9.769253 |
+------+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -c C12_iwpenT1_FR -s 20121022180100 -e 20121022190000 -g b -m -i

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=103%2C1000000&B12_1008_Backtobluefacts2=308%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true d/lpi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=103%2C5722&B12_1008_Backtobluefacts2=308%2C3555&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true lpi/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=5722%2C1000000&B12_1008_Backtobluefacts2=3555%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |               landingpage               | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 6430 | default~default~default~default~control |    357    |  645   |  39%  |  43%  |  57%  | 5243.58 | 3531.56  | 14.687899 |  9.892325 |
| 2847 |  default~default~facts~default~control  |     54    |  122   |  44%  |  36%  |  25%  | 1139.45 |  611.10  | 21.100926 | 11.316667 |
+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -c C12_iwpenT1_FR -s 20121022180100 -e 20121022190000 -g l -m -i

Oct. 22 Banner Impressions:

+-----------------------------+------------+
| banner                      | sum(count) |
+-----------------------------+------------+
| B12_1008_Backtobluefacts2   |     459200 |
| B12_1008_Jimmyinlineappeal2 |     474600 |
+-----------------------------+------------+

(still need Oct. 8 banner impressions)

Oct. 23, 2012 GB Compare test

Setup from Oct. 9 and Oct. 23:

• Jimmy banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Jimmyinlineappeal_Bd_GB
• Jimmy banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Jimmyinlineappeal_Gd_GB
• Blue facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Backtobluefacts3_GB

October 9 Results

+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |              banner              | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 8240 | B12_1008_Jimmyinlineappeal_Gd_GB |    137    |  245   |  36%  |  49%  |  36%  | 2812.71 | 2020.81  | 20.530730 | 14.750438 |
| 7016 | B12_1008_Jimmyinlineappeal_Bd_GB |    141    |  248   |  39%  |  50%  |  35%  | 2671.65 | 2009.93  | 18.947872 | 14.254823 |
| 4749 |   B12_1008_Backtobluefacts3_GB   |    460    |  688   |  34%  |  59%  |  72%  | 8059.08 | 6160.17  | 17.519739 | 13.391674 |
+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -s 20121009090100 -e 20121009103000 -c C12_iwpenGB_FR -g b -m -i

October 23 Results

 +------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |              banner              | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 7438 | B12_1008_Jimmyinlineappeal_Gd_GB |    124    |  228   |  35%  |  40%  |  36%  | 2522.54 | 1869.95  | 20.343065 | 15.080242 |
| 6853 | B12_1008_Jimmyinlineappeal_Bd_GB |    124    |  229   |  34%  |  40%  |  36%  | 2628.93 | 1781.16  | 21.201048 | 14.364194 |
| 4392 |   B12_1008_Backtobluefacts3_GB   |    391    |  583   |  33%  |  56%  |  75%  | 6327.43 | 4966.44  | 16.182685 | 12.701893 |
+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -s 20121023090100 -e 20121023103000 -c C12_iwpenGB_FR -g b -m -i

• Still need banner impression comparison

October 23, 2012 ENT4 Jimmy vs Blue Facts Test

Test setup:

• Jimmy banner: http://en.wikipedia.org/wiki/Australia?banner=B12_1024_T4_Proto_Jimmy
• Blue facts: http://en.wikipedia.org/wiki/Australia?banner=B12_1024_T4_Proto_FactsBlueSWDonate
• Landing page (Australia): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=AU&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1024_T4_Proto_Jimmy_US&utm_campaign=none_US

Full T4 Results

+------+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| lpi  |                banner               | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+------+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| 3970 |       B12_1024_T4_Proto_Jimmy       |    100    |  199   |  49%  |  55%  |  70%  | 3922.20 | 1291.96  | 39.222000 | 12.919600 |
| 2366 | B12_1024_T4_Proto_FactsBlueSWDonate |    259    |  423   |  41%  |  57%  |  79%  | 3545.10 | 2547.98  | 13.687645 |  9.837761 |
+------+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

ecom -s 20121023220000 -e 20121023230000 -c C12_iwpenT4EastAsia_FR -g b -m -i

AU & NZ BANNER breakdown:

+------+-------------------------------------+----------+-----------+--------+-------+-------+-------+---------+----------+------------+-----------+
| lpi  |                banner               | iso_code | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg     |   avg18   |
+------+-------------------------------------+----------+-----------+--------+-------+-------+-------+---------+----------+------------+-----------+
| 2621 |       B12_1024_T4_Proto_Jimmy       |    AU    |     77    |   77   |  43%  |  100% |  100% | 3451.08 |  965.28  | 44.819221  | 12.536104 |
| 1514 | B12_1024_T4_Proto_FactsBlueSWDonate |    AU    |    186    |  186   |  35%  |  100% |  100% | 2708.22 | 2011.17  | 14.560323  | 10.812742 |
| 745  |       B12_1024_T4_Proto_Jimmy       |    NZ    |     19    |   19   |  74%  |  100% |  100% |  417.81 |  275.80  | 21.990000  | 14.515789 |
| 1514 | B12_1024_T4_Proto_FactsBlueSWDonate |    NZ    |     61    |   61   |  59%  |  100% |  100% |  500.19 |  411.19  |  8.199836  |  6.740820 |
+------+-------------------------------------+----------+-----------+--------+-------+-------+-------+---------+----------+------------+-----------+

ecom -s 20121023220000 -e 20121023230000 -c C12_iwpenT4EastAsia_FR -g bc -m -i

AU & NZ Banner Impression breakdown

+-------------------------------------+---------+----------+
|                banner               | country | ban_imps |
+-------------------------------------+---------+----------+
| B12_1024_T4_Proto_FactsBlueSWDonate |    AU   |   1377   |
|       B12_1024_T4_Proto_Jimmy       |    AU   |   1342   |
| B12_1024_T4_Proto_FactsBlueSWDonate |    NZ   |   386    |
|       B12_1024_T4_Proto_Jimmy       |    NZ   |   390    |
+-------------------------------------+---------+----------+

ecom -s 20121023220000 -e 20121023230000 -c C12_iwpenT4EastAsia_FR -g bc -m -i

Time Machine Comparison to 2011 for AU

• PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.

+----------+---------+----------+-----------+----------+---------+----------+-----------+
| 11_count |  11_amt | 11_amt18 |  11_avg18 | 12_count |  12_amt | 12_amt18 |  12_avg18 |
+----------+---------+----------+-----------+----------+---------+----------+-----------+
|   263    | 4952.75 | 3357.46  | 12.766008 |   254    | 5880.73 | 2855.49  | 11.242087 |
+----------+---------+----------+-----------+----------+---------+----------+-----------+

timemachine -c C12_iwpenT4EastAsia_FR --sd11 20111116 --sd12 20121023 --st 220000 --et 230000 --country AU --lang en

• PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
  • If we take just the Jimmy banner results for 2012 and multiply the donations by 2, we can estimate what our AU total would be if we ran Jimmy at 100% in 2012.
  • 77 donations X 2 = 154 compared to 263 donations in 2011
  • $965 Amt18 X 2 = $1,930 compared to $3,357 in 2011

Time Machine Comparison to 2011 for NZ

• PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.

Comparison to Nov. 16, 2011

+----------+--------+----------+-----------+----------+--------+----------+----------+
| 11_count | 11_amt | 11_amt18 |  11_avg18 | 12_count | 12_amt | 12_amt18 | 12_avg18 |
+----------+--------+----------+-----------+----------+--------+----------+----------+
|    20    | 208.29 |  204.46  | 10.223000 |    78    | 909.80 |  678.79  | 8.702436 |
+----------+--------+----------+-----------+----------+--------+----------+----------+

timemachine -c C12_iwpenT4EastAsia_FR --sd11 20111116 --sd12 20121023 --st 220000 --et 230000 --country NZ --lang en

• PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
  • If we take just the Jimmy banner results for 2012 and multiply the donations by 2, we can estimate what our NZ total would be if we ran Jimmy at 100% in 2012.
  • 19 donations X 2 = 38 donations compared to 20 donations in 2011
  • $275.80 Amount18 X 2 = $551.6 compared to $204.46 in 2011

Comparison to Nov. 17, 2011:

+----------+--------+----------+----------+----------+--------+----------+----------+
| 11_count | 11_amt | 11_amt18 | 11_avg18 | 12_count | 12_amt | 12_amt18 | 12_avg18 |
+----------+--------+----------+----------+----------+--------+----------+----------+
|    17    | 146.59 |  145.40  | 8.552941 |    78    | 909.80 |  678.79  | 8.702436 |
+----------+--------+----------+----------+----------+--------+----------+----------+

timemachine -c C12_iwpenT4EastAsia_FR --sd11 20111117 --sd12 20121023 --st 220000 --et 230000 --country NZ --lang en

• PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
  • If we take just the Jimmy banner results for 2012 and multiply the donations by 2, we can estimate what our NZ total would be if we ran Jimmy at 100% in 2012.
  • 19 donations X 2 = 38 donations compared to 20 donations in 2011
  • $275.80 Amount18 X 2 = $551.6 compared to $204.46 in 2011

Friday November 9, 2012 US Paypal credit card option

We tested different presentations of payment methods in the US.

Page A had three buttons:

• Credit Card (credit card donations were processed through Global Collect)
• PayPal
• Amazon

Page B had two buttons:

• PayPal or Credit Card (credit card donations were processed through PayPal)
• Amazon

Results

We don't have confidence on the results for number of donations, but it looks like processing our credit card donations through PayPal did not help increase donations. We will test this again outside the US, particularly in countries where the proportion of PayPal donations is higher than in the US.

ecom -s 20121109220000 -e 20121109250000 -c C12_iwpenUS_FR -g b

+------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+----------+
|         banner         | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |  avg18   |
+------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+----------+
| B12_FpcwDropBtnUS_A_US |    1840   |  2683  |  36%  |  55%  |  84%  | 27221.32 | 18315.21 | 14.794196 | 9.953918 |
| B12_FpcwDropBtnUS_B_US |    1787   |  2610  |   0%  |  None |  67%  | 24310.83 | 16951.33 | 13.604270 | 9.485915 |
+------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_FpcwDropBtnUS_A_US=1840%2C1000000&B12_FpcwDropBtnUS_B_US=1787%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

Meganhernandez (talk) 18:30, 10 November 2012 (UTC)

US Tuesday Nov. 13, 2012 Trying to get editors in the banner

We ran through a few tests this morning of different versions of our facts banner with the image of an editor.

Quick refresher: Last time we tested an editor appeal (Oct. 19), we used two identical facts banners (text-only, no images). One banner led to our standard Jimmy appeal and the other banner led to an editor appeal. The number of donations and amounts were nearly the same. Very encouraging, but the next step was to integrate the editor into the facts banner. Which is what we tried today.

Round 1:

• Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage
• Left editor image facts banner with "Please read a personal appeal from a Wikipedia Editor" tab: http://en.wikipedia.org/wiki/Horseshoe_Curve_(Pennsylvania)?banner=B12_EditorImage

Results: ecom -s 20121113164200 -e 20121113171300 -c C12_iwpenUS_FR -g b

+-------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
|       banner      | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+-------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
|  B12_EditorImage  |    105    |  146   |  46%  |  63%  |  81%  | 2151.00 | 1123.00  | 20.485714 | 10.695238 |
| B12_EditorNoImage |    201    |  289   |  39%  |  54%  |  85%  | 2837.00 | 1955.00  | 14.114428 |  9.726368 |
+-------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_EditorImage=105%2C1000000&B12_EditorNoImage=201%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

Round 2:

• Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage
• Right editor image facts banner with "Please read a personal appeal from a Wikipedia Editor" tab: http://en.wikipedia.org/wiki/Rights?banner=B12_EditorImageRight

Results:

ecom -s 20121113171300 -e 20121113201300 -c C12_iwpenUS_FR -g b

+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
|            banner           | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

|     B12_EditorImageRight    |    137    |  209   |  35%  |  47%  |  83%  | 2398.00 | 1476.00  | 17.503650 | 10.773723 |
|      B12_EditorNoImage      |    259    |  388   |  33%  |  48%  |  82%  | 4047.00 | 2662.00  | 15.625483 | 10.277992 |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_EditorImageRight=137%2C1000000&B12_EditorNoImage=259%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

Round 3:

• Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage2
• Right editor image facts banner with "Support Wikipedia Editors" tab: http://en.wikipedia.org/wiki/Image?banner=B12_EditorImageRightSupport

Results:

ecom -s 20121113172800 -e 20121113202800 -c C12_iwpenUS_FR -g b
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
|            banner           | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
| B12_EditorImageRightSupport |    129    |  218   |  40%  |  46%  |  36%  | 2010.00 | 1329.00  | 15.581395 | 10.302326 |
|      B12_EditorNoImage2     |    299    |  419   |  35%  |  54%  |  86%  | 4402.00 | 3017.00  | 14.722408 | 10.090301 |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

http://www.thumbtack.com/labs/abba/#B12_EditorImageRightSupport=129%2C1000000&B12_EditorNoImage2=299%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

Round 4:

• Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage_SameTab
• Right editor image facts banner with "Please help" tab and "Support our editors" under the editor image: http://en.wikipedia.org/wiki/Support?banner=B12_EditorImage_SameTab

Results:

ecom -s 20121113175500 -e 20121113205500 -c C12_iwpenUS_FR -g b
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
|            banner           | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 |    avg    |   avg18   |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
|   B12_EditorImage_SameTab   |    435    |  697   |  42%  |  49%  |  77%  | 6220.95 | 4341.00  | 14.301034 |  9.979310 |
|  B12_EditorNoImage_SameTab  |    690    |  1050  |  41%  |  53%  |  80%  | 9600.72 | 6639.72  | 13.914087 |  9.622783 |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+

http://www.thumbtack.com/labs/abba/#B12_EditorImage_SameTab=435%2C1000000&B12_EditorNoImage_SameTab=690%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

What this means: The upside: We were able to improve the performance of the editor banner by running through different versions in testing this morning. The downside: our control facts still beat the editor banner overall. More upside: we learned a lot in just a couple hours testing this morning.

Meganhernandez (talk) 21:42, 13 November 2012 (UTC)

Meganhernandez (talk) 21:42, 13 November 2012 (UTC)

US Tuesday Nov. 13, 2012, Gov/$5 vs Gov/Sandwich

A couple months ago when we were using the "google line" in the banner, we that for the third fact, using the "If everyone gave the price of a sandwich" worked better than the "If everyone gave $5.." line.

We later found the "no government funds" banner that beat "Google line" banner. Now we're testing using the sandwich fact with the "no government funds" banner.

Banners:

• "If everyone $5": http://en.wikipedia.org/wiki/Five?banner=B12_FpcwDropIevryUS_A
• "If everyone gave the price of a sandwich" http://en.wikipedia.org/wiki/Sandwich?banner=B12_FpcwDropGovSand_A

Results

ecom -s 20121113224600 -e 20121114020000 -c C12_iwpenUS_FR -g b
+---------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|           banner          | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 |    avg    |   avg18   |
+---------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
|  B12_FpcwDropGovSand_A_US |    1346   |  2075  |  29%  |  50%  |  80%  | 19960.67 | 13045.10 | 14.829621 |  9.691753 |
|  B12_FpcwDropIevryUS_A_US |    1345   |  2096  |  30%  |  51%  |  78%  | 18875.10 | 12901.10 | 14.033532 |  9.591896 |
+---------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_FpcwDropGovSand_A_US=1346%2C1000000&B12_FpcwDropIevryUS_A_US=1345%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

US Sunday Nov. 18 Ask string vs no string in banner

• no ask string in banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_10string1118_string
• ask string in banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_10string1118_nostring

ecom -s 20121118204000 -e 20121118235900 -c C12_iwpenUS_FR -g b -m --having 5

+---------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+
|           banner          | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |    avg    |   avg18   |
+---------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+
| B12_10string1118_nostring |    2794   |  3829  |  33%  |  64%  |  81%  | 47199.29 | 31424.07 | 33056.29 | 41549.29 | 16.893089 | 11.246983 |
|  B12_10string1118_string  |    2878   |  3903  |  34%  |  66%  |  82%  | 47007.22 | 31863.22 | 33531.22 | 41195.22 | 16.333294 | 11.071306 |
+---------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_10string1118_nostring=2794%2C1000000&B12_10string1118_string=2878%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true

US Nov. 19 Super short facts banner

• http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1119control
• http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1119supershort

Nov. 21 $5 & $30 vs no $5 & $30 in the banner US, CA, AU, NZ, GB

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1120cntrl
• No $5 and $30 in the banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1120no530

US Results ecom -s 20121120225500 -e 20121121060000 -c C12_iwpenUS_FR -g b -m --having 30

+-------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+
|          banner         | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |    avg    |   avg18   |
+-------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+
| B12_shortening1120cntrl |    1239   |  1700  |  34%  |  65%  |  82%  | 18311.82 | 12395.82 | 13119.82 | 16624.82 | 14.779516 | 10.004697 |
| B12_shortening1120no530 |    1279   |  1739  |  33%  |  67%  |  83%  | 16975.70 | 11627.70 | 12179.70 | 14847.70 | 13.272635 |  9.091243 |
+-------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrl=1239%2C1000000&B12_shortening1120no530=1279%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.000075

Canada Results ecom -s 20121121000000 -e 20121121060000 -c C12_iwpenCA_FR -g b -m --having 30

+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
|           banner          | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |    avg    |  avg18   |
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
| B12_shortening1120cntrlCA |    487    |  747   |  37%  |  51%  |  78%  | 6575.96 | 4861.51  | 5123.51  | 6170.92  | 13.502998 | 9.982567 |
| B12_shortening1120no530CA |    480    |  729   |  37%  |  51%  |  80%  | 6363.67 | 4779.28  | 5051.28  | 6107.96  | 13.257646 | 9.956833 |
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrlCA=487%2C1000000&B12_shortening1120no530CA=480%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.620773

AU Results ecom -s 20121121013000 -e 20121121053000 -c C12_iwpenAU_FR -g b -m --having 3

+------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
|         banner         | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |    avg    |  avg18   |
+------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
| B12_shortening1120cntr |    377    |  606   |  31%  |  56%  |  81%  | 5225.45 | 3662.12  | 3836.83  | 4914.38  | 13.860610 | 9.713846 |
| B12_shortening1120no53 |    408    |  649   |  35%  |  64%  |  76%  | 5211.51 | 3686.36  | 3848.36  | 4796.50  | 12.773309 | 9.035196 |
+------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntr=377%2C1000000&B12_shortening1120no53=408%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.033164

NZ Results ecom -s 20121121013000 -e 20121121063000 -c C12_iwpenNZ_FR -g b -m

+---------------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+
|           banner          | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 |    avg    |  avg18   |
+---------------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+
| B12_shortening1120cntrlNZ |     55    |   77   |  73%  |  69%  |  83%  | 727.35 |  494.66  |  514.66  |  663.91  | 13.224545 | 8.993818 |
| B12_shortening1120no530NZ |     49    |   66   |  69%  |  81%  |  65%  | 527.16 |  390.00  |  402.00  |  495.44  | 10.758367 | 7.959184 |
+---------------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrlNZ=55%2C1000000&B12_shortening1120no530NZ=49%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.214758

GB Results

ecom -s 20121121110000 -e 20121121150000 -c C12_iwpenGB_FR -g b -m --having 2

+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
|           banner          | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |    avg    |   avg18   |
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
| B12_shortening1120cntrlGB |    531    |  718   |  35%  |  62%  |  83%  | 8381.65 | 6213.96  | 6419.15  | 7926.83  | 15.784652 | 11.702373 |
| B12_shortening1120no530GB |    560    |  759   |  33%  |  62%  |  81%  | 7852.73 | 6021.75  | 6127.75  | 7111.36  | 14.022732 | 10.753125 |
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrlGB=531%2C1000000&B12_shortening1120no530GB=560%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.000257

Nov. 21 Canada test "We run on YOUR donations"

We tested a new variation of the facts banner addressing the reader as "you" in the banner.

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122cntrlCA
• We run on your donations...If you and everyone else gave $5: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122youCA

ecom -s 20121122013000 -e 20121122653000 -c C12_iwpenCA_FR -g b -m --having 5

+-----------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
|      banner     | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |    avg    |   avg18   |
+-----------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
| B12_1122cntrlCA |    310    |  461   |  43%  |  58%  |  76%  | 4836.89 | 3139.22  | 3323.22  | 4357.27  | 15.602871 | 10.126516 |
|  B12_1122youCA  |    298    |  430   |  43%  |  60%  |  78%  | 4549.15 | 3277.48  | 3490.98  | 4296.16  | 15.265604 | 10.998255 |
+-----------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlCA=310%2C1000000&B12_1122youCA=298%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.120951

Nov. 22 $5 fact in center

We tested putting this fact in the center of the banner instead of on the right side of the banner: "If everyone reading this gave $5, the fundraiser would be done within an hour. Please help us forget fundraising and get back to Wikipedia."

Australia:

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1121cntrlAU
• $5 centered: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1121mid5AU

ecom -s 20121122013000 -e 20121122053000 -c C12_iwpenAU_FR -g b -m

ecom -s 20121122013000 -e 20121122053000 -c C12_iwpenAU_FR -g b -m
+--------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
|    banner    | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |    avg    |  avg18   |
+--------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
| B12_1121cntr |    381    |  613   |  34%  |  59%  |  78%  | 5935.32 | 3798.50  | 4004.50  | 5406.71  | 15.578268 | 9.969816 |
| B12_1121mid  |    331    |  544   |  31%  |  59%  |  78%  | 4241.59 | 3109.23  | 3255.23  | 4106.84  | 12.814471 | 9.393444 |
+--------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1121cntr=381%2C1000000&B12_1121mid=331%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.136039

New Zealand: ecom -s 20121122013000 -e 20121122053000 -c C12_iwpenNZ_FR -g b -m

+-----------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+
|      banner     | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 |    avg    |  avg18   |
+-----------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+
| B12_1121cntrlNZ |     57    |   77   |  67%  |  69%  |  86%  | 777.49 |  514.37  |  530.37  |  623.94  | 13.640175 | 9.024035 |
|  B12_1121mid5NZ |     34    |   52   |  62%  |  60%  |  81%  | 440.44 |  250.17  |  256.17  |  286.89  | 12.954118 | 7.357941 |
+-----------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1121cntrlNZ=57%2C1000000&B12_1121mid5NZ=34%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.067449

Nov. 22 GB "If Wikipedia is useful to you"

When we ask donors why they gave to Wikipedia, they usually say they donate because they use Wikipedia and want to support something they find so useful. We tested a new line incorporating this idea: "If Wikipedia is useful to you, please help keep it online another year."

• Control banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122cntrlGB
• If Wikipedia is useful: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122usefulGB

ecom -s 20121122110000 -e 20121122160000 -c C12_iwpenGB_FR -g b -m --having 5

+------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
|      banner      | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |    avg    |   avg18   |
+------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
| B12_1122cntrlGB  |    466    |  660   |  35%  |  59%  |  79%  | 9169.65 | 5752.79  | 5958.79  | 7691.34  | 19.677361 | 12.345043 |
| B12_1122usefulGB |    514    |  716   |  33%  |  56%  |  83%  | 9408.20 | 6095.12  | 6309.12  | 8040.05  | 18.303891 | 11.858210 |
+------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlGB=466%2C1000000&B12_1122usefulGB=514%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.144127

The sample was too small here to see a significant difference, but we'll test this idea again to get more data.

Nov. 24 Top facts vs. dropdown small top bar

So this seems to be a HUGE breakthrough. It's a smaller banner -- much smaller. And I think the grey tab finally makes these top banners look slightly normal and not completely jarring. So: smaller, les jarring. And it does way better! But why? It's probably not the blue info icon -- at least not that alone. A short test, that we'll lengthen tomorrow, seemed to rule that out. But maybe all the differences with this banner are just working in the same direction towards more clicks and donations: the info icon, the nicer tab, the wide-column text. And/or maybe the big factor is that the "system alert" look signifies a direct message from the site's (or browser's) owner. Only the people who run the site your on have the power put an alert like this up there -- so maybe that makes it stand out as a non-BS piece of info that is definitely not going to trick you into reading some ad. More tomorrow.

United States:

• Control top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1123_Control2
• Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1123_Twolineinfo

ecom -s 20121123193500 -e 20121123232200 -c C12_iwpenUS_FR -g b -m --having 50

+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|        banner        | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|  B12_1123_Control2   |    929    |  1221  |  36%  |  69%  |  83%  | 15204.00 | 9732.00  | 10344.00 | 13984.00 | 200.00 | 16.365985 | 11.134553 |
| B12_1123_Twolineinfo |    1269   |  1691  |  32%  |  66%  |  82%  | 18910.92 | 12389.92 | 13089.92 | 16885.92 | 200.00 | 14.902222 | 10.315146 |
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1123_Control2=929%2C1000000&B12_1123_Twolineinfo=1269%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.002240

United States:

• Top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1123newfactsinfodot_Threecol
• Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1123newfactsinfodot_OnecolSmall

ecom -s 20121123214000 -e 20121123230000 -c C12_iwpenUS_FR -g b -m --having 100

+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
|                banner               | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
| B12_1123newfactsinfodot_OnecolSmall |    602    |  817   |  34%  |  68%  |  80%  | 9173.12 | 6247.12  | 6641.12  | 8523.12  | 200.00 | 15.237741 | 11.031761 |
|   B12_1123newfactsinfodot_Threecol  |    290    |  393   |  30%  |  62%  |  83%  | 5084.20 | 3057.20  | 3249.20  | 4434.20  | 200.00 | 17.531724 | 11.204138 |
+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+

Canada:

• Control top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1122cntrlCA
• Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1124_Info_DropCA

 ecom -s 20121124010000 -e 20121124040000 -c C12_iwpenCA_FR -g b -m --having 2
+----------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+---------+-----------+-----------+
|        banner        | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |   max   |    avg    |   avg20   |
+----------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+---------+-----------+-----------+
|   B12_1122cntrlCA    |    328    |  456   |  39%  |  59%  |  83%  | 5177.44 | 3554.99  | 3792.99  | 4771.36  |  250.95 | 15.784878 | 11.563994 |
| B12_1124_Info_DropCA |    562    |  845   |  40%  |  55%  |  77%  | 9270.17 | 5744.08  | 6070.64  | 7707.44  | 1003.80 | 16.494964 | 10.801851 |
+----------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+---------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlCA=328%2C1000000&B12_1124_Info_DropCA=562%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.294505

New Zealand:

• Control top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1121cntrlNZ
• Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1124_Info_DropNZ

ecom -s 20121124010000 -e 20121124040000 -c C12_iwpenNZ_FR -g b -m

+----------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-------+-----------+----------+
|        banner        | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 |  max  |    avg    |  avg20   |
+----------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-------+-----------+----------+
|   B12_1121cntrlNZ    |     36    |   50   |  75%  |  71%  |  75%  | 388.82 |  340.49  |  346.49  |  388.82  | 40.93 | 10.800556 | 9.624722 |
| B12_1124_Info_DropNZ |     64    |  100   |  70%  |  64%  |  63%  | 601.67 |  523.01  |  537.01  |  601.67  | 40.93 |  9.401094 | 8.390781 |
+----------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-------+-----------+----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1121cntrlNZ=36%2C1000000&B12_1124_Info_DropNZ=64%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.361465

UK:

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122cntrlGB
• Top small facts dropdown bar: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1124_Info_DropGB

ecom -s 20121124110000 -e 20121124150000 -c C12_iwpenGB_FR -g b -m

+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|        banner        | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|   B12_1122cntrlGB    |    446    |  590   |  37%  |  63%  |  85%  | 8732.38  | 5397.94  | 5603.94  | 7303.74  | 399.10 | 19.579327 | 12.564888 |
| B12_1124_Info_DropGB |    759    |  1033  |  31%  |  61%  |  81%  | 11936.84 | 8553.42  | 8803.42  | 10881.70 | 159.64 | 15.727062 | 11.598709 |
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlGB=446%2C1000000&B12_1124_Info_DropGB=759%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.001675

Nov. 24 Blue info dot

• Blue dot: http://en.wikipedia.org/wiki/Blue?banner=B12_112418_Control
• No blue dot: http://en.wikipedia.org/wiki/Optical_spectrum?banner=B12_112418_Nodot

ecom -s 20121124180500 -e 20121124200500 -c C12_iwpenUS_FR -g b -m --having 20

+--------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|       banner       | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+--------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
| B12_112418_Control |    1808   |  2415  |  33%  |  65%  |  83%  | 29264.55 | 18624.54 | 19794.54 | 26404.55 | 250.00 | 16.186145 | 10.948308 |
|  B12_112418_Nodot  |    1605   |  2250  |  33%  |  62%  |  80%  | 25925.40 | 16773.61 | 17823.61 | 23505.40 | 200.00 | 16.152897 | 11.105053 |
+--------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112418_Control=1808%2C1000000&B12_112418_Nodot=1605%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.424329

Nov. 24 shorter text bar CA

• Control: http://en.wikipedia.org/wiki/Optical_spectrum?banner=B12_112420_Info_DropCA
• Shorter text: http://en.wikipedia.org/wiki/Optical_spectrum?banner=B12_112420_Info_DropShortCA

ecom -s 20121124200000 -e 20121124220000 -c C12_iwpenCA_FR -g b -m --having 2 http://www.jobquotes.com

+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
|            banner           | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
|    B12_112420_Info_DropCA   |    548    |  789   |  39%  |  57%  |  80%  | 8056.07 | 5381.39  | 5689.39  | 7246.76  | 501.90 | 14.700858 | 10.382099 |
| B12_112420_Info_DropShortCA |    478    |  703   |  38%  |  58%  |  76%  | 7577.93 | 5078.32  | 5402.32  | 7168.43  | 100.38 | 15.853410 | 11.301925 |
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112420_Info_DropCA=548%2C1000000&B12_112420_Info_DropShortCA=478%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.022577

Nov. 25 AU 2 lines vs 3 lines top bar

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1225_Info_DropAU
• Shorter 2 lines: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1225_Info_DropShortAU

ecom -s 20121125000000 -e 20121125030000 -c C12_iwpenAU_FR -g b -m --having 2

+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
|           banner          | donations | clicks | ccpct | cccnv | ppcnv |  amount | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
|    B12_1225_Info_DropAU   |    375    |  563   |  25%  |  58%  |  82%  | 5821.80 | 3993.42  | 4231.42  | 5500.86  | 104.57 | 15.524800 | 11.283787 |
| B12_1225_Info_DropShortAU |    336    |  486   |  29%  |  72%  |  83%  | 6357.62 | 3745.91  | 4001.91  | 5543.24  | 209.14 | 18.921488 | 11.910446 |
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1225_Info_DropAU=375%2C1000000&B12_1225_Info_DropShortAU=336%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.267284

Nov. 25 GB "If Wikipedia is useful to you.."

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1125_Info_DropGB
• Useful: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1125_Info_DropUsefulGB

ecom -s 20121125110000 -e 20121125145000 -c C12_iwpenGB_FR -g b -m --having 10

+----------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|           banner           | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+----------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|    B12_1125_Info_DropGB    |    711    |  961   |  30%  |  61%  |  81%  | 11421.98 | 8143.38  | 8377.38  | 10298.86 | 320.52 | 16.064669 | 11.782532 |
| B12_1125_Info_DropUsefulGB |    826    |  1136  |  31%  |  60%  |  80%  | 13826.33 | 9466.84  | 9782.84  | 12358.33 | 320.52 | 16.738898 | 11.843632 |
+----------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1125_Info_DropGB=711%2C1000000&B12_1125_Info_DropUsefulGB=826%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.857254

Nov. 25 US "Dear Wikipedia reader"

• Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112516_Yellowtab_control
• Dear reader: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112516_Yellowtab_dearr

ecom -s 20121125160000 -e 201211252200000 -c C12_iwpenUS_FR -g b -m --having 20

+------------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|            banner            | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+------------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
| B12_112516_Yellowtab_control |    2103   |  2779  |  34%  |  70%  |  82%  | 35499.75 | 21938.91 | 23290.91 | 31099.75 | 500.00 | 16.880528 | 11.075088 |
|  B12_112516_Yellowtab_dearr  |    2422   |  3147  |  34%  |  69%  |  84%  | 40194.49 | 25614.00 | 27248.00 | 36393.00 | 250.00 | 16.595578 | 11.250206 |
+------------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112516_Yellowtab_control=2103%2C1000000&B12_112516_Yellowtab_dearr=2422%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.610919

Nov. 25 US Blue vs Yellow tab

• Yellow tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112519_Yellowtab
• Blue tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1125BlueTab

ecom -s 20121125193300 -e 201211252200000 -c C12_iwpenUS_FR -g b -m --having 300

+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+---------+-----------+-----------+
|        banner        | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |   max   |    avg    |   avg20   |
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+---------+-----------+-----------+
| B12_112519_Yellowtab |    1264   |  1666  |  33%  |  71%  |  81%  | 22203.67 | 13303.67 | 14153.67 | 18953.67 | 1000.00 | 17.566195 | 11.197524 |
|   B12_1125BlueTab    |    1228   |  1616  |  34%  |  69%  |  84%  | 19622.55 | 12854.55 | 13652.55 | 18022.55 |  200.00 | 15.979275 | 11.117712 |
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+---------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112519_Yellowtab=1264%2C1000000&B12_1125BlueTab=1228%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.944970

Nov. 25 US Yellow tab vs Grey button

• Yellow tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112520_Yellowtab2
• Grey button: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112520_NoTab2

ecom -s 20121125204000 -e 201211252200000 -c C12_iwpenUS_FR -g b -m --having 100

+-----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|         banner        | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+-----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|   B12_112520_NoTab2   |    1121   |  1486  |  33%  |  67%  |  82%  | 19748.67 | 11973.67 | 12763.67 | 17288.67 | 250.00 | 17.617012 | 11.385968 |
| B12_112520_Yellowtab2 |    1273   |  1674  |  31%  |  68%  |  83%  | 19771.95 | 13273.50 | 14129.50 | 18181.95 | 250.00 | 15.531775 | 11.099372 |
+-----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112520_NoTab2=1121%2C1000000&B12_112520_Yellowtab2=1273%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.121782

Nov. 26 "If Wikipedia is useful to you" vs "If Wikipedia is useful to you... Please help us forget fundraising and get back to Wikipedia"

US, CA, GB, AU, NZ

• Useful: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112605_Control_US
• Useful + Forget fundraising: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112605_Min_US

ecom -s 20121126073000 -e 20131111000000 -g b --sub B12_112605_ --left 14

+----------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
|     banner     | donations | clicks | ccpct | cccnv | ppcnv |  amount  | amount18 | amount20 | amount50 |  max   |    avg    |   avg20   |
+----------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
| B12_112605_Con |    4121   |  6120  |  28%  |  54%  |  79%  | 61233.86 | 42116.06 | 44099.06 | 55607.44 | 320.52 | 14.858981 | 10.701058 |
| B12_112605_Min |    4879   |  7145  |  29%  |  55%  |  80%  | 73098.62 | 50374.75 | 52748.75 | 66466.80 | 209.14 | 14.982296 | 10.811386 |
+----------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+

d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112605_Con=4121%2C1000000&B12_112605_Min=4879%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true