Fundraising 2012/We Need A Breakthrough
May 2012Edit
June 2012Edit
July 2012Edit
August 2012Edit
August 24, 2012 "If everyone donated $5.." in Facts banner. Hint: BIG FIND!Edit
Pop quiz from Fundraiser 2011: What banner text did we finally find that actually beat the "Jimmy appeal" banner at some point during the fundraiser? Can't remember? I'll tell you!
Answer: "If everyone reading this donated $5, we would only have to fundraise for one day a year. Please donate to keep Wikipedia free. "
We know this line is really important in the appeal too. We've met with donors and listened to them tell us how important that line is in the Jimmy message. So we're way past due to test this line in our new facts banner.
Here's the test setup:
- Control facts banner: http://en.wikipedia.org/wiki/Control?banner=B12_082412_NofaceTopGreyFactsask
- Facts banner with $5 line: http://en.wikipedia.org/wiki/Five?banner=B12_082412_NofaceTopGreyFactsaskFive
AND THE VERY EXCITING RESULTS THAT MAKE MEGAN SMILE:
+--------------------------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +--------------------------------------+-----------+----------+----------+-----------+-----------+ | B12_082412_NofaceTopGreyFactsask | 579 | 11500.00 | 7048.00 | 19.861831 | 12.172712 | | B12_082412_NofaceTopGreyFactsaskFive | 1041 | 14700.00 | 10283.00 | 14.121037 | 9.878002 | +--------------------------------------+-----------+----------+----------+-----------+-----------+
Here's the confidence on the number of donations:
I'm awake! Great test. Need to find a few more improvements like this before we launch in November.
4 September, 2012 Reality Checking Top facts vs Inline JimmyEdit
In June, we found a top facts banner that actually beat our Jimmy banner from last year. Big find! Over the past few months, we've gone through a bunch of variations of that top facts banner so we wanted to run a reality check on how well we're doing with our current best facts banner against where we were last year.
Today we're running:
- Inline Jimmy: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_JW
- Top facts $5: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFive
Here are the half-time results after 30 minutes of testing:
+--------------------------------------+-----------+---------+----------+--------+ | banner | donations | amount | amount18 | max | +--------------------------------------+-----------+---------+----------+--------+ | B12_082412_JW | 180 | 3516.00 | 2110.00 | 100.00 | | B12_082412_NofaceTopGreyFactsaskFive | 304 | 5037.50 | 3150.50 | 300.00 | +--------------------------------------+-----------+---------+----------+--------+
And here are the results from the full hour of testing:
+--------------------------------------+-----------+---------+----------+--------+ | banner | donations | amount | amount18 | max | +--------------------------------------+-----------+---------+----------+--------+ | B12_082412_JW | 351 | 6993.00 | 4082.00 | 250.00 | | B12_082412_NofaceTopGreyFactsaskFive | 566 | 9322.50 | 5678.50 | 300.00 | +--------------------------------------+-----------+---------+----------+--------+
It's very encouraging to see this new banner we've been working on is coming out confidently ahead of where we were last year. And we still have a few months of testing to go before the campaign launches! Meganhernandez (talk) 15:04, 4 September 2012 (UTC)
6 September, 2012 Quick 10 minute Test without Google/Yahoo lineEdit
We're trying out some alternatives to the Google/Yahoo line in the facts banner. ("Google might have close to a million servers. Yahoo has something like 12,000 staff. We have 679 servers and 138 employees.") A brief background: We added this comparison with Google/Yahoo in 2011 and found that it was a huge gain for both donations and total amount raised. We're testing a couple new variations of the same concept.
Today we tested (very quickly- just 10 minutes):
- Google/yahoo Facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFive
- No google/yahoo line Facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFiveNoG
Here are the results (remember, after just 10 minutes - not long enough to be totally confident in these results):
+-----------------------------------------+-----------+---------+----------+ | banner | donations | amount | amount18 | +-----------------------------------------+-----------+---------+----------+ | B12_082412_NofaceTopGreyFactsaskFive | 293 | 4098.00 | 2957.00 | | B12_082412_NofaceTopGreyFactsaskFiveNoG | 218 | 3074.00 | 2073.00 | +-----------------------------------------+-----------+---------+----------+
6 September, 2012 Another quick 15-min Test: Trying to fix the Google/Yahoo line!Edit
Potential good news here. We're trying to get away from the Google/Yahoo line that has caused some unhappiness/concern over the past year. We just ran a quick 15-min test (so, this is a test we'll want to run again to confirm the results), but it looks like we found a new variation of that line that might work just as well. Great potential news! (Numbers below.) We've been testing a bunch of different lines over the last few weeks, but this is the closest we've come so far.
We tested:
- Old text with the Google/Yahoo numbers: http://en.wikipedia.org/wiki/International_Olympic_Committee?banner=B12_082412_NofaceTopGreyFactsaskFive
- New text: http://en.wikipedia.org/wiki/International_Olympic_Committee?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
And the numbers:
+-------------------------------------------+-----------+---------+----------+-----------+----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+---------+----------+-----------+----------+ | B12_082412_NofaceTopGreyFactsaskFive | 407 | 6103.31 | 4012.81 | 14.995848 | 9.859484 | | B12_082412_NofaceTopGreyFactsaskFiveThous | 409 | 5311.00 | 3816.00 | 12.985330 | 9.330073 | +-------------------------------------------+-----------+---------+----------+-----------+----------+
So really exciting result. It was a short test, so we'll re-run it to confirm. More later, running to meet a donor!
7 September, 2012 Repeating the Google line test from yesterdayEdit
Here are the results combined from yesterday and today:
+-------------------------------------------+-----------+----------+----------+-----------+----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+----------+----------+-----------+----------+ | B12_082412_NofaceTopGreyFactsaskFive | 1114 | 15928.31 | 10981.81 | 14.298303 | 9.857998 | | B12_082412_NofaceTopGreyFactsaskFiveThous | 1022 | 13860.00 | 9684.00 | 13.561644 | 9.475538 | +-------------------------------------------+-----------+----------+----------+-----------+----------+
September 12, 2012 Retesting the Google/Yahoo lineEdit
We just reran the test from last week:
- Google millions facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFive
- Google thousands facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
Here are the results from the one-hour test this morning:
+-------------------------------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+----------+----------+-----------+-----------+ | B12_082412_NofaceTopGreyFactsaskFive | 701 | 10754.00 | 7206.00 | 15.340942 | 10.279601 | | B12_082412_NofaceTopGreyFactsaskFiveThous | 636 | 10426.00 | 6239.00 | 16.393082 | 9.809748 | +-------------------------------------------+-----------+----------+----------+-----------+-----------+
Confidence testing on number of donations:
And here are the combined results from last week and today:
+-------------------------------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+----------+----------+-----------+-----------+ | B12_082412_NofaceTopGreyFactsaskFive | 1857 | 27607.31 | 18626.81 | 14.866618 | 10.030592 | | B12_082412_NofaceTopGreyFactsaskFiveThous | 1695 | 24939.00 | 16302.00 | 14.713274 | 9.617699 | +-------------------------------------------+-----------+----------+----------+-----------+-----------+
Confidence on number of donations from the combined results:
Sept. 14, 2012 Expander BannerEdit
Wait! After this test ran (and I posted the update), we saw there was a mistake in the ask amounts on the new expanding/dropdown form. We somehow accidentally swapped out the $20 option and the ask amounts were different on the two forms. Skip down to the Sept. 17 update to see the results from this test once we fixed the ask button. Meganhernandez (talk) 18:17, 18 September 2012 (UTC)
We're testing a new type of banner today. The banner stays in place when you click on it and the donation form drops down from the banner instead of taking you to a new page.
Take a look:
- Control facts banner: http://en.wikipedia.org/wiki/2012?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
- New expanding/dropdown banner: http://en.wikipedia.org/wiki/September_14?banner=B12_Expandable_Facts_PlainButtons_Image
The results:
+-------------------------------------------+-----------+----------+----------+-----------+----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+----------+----------+-----------+----------+ | B12_082412_NofaceTopGreyFactsaskFiveThous | 941 | 13245.11 | 8987.11 | 14.075569 | 9.550595 | | B12_Expandable_Facts_PlainButtons_Image | 1054 | 13821.59 | 9751.59 | 13.113463 | 9.251983 | +-------------------------------------------+-----------+----------+----------+-----------+----------+
Confidence:
September 17, 2012 Expanding/dropdown banner testEdit
We're re-testing the dropdown banner.
The banners:
- Control facts banner: http://en.wikipedia.org/wiki/Fact?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
- Dropdown facts banner: http://en.wikipedia.org/wiki/Dropkick?banner=B12_Expandable_Facts_PlainButtons_Image_Amts
The results:
+----------------------------------------------+-----------+----------+----------+-----------+----------+ | banner | donations | amount | amount18 | avg | avg18 | +----------------------------------------------+-----------+----------+----------+-----------+----------+ | B12_082412_NofaceTopGreyFactsaskFiveThous | 673 | 10465.25 | 6392.65 | 15.550149 | 9.498737 | | B12_Expandable_Facts_PlainButtons_Image_Amts | 814 | 10438.77 | 7647.77 | 12.824042 | 9.395295 | +----------------------------------------------+-----------+----------+----------+-----------+----------+
September 20, 2012 Less wordy facts & Amazon testEdit
We ran a multivariate test in the US today. I'll report on the landing page test first, since the results from that test were more inspirational than the banner test results.
Amazon landing page test
We tested a form offering Amazon Payments as a new payment method.
Our control form allows users to donate by:
- Credit/Debit card
- PayPal
The new form included:
- Credit/Debit card
- PayPal
- Amazon
Here are the pages:
- Control credit/PayPal: https://donate.wikimedia.org/wiki/L12_0628_norec_JW/en
- New form with Amazon: https://donate.wikimedia.org/wiki/L12_0917_amazon_JW/en
Results:
Results by landing page:
Results broken out by payment method:
+--------------------+----------------+-----------+---------+----------+-----------+-----------+ | landing_page | payment_method | donations | amount | amount18 | avg | avg18 | +--------------------+----------------+-----------+---------+----------+-----------+-----------+ | L12_0628_norec_JW | cc | 266 | 4825.03 | 2930.47 | 18.139211 | 11.016805 | | L12_0628_norec_JW | pp | 295 | 3430.99 | 2669.99 | 11.630475 | 9.050814 | | L12_0917_amazon_JW | amazon | 129 | 1552.00 | 1196.00 | 12.031008 | 9.271318 | | L12_0917_amazon_JW | cc | 231 | 4069.10 | 2518.10 | 17.615152 | 10.900866 | | L12_0917_amazon_JW | pp | 261 | 2836.00 | 2256.00 | 10.865900 | 8.643678 | +--------------------+----------------+-----------+---------+----------+-----------+-----------+
So, it looks like offering Amazon may be increasing the number of donations coming in, and could possibly also affect the amount raised (look at amt18). We'll run some confidence tests on those results and post back. We will probably want to re-run this test for longer at some point since the results look pretty close. It's really encouraging if we're making it easier for more readers to donate.
Banner Test
We tested our control facts banner against a new banner with fewer words and with vertical lines separating each fact.
- Control facts banner: http://en.wikipedia.org/wiki/Words?banner=B12_082412_NofaceTopGreyFactsaskFiveThous
- New banner with fewer words & vertical lines: http://en.wikipedia.org/wiki/Vertical_direction?banner=B12_091912_Facts_LessText_LinesBold
And the banner results which really bum Megan out:
+-------------------------------------------+-----------+---------+----------+-----------+----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+---------+----------+-----------+----------+ | B12_082412_NofaceTopGreyFactsaskFiveThous | 699 | 9688.24 | 6788.24 | 13.860143 | 9.711359 | | B12_091912_Facts_LessText_LinesBold | 483 | 7024.88 | 4782.32 | 14.544265 | 9.901284 | +-------------------------------------------+-----------+---------+----------+-----------+----------+
I thought we could make the banner easier for people to read if we made it less wordy, made the font size bigger and added some lines separating each fact. Clearly, I was wrong. We still have a ton of variations of that facts banner to test though. Improvements on the way!
Meganhernandez (talk) 20:34, 20 September 2012 (UTC)
September 24, 2012 Retesting a 2011 campaignEdit
Results from Sept. 24, 2012:
+------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +------------------+-----------+----------+----------+-----------+-----------+ | B12_0924_btPf_US | 511 | 11548.00 | 6718.00 | 22.598826 | 13.146771 | +------------------+-----------+----------+----------+-----------+-----------+
Results from Nov. 15, 2011:
+------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +------------------+-----------+----------+----------+-----------+-----------+ | B11_1115_btPf_US | 543 | 15105.00 | 8058.00 | 27.817680 | 14.839779 | +------------------+-----------+----------+----------+-----------+-----------+
September 26, 2012 More Dropdown banner testingEdit
Last week (Sept. 17), we tested a new dropdown/expander banner and it looked like we saw a big increase from the standard facts banner. We showed this new banner around to some donors and got some great feedback that it wasn't necessarily clear to everyone that they were supposed to click on the banner.
So today we tested a couple variations of dropdown banners (with arrows & butons) and tweaked the messaging a bit.
The banners:
- Arrow & control text : http://en.wikipedia.org/wiki/Arrow?banner=B12_ExfactsTabarrowSeptext
- Button & control text: http://en.wikipedia.org/wiki/Button?banner=B12_ExfactsTabbuttonSeptext
- Arrow & new "long way" text: http://en.wikipedia.org/wiki/L%C3%B3ng?banner=B12_ExfactsTabarrowLongway
- Button & new "long way" text: http://en.wikipedia.org/wiki/The_Way_(film)?banner=B12_ExfactsTabbuttonLongway
The results:
+-----------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-----------------------------+-----------+---------+----------+-----------+-----------+ | B12_ExfactsTabarrowLongway | 376 | 5998.00 | 3927.00 | 15.952128 | 10.444149 | | B12_ExfactsTabarrowSeptext | 454 | 6719.00 | 4422.00 | 14.799559 | 9.740088 | | B12_ExfactsTabbuttonLongway | 329 | 4879.37 | 3358.37 | 14.830912 | 10.207812 | | B12_ExfactsTabbuttonSeptext | 418 | 6018.00 | 4075.00 | 14.397129 | 9.748804 | +-----------------------------+-----------+---------+----------+-----------+-----------+
Arrow vs. Button:
It looks like Arrow is a little ahead of Button, but not with a ton of confidence .
Control vs. new text:
We have more confidence that our control text comes brings in more donations than the new "donations go a long way" text.
Meganhernandez (talk) 16:48, 27 September 2012 (UTC)
October 1 "Forget fundraising & get back to Wikipedia" and AmazonEdit
Zack came up with a new fact for the banner and it looks like it's helping big time!
- Control facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1001_Facts
- New "back to Wikipedia" facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1001_FactsBackto
Results
+----------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +----------------------+-----------+----------+----------+-----------+-----------+ | B12_1001_Facts | 848 | 12163.85 | 8069.85 | 14.344163 | 9.516333 | | B12_1001_FactsBackto | 1123 | 15620.59 | 10474.65 | 13.909697 | 9.327382 | | B12_1001_Half1Cols | 853 | 11620.75 | 8701.75 | 13.623388 | 10.201348 | +----------------------+-----------+----------+----------+-----------+-----------+
The last fact on the "back to Wikipedia" banner helps a lot. We'll test more to see if the "one hour" and "back to Wikipedia" texts are both helping.
The "half of 1%" is also doing about as well as our control facts banner. Which is great since it's missing our really strong fact comparing Wikipedia to Google and Yahoo. There must be parts of this banner that are helping and hurting donations. We'll test more to isolate the parts that are helping.
Amazon retestEdit
We also re-tested offering Amazon as a payment option (first test was Sept. 20).
- Credit card & Paypal donation page: https://donate.wikimedia.org/wiki/L12_0628_norec_JW/en
- Credit card, Paypal & Amazon LP: https://donate.wikimedia.org/wiki/L12_0917_amazon_JW/en
Results by landing page
Results by payment method
+--------------------+----------------+-----------+----------+----------+-----------+-----------+ | L12_0628_norec_JW | cc | 619 | 9945.17 | 6480.23 | 16.066511 | 10.468869 | | L12_0628_norec_JW | pp | 723 | 10206.15 | 6695.15 | 14.116390 | 9.260235 | | L12_0917_amazon_JW | amazon | 327 | 4327.15 | 2953.15 | 13.232875 | 9.031040 | | L12_0917_amazon_JW | cc | 489 | 7248.78 | 5096.78 | 14.823681 | 10.422863 | | L12_0917_amazon_JW | pp | 666 | 7677.94 | 6020.94 | 11.528438 | 9.040450 | +--------------------+----------------+-----------+----------+----------+-----------+-----------+
Our two Amazon tests (Sept. 20 & Oct. 1) show the Amazon page coming slightly ahead. We'll run the confidence test on the amount raised later, but both tests look like offering Amazon along with credit card and Paypal increases the amount we raise. (Plus hopefully makes some people happier by having their preferred payment method available :)
Meganhernandez (talk) 19:09, 2 October 2012 (UTC)
ITALIAN TEST SEPTEMBER 30 - OCTOBER 7, 2012Edit
Ask amount Test €5 vs €10Edit
+-------------------------------------------+-----------+----------+----------+-----------+-----------+ | landing_page | donations | amount | amount18 | avg | avg18 | +-------------------------------------------+-----------+----------+----------+-----------+-----------+ | default~default~default~default~control | 1269 | 19764.79 | 13191.77 | 15.575091 | 10.395406 | | default~default~default~default~variable1 | 1026 | 17323.41 | 12624.41 | 16.884415 | 12.304493 | +-------------------------------------------+-----------+----------+----------+-----------+-----------+
-s 20120930174200 -e 20121002132400 -c C12_iwpitWW_FR_IT -l -t
Jimmy photo-at-thonEdit
Banners:
- http://it.wikipedia.org/wiki/The_Jacket?banner=B12_InPaJwJkRn_IT
- http://it.wikipedia.org/wiki/Face?banner=B12_InPaJwPfRn_IT
- http://it.wikipedia.org/wiki/Space_Jam?banner=B12_InPaJwKsRn_IT
- http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_InPaJwJoRn_IT
+----------------------+-----------+---------+----------+------------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +----------------------+-----------+---------+----------+------------+-----------+ | B12_InPaJwJkRn_IT_IT | 316 | 4808.21 | 3580.76 | 15.215854 | 11.331519 | | B12_InPaJwJoRn_IT_IT | 315 | 4544.40 | 3503.32 | 14.426667 | 11.121651 | | B12_InPaJwKsRn_IT_IT | 294 | 4382.08 | 3344.86 | 14.905034 | 11.377075 | | B12_InPaJwPfRn_IT_IT | 346 | 5508.44 | 3967.21 | 15.920347 | 11.465925 | +----------------------+-----------+---------+----------+------------+-----------+
ecom -s 20120930174200 -e 20121001095700 -c C12_iwpitWW_FR -b -t
Green background Jimmy vs White background JimmyEdit
- Green: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_InPaJwG2Rn_IT
- White: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_InPaJwJoRn_IT
+----------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +----------------------+-----------+----------+----------+-----------+-----------+ | B12_InPaJwG2Rn_IT_IT | 1139 | 15783.10 | 11688.99 | 13.856980 | 10.262502 | | B12_InPaJwJoRn_IT_IT | 1013 | 14156.89 | 10358.20 | 13.975212 | 10.225271 | +----------------------+-----------+----------+----------+-----------+-----------+
-s 20121002132400 -e 20121003191100 -c C12_iwpitWW_FR_IT -b -t
Jimmy vs dropdownEdit
- Jimmy banner: http://it.wikipedia.org/wiki/Comune_italiano?banner=B12_InPaJwG3Rn_IT
- Dropdown banner: http://it.wikipedia.org/wiki/Palazzi_di_Cesena?banner=B12_ExpandFactsImage_IT
+-------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------+-----------+---------+----------+-----------+-----------+ | B12_ExpandFactsImage_IT | 755 | 7482.60 | 6447.42 | 9.910728 | 8.539629 | | B12_InPaJwG3Rn_IT_IT | 702 | 8955.38 | 6943.37 | 12.756952 | 9.890840 | +-------------------------+-----------+---------+----------+-----------+-----------+
ecom -s 20121003191100 -e 20121004113100 -c C12_iwpitWW_FR -b -t
Italy Grey dropdown facts vs blue modified factsEdit
- grey facts:http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandFactsImage_IT
- blue (fact 3: focus on wikipedia , 20 average amount highlighted, $5/1 day on right side of form): http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandHm
+-------------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------+-----------+----------+----------+-----------+-----------+ | B12_ExpandFactsImage_IT | 954 | 10286.56 | 8348.97 | 10.782558 | 8.751541 | | B12_ExpandHm | 1452 | 16851.80 | 13411.37 | 11.605923 | 9.236481 | +-------------------------+-----------+----------+----------+-----------+-----------+
ecom -s 20121004113100 -e 20121005120000 -c C12_iwpitWW_FR -b -t
Italy blue modified facts vs New fact 3Edit
- Blue modified: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandHm2
- New 3rd fact translation: http://it.wikipedia.org/wiki/Pagina_principale?banner=B12_ExpandHmTrans
+-------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------+-----------+---------+----------+-----------+-----------+ | B12_ExpandHm2 | 552 | 6173.19 | 5110.62 | 11.183315 | 9.258370 | | B12_ExpandHmTrans | 491 | 5724.97 | 4704.82 | 11.659817 | 9.582118 | +-------------------------+-----------+---------+----------+-----------+-----------+
ecom -s 20121005134500 -e 2012101500000000 -c C12_iwpitWW_FR -b -t
Reports left to runEdit
- Second step address test
US Long & Short appeals from Dropdown - Friday October 5, 2012Edit
Our "facts banner" provides much more info directly from the banner than last year's "Please read: a personal appeal..." banner. Now that our readers are already learning the reasons to donate directly from the banner, we experimented today with showing a much shorter appeal than last year's appeal once you click on the banner. We're trying to limit repetition and make it quick and easy for people to donate.
- Long appeal: http://en.wikipedia.org/wiki/Main_Page?banner=B12_DropdownL
- Short appeal: http://en.wikipedia.org/wiki/Main_Page?banner=B12_DropdownS
+---------------+-----------+----------+----------+-----------+----------+ | banner | donations | amount | amount18 | avg | avg18 | +---------------+-----------+----------+----------+-----------+----------+ | B12_DropdownL | 1374 | 19980.58 | 13213.33 | 14.541907 | 9.616689 | | B12_DropdownS | 1478 | 20837.64 | 14281.64 | 14.098539 | 9.662815 | +---------------+-----------+----------+----------+-----------+----------+
ecom -s 20121005131000 -e 20121005171000 -c C12_1005_LvS -t
Monday October 8, 2012 EN T1 Jimmy vs Blue FactsEdit
Hour 1:
- jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal
- Bluetop facts: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts
Hour 2:
+------+-----------------------------+-----------+---------+----------+-----------+-----------+ | lpi | banner | donations | amount | amount18 | avg | avg18 | +------+-----------------------------+-----------+---------+----------+-----------+-----------+ | 4250 | B12_1008_Jimmyinlineappeal2 | 108 | 1997.65 | 1281.56 | 18.496759 | 11.866296 | | 3955 | B12_1008_Jimmyinlineappeal | 106 | 2884.08 | 1321.77 | 27.208302 | 12.469528 | | 3209 | B12_1008_Backtobluefacts | 375 | 5631.31 | 3684.99 | 15.016827 | 9.826640 | | 3091 | B12_1008_Backtobluefacts2 | 320 | 4198.77 | 2939.87 | 13.121156 | 9.187094 | +------+-----------------------------+-----------+---------+----------+-----------+-----------+
EN T2 (including GB) Jimmy vs. Blue Facts Tuesday October 9, 2012Edit
First 38 minutes when GB was included in EN T2:
- Jimmy banner; http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Gd
- Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Bd
- Blue banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts3
+-------------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------+-----------+---------+----------+-----------+-----------+ | B12_1008_Backtobluefacts3 | 427 | 5526.81 | 4613.39 | 12.943349 | 10.804192 | | B12_1008_Jimmyinlineappeal_Bd | 51 | 923.62 | 681.31 | 18.110196 | 13.359020 | | B12_1008_Jimmyinlineappeal_Gd | 73 | 1454.45 | 928.01 | 19.923973 | 12.712466 | +-------------------------------+-----------+---------+----------+-----------+-----------+
ecom -s 20121009081500 -e 20121009084800 -c C12_iwpenT2EurAf_FR -b --sql -m | db -t
GB-only Jimmy vs Blue facts banner Tuesday October 9, 2012Edit
- Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Bd_GB
+----------------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +----------------------------------+-----------+---------+----------+-----------+-----------+ | B12_1008_Backtobluefacts3_GB | 474 | 8285.70 | 6350.45 | 17.480380 | 13.397574 | | B12_1008_Jimmyinlineappeal_Bd_GB | 145 | 2719.86 | 2064.25 | 18.757655 | 14.236207 | | B12_1008_Jimmyinlineappeal_Gd_GB | 142 | 3018.45 | 2095.70 | 21.256690 | 14.758451 | +----------------------------------+-----------+---------+----------+-----------+-----------+
ecom -s 20121009090100 -e 20121009110900 -c C12_iwpenGB_FR -b --sql -m | db -t
EN T2 (excluding GB) Jimmy vs Blue facts Tuesday October 9, 2012Edit
note: removed GB from this T2 EN campaign and restarted this EN test from earlier in the morning.
Jimmy banner; http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Gd
Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Jimmyinlineappeal_Bd
blue banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1008_Backtobluefacts3
- NOTE: Blue banner was running at 50%, each Jimmy banner was running at 25% **
+-------------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-------------------------------+-----------+---------+----------+-----------+-----------+ | B12_1008_Backtobluefacts3 | 682 | 9888.51 | 7075.81 | 14.499282 | 10.375088 | | B12_1008_Jimmyinlineappeal_Bd | 103 | 2175.41 | 1214.84 | 21.120485 | 11.794563 | | B12_1008_Jimmyinlineappeal_Gd | 117 | 1860.64 | 1368.90 | 15.902906 | 11.700000 | +-------------------------------+-----------+---------+----------+-----------+-----------+
ecom -s 20121009091200 -e 20121009102600 -c C12_iwpenT2EurAf_FR -b --sql -m | db -t
Japanese Jimmy Test Tuesday October 9, 2012Edit
Jimmy banner: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1009_JA2HR
+----------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +----------------+-----------+---------+----------+-----------+-----------+ | B12_1009_JA2HR | 152 | 4373.23 | 2221.48 | 28.771250 | 14.615000 | +----------------+-----------+---------+----------+-----------+-----------+
ecom -s 20121009071500 -e 20121009120000 -c C12_iwpjaWW_FR -b --sql -m | db -t+
2011 / 2012 Comparison
+----------+---------+----------+----------+---------+----------+ | 11_count | 11_amt | 11_amt18 | 12_count | 12_amt | 12_amt18 | +----------+---------+----------+----------+---------+----------+ | 195 | 6386.34 | 2882.21 | 139 | 4036.25 | 2031.64 | +----------+---------+----------+----------+---------+----------+
October 09, 2012 Tuesday US Dropdown tab test ("Please help" vs "Personal appeal") & highlighting the $20 averageEdit
We ran a multivariate test in the US today. We're testing a new tab for our dropdown banner with the text "Please read a personal appeal from Wikipedia's founder" against our control "Please help" tab.
We also tested two versions of the dropdown landing page: one with the $20 average amount highlighted, and one without $20 highlighted.
The banners:
- Personal appeal tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_PersonalAppeal_Tab
- Personal appeal tab / average donation: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_PersonalAppeal_TabAve
- Please help tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1009_Plaintab2
- Please help tab // average donation: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1009_PlaintabAvg2
Results
+---------------------------+-----------+----------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +---------------------------+-----------+----------+----------+-----------+-----------+ | B12_1009_Plaintab2 | 874 | 10326.16 | 8034.16 | 11.814828 | 9.192403 | | B12_1009_PlaintabAvg2 | 910 | 13626.00 | 9458.00 | 14.973626 | 10.393407 | | B12_PersonalAppeal_Tab | 856 | 10719.80 | 7923.91 | 12.523131 | 9.256904 | | B12_PersonalAppeal_TabAve | 790 | 10334.86 | 7951.43 | 13.082101 | 10.065101 | +---------------------------+-----------+----------+----------+-----------+-----------+
ecom -c C12_US_1009_Avg2 -s 20121009142300 -e 201210092000 --sql | db -t
Combined results
Commentary: SO! What a great day (see also below). Above you see one interesting discovery and one exciting discovery. First, we have two great banners that work well for us: "Please read a personal appeal from Wikipedia founder" and a long "Facts" banner that makes the big points from our traditional appeal right in-your-face in the banner. Both work great. So wouldn't they work even better if we combined them in the same banner? We combined them today. But the combo didn't do well. It didn't do horribly, but significant drop vs the plain facts. (These Facts are out-performing the appeal banner in the US btw).
Second, we have been looking for a way to bring up the average donation. Last year we basically doubled our number of donations while not-quite halving our average. We're talking people to make a lower donation than they might actually favor doing with the $5 line. So we tried indicating that our average donation is about $20 on the landing page. It worked! The normalized average went up about $1.
EN T1 $5 vs "price of a sandwich" Tuesday October 9. 2012Edit
We are really excited about this test for two reasons:
1) The $5 line that we've been using is great for bringing in a ton of donations -- like tons of $5 donations! But that brings the average donation down too far. We're talking a lot of people who would probably like to give $20 into giving $5. By saying, "price of a sandwich" instead of $5, it makes it a sliding scale: how much does your typical sandwich cost? For some people $5, for others $10, and it wasn't that long ago that I had a sandwich for less than a dollar while traveling in a low-income country. And lo and behold, sandwich brought up the average by a whole dollar (looking at avg18) and two looking at plain avg (which can be thrown off by one large donation.)
2) The $5 line has another problem: We have to convert it into different currencies and amounts in different countries. In fact, testing late last night (for me) and early in the morning (for Megan) we forgot about this and subjected millions in Africa and Europe to "$5".
Next to try: "A couple cups of coffee", "A movie ticket", what else?
Banners:
+------------------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +------------------------------+-----------+---------+----------+-----------+-----------+ | B12_1008_Backtobluefacts2 | 509 | 7459.98 | 4961.57 | 14.656149 | 9.747682 | | B12_1008_BacktobluefactsSand | 506 | 8360.60 | 5441.11 | 16.522925 | 10.753182 | +------------------------------+-----------+---------+----------+-----------+-----------+>
ecom -c C12_iwpenT1_FR -s 20121009150000 -e 20121009200000 --sql -m | db -t
October 11, 2012 Thursday US Sandwich & Average TestEdit
We're working to increase our average donation in our testing this week. Over the last couple of testing months, we found a line that brings in a lot of donations, but lowers our average donation ("If everyone reading this donated $5, the fundraiser would be done in an hour.") We saw some promising results from the "price of a sandwich" test (update above) that ran outside of the US, so we wanted to confirm those results with a US test.
We also re-tested highlighting the $20 average on our donation page.
The banners:
- $5 / no average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_FiveNoavg
- $5 / average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_FiveAvg
- Sandwich / no average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_SandNoavg
- Sandwhich / average: http://en.wikipedia.org/wiki/Sandwich?banner=B12_1010_SandNoavg
The Results
+--------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +--------------------+-----------+---------+----------+-----------+-----------+ | B12_1010_FiveAvg | 630 | 9558.00 | 6648.00 | 15.171429 | 10.552381 | | B12_1010_FiveNoavg | 624 | 8010.00 | 5783.00 | 12.836538 | 9.267628 | | B12_1010_SandAvg | 550 | 9841.73 | 6507.73 | 17.894055 | 11.832236 | | B12_1010_SandNoavg | 633 | 9310.00 | 6272.00 | 14.707741 | 9.908373 | +--------------------+-----------+---------+----------+-----------+-----------+
Combined Results
October 12, Friday US No Google line & Average TestEdit
Today we tested a banner without the Google/Yahoo comparison. We also tested the new "average donation" page against our control landing page. (When we tested "average donation" page earlier this week, we eliminated the $250 radio button option.)
Banners
- No Google // no average: http://en.wikipedia.org/wiki/Extraterrestrial_life?banner=B12_1011_FactsNoGNAve
- No Google // average: http://en.wikipedia.org/wiki/Extraterrestrial_life?banner=B12_1011_FactsNoGAve
- Google // no average: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1012_FactsGNAve
- Google // average: http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B12_1012_FactsGAve
Results
+-----------------------+-----------+---------+----------+-----------+-----------+ | banner | donations | amount | amount18 | avg | avg18 | +-----------------------+-----------+---------+----------+-----------+-----------+ | B12_1011_FactsNoGAve | 332 | 5259.50 | 3737.50 | 15.841867 | 11.257530 | | B12_1011_FactsNoGNAve | 397 | 6530.00 | 4051.00 | 16.448363 | 10.204030 | | B12_1012_FactsGAve | 469 | 8258.61 | 5504.61 | 17.608977 | 11.736908 | | B12_1012_FactsGNAve | 468 | 6738.00 | 4731.00 | 14.397436 | 10.108974 | +-----------------------+-----------+---------+----------+-----------+-----------+
Combined Results
October 16, 2012 Tuesday US "No government funds"Edit
We found a new line for our facts banner that actually does a bit better than the Google/Yahoo comparison that has performed very well over the past year.
The banners
- Control Google/Yahoo banner: http://en.wikipedia.org/wiki/Google?banner=B12_1016_FactsGoog
- No government funds banner: http://en.wikipedia.org/wiki/Government?banner=B12_1016_FactsGovmore
The results
+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+ | banner | donations | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+ | B12_1016_FactsGoog | 1520 | 79% | 62% | 79% | 19336.82 | 14235.19 | 12.721592 | 9.365257 | | B12_1016_FactsGovmore | 1680 | 84% | 61% | 77% | 23991.12 | 17488.00 | 14.280429 | 10.409524 | +-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+
ecom -c C12_US_1016_GgvGv -s 20121016133800 -e 20121016173800 http://www.thumbtack.com/labs/abba/#B12_1016_FactsGoog=1520%2C1000000&B12_1016_FactsGovmore=1680%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
Great news! Not only could this allow us to retire the Google line in the banner, but it also looks like this new text even performs a little better.
We're brainstorming some new versions to try to get our average back up. (Since we've been using the $5 line in the banner, we've increased the number of donations but decreased the average amount.)
More testing to come later this week.
October 19, 2012 US First Editor Test!Edit
We've tested our first editor appeal of 2012. Last year, the messages from editors gave the campaign personality and introduced our volunteer community to our readers. It was the fun part of the fundraiser and the comments we got from readers were really inspiring.
Our first editor this year is a retired math teacher from India. Poongothai writes articles on quadratic functions, probability, charts, and graphs.
We tested Poongothai's appeal coming from our facts dropdown banner. Next step is to test a new version of the facts banner that includes her photo.
The banners
- Control facts banner: http://en.wikipedia.org/wiki/Fact?banner=B12_1018_FactsGovmore
- Editor banner: http://en.wikipedia.org/wiki/Editor?banner=B12_1018_FactsEditor
The Results
+-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+ | banner | donations | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+ | B12_1018_FactsEditor | 768 | 42% | 62% | 79% | 12558.00 | 8228.00 | 16.351563 | 10.713542 | | B12_1018_FactsGovmore | 784 | 45% | 66% | 78% | 12164.08 | 8160.08 | 15.515408 | 10.408265 | +-----------------------+-----------+-------+-------+-------+----------+----------+-----------+-----------+
ecom -c C12_US_1015_Editor -s 20121019130500 -e 20121019160500
Those results are really encouraging! In previous years, our editor banners always received great feedback from our readers, but they always brought in fewer donations than our standard founder appeal. In our test this morning, the Jimmy appeal and the editor appeal performed just about the same. Both appeals had the exact same banner, so it looks like that facts banner is really doing the work of compelling readers to donate, and that the message we put on the appeal page has a lesser impact than it did when we used the "personal appeal" banners last year. Great first editor test!
Next week, we'll test a new editor banner that incorporates the facts along with the photo and an invitation to read the editor message. Can't wait.
October 22, 2012 US Comparing where we're at to 2011Edit
We duplicated a campaign setup from 2011 to see how our banner impressions, click rate, and donation rates are comparing to last year when we hold the creative content constant.
Here's the setup from 2011:
3 jimmy banners:
- http://en.wikipedia.org/wiki/Main_Page?banner=B11_1114_JKwfCf_US
- http://en.wikipedia.org/wiki/Main_Page?banner=B11_1114_JKwfLf_US
- http://en.wikipedia.org/wiki/Main_Page?banner=B11_Foundation_Jimmy_KevinSpaceywhite_US
1 landing page: http://wikimediafoundation.org/wiki/L11_1112_JBT/en/US
The setup from 2012:
3 Jimmy banners:
- http://en.wikipedia.org/wiki/Main_Page?banner=B12_1022_JKwfCf_US
- http://en.wikipedia.org/wiki/Main_Page?banner=B12_1022_JKwfLf_US
- http://en.wikipedia.org/wiki/Main_Page?banner=B12_Foundation_Jimmy_KevinSpaceywhite_US
1 landing page: https://donate.wikimedia.org/wiki/L11_1112_JBT/en/US
2011 US Donation Results
+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | B11_1114_JKwfCf_US | 497 | 722 | 41% | 59% | 78% | 11600.00 | 7449.00 | 23.340040 | 14.987928 | | B11_1114_JKwfLf_US | 513 | 772 | 44% | 55% | 79% | 13978.00 | 7655.00 | 27.247563 | 14.922027 | | B11_Foundation_Jimmy_KevinSpaceywhite_US | 536 | 834 | 49% | 58% | 71% | 12618.00 | 7965.00 | 23.541045 | 14.860075 | +------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
ecom -c C11_1114_CFLFKS_US -s 20111114182500 -e 20111114192700 -g b
2012 US Donation Results
+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | B12_1022_JKwfCf_US | 323 | 533 | 45% | 54% | 68% | 8594.30 | 4922.30 | 26.607740 | 15.239319 | | B12_1022_JKwfLf_US | 344 | 541 | 52% | 59% | 69% | 9002.54 | 5076.54 | 26.170174 | 14.757384 | | B12_Foundation_Jimmy_KevinSpaceywhite_US | 314 | 522 | 51% | 55% | 66% | 10153.35 | 4807.00 | 32.335510 | 15.308917 | +------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
ecom -c C12_US_1022_RepTst -s 20121022172500 -e 20121022182700 -g b
2011 Banner Impressions
| utm_source | sum(counts) | +------------------------------------------+-------------+ | B11_1114_JKwfCf_US | 1981114 | | B11_1114_JKwfLf_US | 1982674 | | B11_Foundation_Jimmy_KevinSpaceywhite_US | 1979817 |
2012 Banner Impressions
+------------------------------------------+------------+ | banner | sum(count) | +------------------------------------------+------------+ | B12_1022_JKwfCf_US | 1725400 | | B12_1022_JKwfLf_US | 1694200 | | B12_Foundation_Jimmy_KevinSpaceywhite_US | 1717600 | +------------------------------------------+------------+
2011 Landing page impressions
+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | 31199 | B11_Foundation_Jimmy_KevinSpaceywhite_US | 536 | 834 | 0% | 0% | 71% | 12618.00 | 7965.00 | 23.541045 | 14.860075 | | 30669 | B11_1114_JKwfLf_US | 513 | 772 | 0% | 0% | 79% | 13978.00 | 7655.00 | 27.247563 | 14.922027 | | 30279 | B11_1114_JKwfCf_US | 497 | 722 | 0% | 0% | 78% | 11600.00 | 7449.00 | 23.340040 | 14.987928 | +-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
ecom -c C11_1114_CFLFKS_US -s 20111114182500 -e 20111114192700 -g b -i
2012 Landing page impressions
+-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | 20642 | B12_Foundation_Jimmy_KevinSpaceywhite_US | 314 | 522 | 51% | 55% | 66% | 10153.35 | 4807.00 | 32.335510 | 15.308917 | | 20355 | B12_1022_JKwfLf_US | 344 | 541 | 52% | 59% | 69% | 9002.54 | 5076.54 | 26.170174 | 14.757384 | | 20077 | B12_1022_JKwfCf_US | 323 | 533 | 45% | 54% | 68% | 8594.30 | 4922.30 | 26.607740 | 15.239319 | +-------+------------------------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
ecom -c C12_US_1022_RepTst -s 20121022172500 -e 20121022182700 -g b -i
October 22, 2012 EN T1 Comparison from Oct. 8, 2012 TestEdit
Test setup from Oct. 8, 2012 and Oct. 22, 2012:
- Jimmy banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Jimmyinlineappeal2
- Blue facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Backtobluefacts2
Facts landing page (CA running from Jimmy banner): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=CA&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-facts&infobox=Appeal-default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1008_Jimmyinlineappeal2_CA&utm_campaign=C12_iwpenT1_FR_CA
October 8, 2012 Results:
+------+---------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+---------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 4742 | B12_1008_Jimmyinlineappeal2 | 104 | 196 | 44% | 45% | 30% | 1956.78 | 1248.23 | 18.815192 | 12.002212 | | 3537 | B12_1008_Backtobluefacts2 | 309 | 568 | 34% | 38% | 69% | 4046.52 | 2855.49 | 13.095534 | 9.241068 | +------+---------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -c C12_iwpenT1_FR -s 20121008180100 -e 20121008190000 -g b -m -i
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=104%2C1000000&B12_1008_Backtobluefacts2=309%2C1000000&B12_1008_Backtobluefacts=9%2C1000000&B12_1008_Jimmyinlineappeal=3%2C1000000&B12_1008_Backtobluefacts_undefi=0%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true d/lpi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=104%2C4742&B12_1008_Backtobluefacts2=309%2C3537&B12_1008_Backtobluefacts=9%2C167&B12_1008_Jimmyinlineappeal=3%2C81&B12_1008_Backtobluefacts_undefi=0%2C2&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true lpi/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=4742%2C1000000&B12_1008_Backtobluefacts2=3537%2C1000000&B12_1008_Backtobluefacts=167%2C1000000&B12_1008_Jimmyinlineappeal=81%2C1000000&B12_1008_Backtobluefacts_undefi=2%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | landingpage | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 6100 | default~default~default~default~control | 377 | 706 | 35% | 39% | 58% | 5242.07 | 3615.28 | 13.904695 | 9.589602 | | 2429 | default~default~facts~default~control | 48 | 88 | 46% | 45% | 30% | 880.13 | 597.37 | 18.336042 | 12.445208 | +------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -c C12_iwpenT1_FR -s 20121008180100 -e 20121008190000 -g l -m -i
Oct. 22 Results:
+------+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 5722 | B12_1008_Jimmyinlineappeal2 | 103 | 228 | 48% | 38% | 24% | 2086.04 | 1133.73 | 20.252816 | 11.007087 | | 3555 | B12_1008_Backtobluefacts2 | 308 | 539 | 37% | 43% | 70% | 4296.99 | 3008.93 | 13.951266 | 9.769253 | +------+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -c C12_iwpenT1_FR -s 20121022180100 -e 20121022190000 -g b -m -i
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=103%2C1000000&B12_1008_Backtobluefacts2=308%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true d/lpi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=103%2C5722&B12_1008_Backtobluefacts2=308%2C3555&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true lpi/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1008_Jimmyinlineappeal2=5722%2C1000000&B12_1008_Backtobluefacts2=3555%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
+------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | landingpage | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 6430 | default~default~default~default~control | 357 | 645 | 39% | 43% | 57% | 5243.58 | 3531.56 | 14.687899 | 9.892325 | | 2847 | default~default~facts~default~control | 54 | 122 | 44% | 36% | 25% | 1139.45 | 611.10 | 21.100926 | 11.316667 | +------+-----------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -c C12_iwpenT1_FR -s 20121022180100 -e 20121022190000 -g l -m -i
Oct. 22 Banner Impressions:
+-----------------------------+------------+ | banner | sum(count) | +-----------------------------+------------+ | B12_1008_Backtobluefacts2 | 459200 | | B12_1008_Jimmyinlineappeal2 | 474600 | +-----------------------------+------------+
(still need Oct. 8 banner impressions)
Oct. 23, 2012 GB Compare testEdit
Setup from Oct. 9 and Oct. 23:
- Jimmy banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Jimmyinlineappeal_Bd_GB
- Jimmy banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Jimmyinlineappeal_Gd_GB
- Blue facts banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1008_Backtobluefacts3_GB
October 9 Results
+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 8240 | B12_1008_Jimmyinlineappeal_Gd_GB | 137 | 245 | 36% | 49% | 36% | 2812.71 | 2020.81 | 20.530730 | 14.750438 | | 7016 | B12_1008_Jimmyinlineappeal_Bd_GB | 141 | 248 | 39% | 50% | 35% | 2671.65 | 2009.93 | 18.947872 | 14.254823 | | 4749 | B12_1008_Backtobluefacts3_GB | 460 | 688 | 34% | 59% | 72% | 8059.08 | 6160.17 | 17.519739 | 13.391674 | +------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -s 20121009090100 -e 20121009103000 -c C12_iwpenGB_FR -g b -m -i
October 23 Results
+------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 7438 | B12_1008_Jimmyinlineappeal_Gd_GB | 124 | 228 | 35% | 40% | 36% | 2522.54 | 1869.95 | 20.343065 | 15.080242 | | 6853 | B12_1008_Jimmyinlineappeal_Bd_GB | 124 | 229 | 34% | 40% | 36% | 2628.93 | 1781.16 | 21.201048 | 14.364194 | | 4392 | B12_1008_Backtobluefacts3_GB | 391 | 583 | 33% | 56% | 75% | 6327.43 | 4966.44 | 16.182685 | 12.701893 | +------+----------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -s 20121023090100 -e 20121023103000 -c C12_iwpenGB_FR -g b -m -i
- Still need banner impression comparison
October 23, 2012 ENT4 Jimmy vs Blue Facts TestEdit
Test setup:
- Jimmy banner: http://en.wikipedia.org/wiki/Australia?banner=B12_1024_T4_Proto_Jimmy
- Blue facts: http://en.wikipedia.org/wiki/Australia?banner=B12_1024_T4_Proto_FactsBlueSWDonate
- Landing page (Australia): https://donate.wikimedia.org/wiki/Special:FundraiserLandingPage?uselang=en&country=AU&template=Lp-layout-default&appeal-template=Appeal-template-default&appeal=Appeal-default&infobox=Default&form-template=Form-template-default&form-countryspecific=Form-countryspecific-control&utm_medium=sitenotice&utm_source=B12_1024_T4_Proto_Jimmy_US&utm_campaign=none_US
Full T4 Results
+------+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | lpi | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | 3970 | B12_1024_T4_Proto_Jimmy | 100 | 199 | 49% | 55% | 70% | 3922.20 | 1291.96 | 39.222000 | 12.919600 | | 2366 | B12_1024_T4_Proto_FactsBlueSWDonate | 259 | 423 | 41% | 57% | 79% | 3545.10 | 2547.98 | 13.687645 | 9.837761 | +------+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
ecom -s 20121023220000 -e 20121023230000 -c C12_iwpenT4EastAsia_FR -g b -m -i
AU & NZ BANNER breakdown:
+------+-------------------------------------+----------+-----------+--------+-------+-------+-------+---------+----------+------------+-----------+ | lpi | banner | iso_code | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------+-------------------------------------+----------+-----------+--------+-------+-------+-------+---------+----------+------------+-----------+ | 2621 | B12_1024_T4_Proto_Jimmy | AU | 77 | 77 | 43% | 100% | 100% | 3451.08 | 965.28 | 44.819221 | 12.536104 | | 1514 | B12_1024_T4_Proto_FactsBlueSWDonate | AU | 186 | 186 | 35% | 100% | 100% | 2708.22 | 2011.17 | 14.560323 | 10.812742 | | 745 | B12_1024_T4_Proto_Jimmy | NZ | 19 | 19 | 74% | 100% | 100% | 417.81 | 275.80 | 21.990000 | 14.515789 | | 1514 | B12_1024_T4_Proto_FactsBlueSWDonate | NZ | 61 | 61 | 59% | 100% | 100% | 500.19 | 411.19 | 8.199836 | 6.740820 | +------+-------------------------------------+----------+-----------+--------+-------+-------+-------+---------+----------+------------+-----------+
ecom -s 20121023220000 -e 20121023230000 -c C12_iwpenT4EastAsia_FR -g bc -m -i
AU & NZ Banner Impression breakdown
+-------------------------------------+---------+----------+ | banner | country | ban_imps | +-------------------------------------+---------+----------+ | B12_1024_T4_Proto_FactsBlueSWDonate | AU | 1377 | | B12_1024_T4_Proto_Jimmy | AU | 1342 | | B12_1024_T4_Proto_FactsBlueSWDonate | NZ | 386 | | B12_1024_T4_Proto_Jimmy | NZ | 390 | +-------------------------------------+---------+----------+
ecom -s 20121023220000 -e 20121023230000 -c C12_iwpenT4EastAsia_FR -g bc -m -i
Time Machine Comparison to 2011 for AU
- PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
+----------+---------+----------+-----------+----------+---------+----------+-----------+ | 11_count | 11_amt | 11_amt18 | 11_avg18 | 12_count | 12_amt | 12_amt18 | 12_avg18 | +----------+---------+----------+-----------+----------+---------+----------+-----------+ | 263 | 4952.75 | 3357.46 | 12.766008 | 254 | 5880.73 | 2855.49 | 11.242087 | +----------+---------+----------+-----------+----------+---------+----------+-----------+
timemachine -c C12_iwpenT4EastAsia_FR --sd11 20111116 --sd12 20121023 --st 220000 --et 230000 --country AU --lang en
- PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
- If we take just the Jimmy banner results for 2012 and multiply the donations by 2, we can estimate what our AU total would be if we ran Jimmy at 100% in 2012.
- 77 donations X 2 = 154 compared to 263 donations in 2011
- $965 Amt18 X 2 = $1,930 compared to $3,357 in 2011
Time Machine Comparison to 2011 for NZ
- PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
Comparison to Nov. 16, 2011
+----------+--------+----------+-----------+----------+--------+----------+----------+ | 11_count | 11_amt | 11_amt18 | 11_avg18 | 12_count | 12_amt | 12_amt18 | 12_avg18 | +----------+--------+----------+-----------+----------+--------+----------+----------+ | 20 | 208.29 | 204.46 | 10.223000 | 78 | 909.80 | 678.79 | 8.702436 | +----------+--------+----------+-----------+----------+--------+----------+----------+
timemachine -c C12_iwpenT4EastAsia_FR --sd11 20111116 --sd12 20121023 --st 220000 --et 230000 --country NZ --lang en
- PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
- If we take just the Jimmy banner results for 2012 and multiply the donations by 2, we can estimate what our NZ total would be if we ran Jimmy at 100% in 2012.
- 19 donations X 2 = 38 donations compared to 20 donations in 2011
- $275.80 Amount18 X 2 = $551.6 compared to $204.46 in 2011
Comparison to Nov. 17, 2011:
+----------+--------+----------+----------+----------+--------+----------+----------+ | 11_count | 11_amt | 11_amt18 | 11_avg18 | 12_count | 12_amt | 12_amt18 | 12_avg18 | +----------+--------+----------+----------+----------+--------+----------+----------+ | 17 | 146.59 | 145.40 | 8.552941 | 78 | 909.80 | 678.79 | 8.702436 | +----------+--------+----------+----------+----------+--------+----------+----------+
timemachine -c C12_iwpenT4EastAsia_FR --sd11 20111117 --sd12 20121023 --st 220000 --et 230000 --country NZ --lang en
- PLEASE NOTE: In 2011, Jimmy was running 100%. In 2012, Jimmy was running 50% and blue facts were running 50%.
- If we take just the Jimmy banner results for 2012 and multiply the donations by 2, we can estimate what our NZ total would be if we ran Jimmy at 100% in 2012.
- 19 donations X 2 = 38 donations compared to 20 donations in 2011
- $275.80 Amount18 X 2 = $551.6 compared to $204.46 in 2011
Friday November 9, 2012 US Paypal credit card optionEdit
We tested different presentations of payment methods in the US.
Page A had three buttons:
- Credit Card (credit card donations were processed through Global Collect)
- PayPal
- Amazon
Page B had two buttons:
- PayPal or Credit Card (credit card donations were processed through PayPal)
- Amazon
Results
We don't have confidence on the results for number of donations, but it looks like processing our credit card donations through PayPal did not help increase donations. We will test this again outside the US, particularly in countries where the proportion of PayPal donations is higher than in the US.
ecom -s 20121109220000 -e 20121109250000 -c C12_iwpenUS_FR -g b
+------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+----------+ | B12_FpcwDropBtnUS_A_US | 1840 | 2683 | 36% | 55% | 84% | 27221.32 | 18315.21 | 14.794196 | 9.953918 | | B12_FpcwDropBtnUS_B_US | 1787 | 2610 | 0% | None | 67% | 24310.83 | 16951.33 | 13.604270 | 9.485915 | +------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_FpcwDropBtnUS_A_US=1840%2C1000000&B12_FpcwDropBtnUS_B_US=1787%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
Meganhernandez (talk) 18:30, 10 November 2012 (UTC)
US Tuesday Nov. 13, 2012 Trying to get editors in the bannerEdit
We ran through a few tests this morning of different versions of our facts banner with the image of an editor.
Quick refresher: Last time we tested an editor appeal (Oct. 19), we used two identical facts banners (text-only, no images). One banner led to our standard Jimmy appeal and the other banner led to an editor appeal. The number of donations and amounts were nearly the same. Very encouraging, but the next step was to integrate the editor into the facts banner. Which is what we tried today.
Round 1:
- Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage
- Left editor image facts banner with "Please read a personal appeal from a Wikipedia Editor" tab: http://en.wikipedia.org/wiki/Horseshoe_Curve_(Pennsylvania)?banner=B12_EditorImage
Results: ecom -s 20121113164200 -e 20121113171300 -c C12_iwpenUS_FR -g b
+-------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | B12_EditorImage | 105 | 146 | 46% | 63% | 81% | 2151.00 | 1123.00 | 20.485714 | 10.695238 | | B12_EditorNoImage | 201 | 289 | 39% | 54% | 85% | 2837.00 | 1955.00 | 14.114428 | 9.726368 | +-------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_EditorImage=105%2C1000000&B12_EditorNoImage=201%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
Round 2:
- Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage
- Right editor image facts banner with "Please read a personal appeal from a Wikipedia Editor" tab: http://en.wikipedia.org/wiki/Rights?banner=B12_EditorImageRight
Results:
ecom -s 20121113171300 -e 20121113201300 -c C12_iwpenUS_FR -g b
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | B12_EditorImageRight | 137 | 209 | 35% | 47% | 83% | 2398.00 | 1476.00 | 17.503650 | 10.773723 | | B12_EditorNoImage | 259 | 388 | 33% | 48% | 82% | 4047.00 | 2662.00 | 15.625483 | 10.277992 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_EditorImageRight=137%2C1000000&B12_EditorNoImage=259%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
Round 3:
- Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage2
- Right editor image facts banner with "Support Wikipedia Editors" tab: http://en.wikipedia.org/wiki/Image?banner=B12_EditorImageRightSupport
Results:
ecom -s 20121113172800 -e 20121113202800 -c C12_iwpenUS_FR -g b +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | B12_EditorImageRightSupport | 129 | 218 | 40% | 46% | 36% | 2010.00 | 1329.00 | 15.581395 | 10.302326 | | B12_EditorNoImage2 | 299 | 419 | 35% | 54% | 86% | 4402.00 | 3017.00 | 14.722408 | 10.090301 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
Round 4:
- Control facts banner: http://en.wikipedia.org/wiki/Scientific_control?banner=B12_EditorNoImage_SameTab
- Right editor image facts banner with "Please help" tab and "Support our editors" under the editor image: http://en.wikipedia.org/wiki/Support?banner=B12_EditorImage_SameTab
Results:
ecom -s 20121113175500 -e 20121113205500 -c C12_iwpenUS_FR -g b +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+ | B12_EditorImage_SameTab | 435 | 697 | 42% | 49% | 77% | 6220.95 | 4341.00 | 14.301034 | 9.979310 | | B12_EditorNoImage_SameTab | 690 | 1050 | 41% | 53% | 80% | 9600.72 | 6639.72 | 13.914087 | 9.622783 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+-----------+-----------+
What this means: The upside: We were able to improve the performance of the editor banner by running through different versions in testing this morning. The downside: our control facts still beat the editor banner overall. More upside: we learned a lot in just a couple hours testing this morning.
Meganhernandez (talk) 21:42, 13 November 2012 (UTC)
Meganhernandez (talk) 21:42, 13 November 2012 (UTC)
US Tuesday Nov. 13, 2012, Gov/$5 vs Gov/SandwichEdit
A couple months ago when we were using the "google line" in the banner, we that for the third fact, using the "If everyone gave the price of a sandwich" worked better than the "If everyone gave $5.." line.
We later found the "no government funds" banner that beat "Google line" banner. Now we're testing using the sandwich fact with the "no government funds" banner.
Banners:
- "If everyone $5": http://en.wikipedia.org/wiki/Five?banner=B12_FpcwDropIevryUS_A
- "If everyone gave the price of a sandwich" http://en.wikipedia.org/wiki/Sandwich?banner=B12_FpcwDropGovSand_A
Results
ecom -s 20121113224600 -e 20121114020000 -c C12_iwpenUS_FR -g b +---------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | avg | avg18 | +---------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+ | B12_FpcwDropGovSand_A_US | 1346 | 2075 | 29% | 50% | 80% | 19960.67 | 13045.10 | 14.829621 | 9.691753 | | B12_FpcwDropIevryUS_A_US | 1345 | 2096 | 30% | 51% | 78% | 18875.10 | 12901.10 | 14.033532 | 9.591896 | +---------------------------+-----------+--------+-------+-------+-------+----------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_FpcwDropGovSand_A_US=1346%2C1000000&B12_FpcwDropIevryUS_A_US=1345%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
US Sunday Nov. 18 Ask string vs no string in bannerEdit
- no ask string in banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_10string1118_string
- ask string in banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_10string1118_nostring
ecom -s 20121118204000 -e 20121118235900 -c C12_iwpenUS_FR -g b -m --having 5
+---------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +---------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+ | B12_10string1118_nostring | 2794 | 3829 | 33% | 64% | 81% | 47199.29 | 31424.07 | 33056.29 | 41549.29 | 16.893089 | 11.246983 | | B12_10string1118_string | 2878 | 3903 | 34% | 66% | 82% | 47007.22 | 31863.22 | 33531.22 | 41195.22 | 16.333294 | 11.071306 | +---------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_10string1118_nostring=2794%2C1000000&B12_10string1118_string=2878%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true
US Nov. 19 Super short facts bannerEdit
- http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1119control
- http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1119supershort
Nov. 21 $5 & $30 vs no $5 & $30 in the banner US, CA, AU, NZ, GBEdit
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1120cntrl
- No $5 and $30 in the banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_shortening1120no530
US Results ecom -s 20121120225500 -e 20121121060000 -c C12_iwpenUS_FR -g b -m --having 30
+-------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +-------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+ | B12_shortening1120cntrl | 1239 | 1700 | 34% | 65% | 82% | 18311.82 | 12395.82 | 13119.82 | 16624.82 | 14.779516 | 10.004697 | | B12_shortening1120no530 | 1279 | 1739 | 33% | 67% | 83% | 16975.70 | 11627.70 | 12179.70 | 14847.70 | 13.272635 | 9.091243 | +-------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrl=1239%2C1000000&B12_shortening1120no530=1279%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.000075
Canada Results
ecom -s 20121121000000 -e 20121121060000 -c C12_iwpenCA_FR -g b -m --having 30
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+ | B12_shortening1120cntrlCA | 487 | 747 | 37% | 51% | 78% | 6575.96 | 4861.51 | 5123.51 | 6170.92 | 13.502998 | 9.982567 | | B12_shortening1120no530CA | 480 | 729 | 37% | 51% | 80% | 6363.67 | 4779.28 | 5051.28 | 6107.96 | 13.257646 | 9.956833 | +---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrlCA=487%2C1000000&B12_shortening1120no530CA=480%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.620773
AU Results ecom -s 20121121013000 -e 20121121053000 -c C12_iwpenAU_FR -g b -m --having 3
+------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+ | B12_shortening1120cntr | 377 | 606 | 31% | 56% | 81% | 5225.45 | 3662.12 | 3836.83 | 4914.38 | 13.860610 | 9.713846 | | B12_shortening1120no53 | 408 | 649 | 35% | 64% | 76% | 5211.51 | 3686.36 | 3848.36 | 4796.50 | 12.773309 | 9.035196 | +------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntr=377%2C1000000&B12_shortening1120no53=408%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.033164
NZ Results ecom -s 20121121013000 -e 20121121063000 -c C12_iwpenNZ_FR -g b -m
+---------------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +---------------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+ | B12_shortening1120cntrlNZ | 55 | 77 | 73% | 69% | 83% | 727.35 | 494.66 | 514.66 | 663.91 | 13.224545 | 8.993818 | | B12_shortening1120no530NZ | 49 | 66 | 69% | 81% | 65% | 527.16 | 390.00 | 402.00 | 495.44 | 10.758367 | 7.959184 | +---------------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrlNZ=55%2C1000000&B12_shortening1120no530NZ=49%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.214758
GB Results
ecom -s 20121121110000 -e 20121121150000 -c C12_iwpenGB_FR -g b -m --having 2
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+ | B12_shortening1120cntrlGB | 531 | 718 | 35% | 62% | 83% | 8381.65 | 6213.96 | 6419.15 | 7926.83 | 15.784652 | 11.702373 | | B12_shortening1120no530GB | 560 | 759 | 33% | 62% | 81% | 7852.73 | 6021.75 | 6127.75 | 7111.36 | 14.022732 | 10.753125 | +---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_shortening1120cntrlGB=531%2C1000000&B12_shortening1120no530GB=560%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.000257
Nov. 21 Canada test "We run on YOUR donations"Edit
We tested a new variation of the facts banner addressing the reader as "you" in the banner.
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122cntrlCA
- We run on your donations...If you and everyone else gave $5: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122youCA
ecom -s 20121122013000 -e 20121122653000 -c C12_iwpenCA_FR -g b -m --having 5
+-----------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +-----------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+ | B12_1122cntrlCA | 310 | 461 | 43% | 58% | 76% | 4836.89 | 3139.22 | 3323.22 | 4357.27 | 15.602871 | 10.126516 | | B12_1122youCA | 298 | 430 | 43% | 60% | 78% | 4549.15 | 3277.48 | 3490.98 | 4296.16 | 15.265604 | 10.998255 | +-----------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlCA=310%2C1000000&B12_1122youCA=298%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.120951
Nov. 22 $5 fact in centerEdit
We tested putting this fact in the center of the banner instead of on the right side of the banner: "If everyone reading this gave $5, the fundraiser would be done within an hour. Please help us forget fundraising and get back to Wikipedia."
Australia:
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1121cntrlAU
- $5 centered: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1121mid5AU
ecom -s 20121122013000 -e 20121122053000 -c C12_iwpenAU_FR -g b -m
ecom -s 20121122013000 -e 20121122053000 -c C12_iwpenAU_FR -g b -m +--------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +--------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+ | B12_1121cntr | 381 | 613 | 34% | 59% | 78% | 5935.32 | 3798.50 | 4004.50 | 5406.71 | 15.578268 | 9.969816 | | B12_1121mid | 331 | 544 | 31% | 59% | 78% | 4241.59 | 3109.23 | 3255.23 | 4106.84 | 12.814471 | 9.393444 | +--------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1121cntr=381%2C1000000&B12_1121mid=331%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.136039
New Zealand: ecom -s 20121122013000 -e 20121122053000 -c C12_iwpenNZ_FR -g b -m
+-----------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +-----------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+ | B12_1121cntrlNZ | 57 | 77 | 67% | 69% | 86% | 777.49 | 514.37 | 530.37 | 623.94 | 13.640175 | 9.024035 | | B12_1121mid5NZ | 34 | 52 | 62% | 60% | 81% | 440.44 | 250.17 | 256.17 | 286.89 | 12.954118 | 7.357941 | +-----------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1121cntrlNZ=57%2C1000000&B12_1121mid5NZ=34%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.067449
Nov. 22 GB "If Wikipedia is useful to you"Edit
When we ask donors why they gave to Wikipedia, they usually say they donate because they use Wikipedia and want to support something they find so useful. We tested a new line incorporating this idea: "If Wikipedia is useful to you, please help keep it online another year."
- Control banner: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122cntrlGB
- If Wikipedia is useful: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122usefulGB
ecom -s 20121122110000 -e 20121122160000 -c C12_iwpenGB_FR -g b -m --having 5
+------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | avg | avg18 | +------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+ | B12_1122cntrlGB | 466 | 660 | 35% | 59% | 79% | 9169.65 | 5752.79 | 5958.79 | 7691.34 | 19.677361 | 12.345043 | | B12_1122usefulGB | 514 | 716 | 33% | 56% | 83% | 9408.20 | 6095.12 | 6309.12 | 8040.05 | 18.303891 | 11.858210 | +------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlGB=466%2C1000000&B12_1122usefulGB=514%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.144127
The sample was too small here to see a significant difference, but we'll test this idea again to get more data.
Nov. 24 Top facts vs. dropdown small top barEdit
So this seems to be a HUGE breakthrough. It's a smaller banner -- much smaller. And I think the grey tab finally makes these top banners look slightly normal and not completely jarring. So: smaller, les jarring. And it does way better! But why? It's probably not the blue info icon -- at least not that alone. A short test, that we'll lengthen tomorrow, seemed to rule that out. But maybe all the differences with this banner are just working in the same direction towards more clicks and donations: the info icon, the nicer tab, the wide-column text. And/or maybe the big factor is that the "system alert" look signifies a direct message from the site's (or browser's) owner. Only the people who run the site your on have the power put an alert like this up there -- so maybe that makes it stand out as a non-BS piece of info that is definitely not going to trick you into reading some ad. More tomorrow.
United States:
- Control top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1123_Control2
- Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1123_Twolineinfo
ecom -s 20121123193500 -e 20121123232200 -c C12_iwpenUS_FR -g b -m --having 50
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_1123_Control2 | 929 | 1221 | 36% | 69% | 83% | 15204.00 | 9732.00 | 10344.00 | 13984.00 | 200.00 | 16.365985 | 11.134553 | | B12_1123_Twolineinfo | 1269 | 1691 | 32% | 66% | 82% | 18910.92 | 12389.92 | 13089.92 | 16885.92 | 200.00 | 14.902222 | 10.315146 | +----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1123_Control2=929%2C1000000&B12_1123_Twolineinfo=1269%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.002240
United States:
- Top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1123newfactsinfodot_Threecol
- Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1123newfactsinfodot_OnecolSmall
ecom -s 20121123214000 -e 20121123230000 -c C12_iwpenUS_FR -g b -m --having 100
+-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+ | B12_1123newfactsinfodot_OnecolSmall | 602 | 817 | 34% | 68% | 80% | 9173.12 | 6247.12 | 6641.12 | 8523.12 | 200.00 | 15.237741 | 11.031761 | | B12_1123newfactsinfodot_Threecol | 290 | 393 | 30% | 62% | 83% | 5084.20 | 3057.20 | 3249.20 | 4434.20 | 200.00 | 17.531724 | 11.204138 | +-------------------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
Canada:
- Control top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1122cntrlCA
- Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1124_Info_DropCA
ecom -s 20121124010000 -e 20121124040000 -c C12_iwpenCA_FR -g b -m --having 2 +----------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+---------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+---------+-----------+-----------+ | B12_1122cntrlCA | 328 | 456 | 39% | 59% | 83% | 5177.44 | 3554.99 | 3792.99 | 4771.36 | 250.95 | 15.784878 | 11.563994 | | B12_1124_Info_DropCA | 562 | 845 | 40% | 55% | 77% | 9270.17 | 5744.08 | 6070.64 | 7707.44 | 1003.80 | 16.494964 | 10.801851 | +----------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+---------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlCA=328%2C1000000&B12_1124_Info_DropCA=562%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.294505
New Zealand:
- Control top tall facts: http://en.wikipedia.org/wiki/November_24?banner=B12_1121cntrlNZ
- Top small facts dropdown bar: http://en.wikipedia.org/wiki/November_24?banner=B12_1124_Info_DropNZ
ecom -s 20121124010000 -e 20121124040000 -c C12_iwpenNZ_FR -g b -m
+----------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-------+-----------+----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-------+-----------+----------+ | B12_1121cntrlNZ | 36 | 50 | 75% | 71% | 75% | 388.82 | 340.49 | 346.49 | 388.82 | 40.93 | 10.800556 | 9.624722 | | B12_1124_Info_DropNZ | 64 | 100 | 70% | 64% | 63% | 601.67 | 523.01 | 537.01 | 601.67 | 40.93 | 9.401094 | 8.390781 | +----------------------+-----------+--------+-------+-------+-------+--------+----------+----------+----------+-------+-----------+----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1121cntrlNZ=36%2C1000000&B12_1124_Info_DropNZ=64%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.361465
UK:
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1122cntrlGB
- Top small facts dropdown bar: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1124_Info_DropGB
ecom -s 20121124110000 -e 20121124150000 -c C12_iwpenGB_FR -g b -m
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_1122cntrlGB | 446 | 590 | 37% | 63% | 85% | 8732.38 | 5397.94 | 5603.94 | 7303.74 | 399.10 | 19.579327 | 12.564888 | | B12_1124_Info_DropGB | 759 | 1033 | 31% | 61% | 81% | 11936.84 | 8553.42 | 8803.42 | 10881.70 | 159.64 | 15.727062 | 11.598709 | +----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1122cntrlGB=446%2C1000000&B12_1124_Info_DropGB=759%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.001675
Nov. 24 Blue info dotEdit
- Blue dot: http://en.wikipedia.org/wiki/Blue?banner=B12_112418_Control
- No blue dot: http://en.wikipedia.org/wiki/Optical_spectrum?banner=B12_112418_Nodot
ecom -s 20121124180500 -e 20121124200500 -c C12_iwpenUS_FR -g b -m --having 20
+--------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +--------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_112418_Control | 1808 | 2415 | 33% | 65% | 83% | 29264.55 | 18624.54 | 19794.54 | 26404.55 | 250.00 | 16.186145 | 10.948308 | | B12_112418_Nodot | 1605 | 2250 | 33% | 62% | 80% | 25925.40 | 16773.61 | 17823.61 | 23505.40 | 200.00 | 16.152897 | 11.105053 | +--------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112418_Control=1808%2C1000000&B12_112418_Nodot=1605%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.424329
Nov. 24 shorter text bar CAEdit
- Control: http://en.wikipedia.org/wiki/Optical_spectrum?banner=B12_112420_Info_DropCA
- Shorter text: http://en.wikipedia.org/wiki/Optical_spectrum?banner=B12_112420_Info_DropShortCA
ecom -s 20121124200000 -e 20121124220000 -c C12_iwpenCA_FR -g b -m --having 2 http://www.jobquotes.com
+-----------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+ | B12_112420_Info_DropCA | 548 | 789 | 39% | 57% | 80% | 8056.07 | 5381.39 | 5689.39 | 7246.76 | 501.90 | 14.700858 | 10.382099 | | B12_112420_Info_DropShortCA | 478 | 703 | 38% | 58% | 76% | 7577.93 | 5078.32 | 5402.32 | 7168.43 | 100.38 | 15.853410 | 11.301925 | +-----------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112420_Info_DropCA=548%2C1000000&B12_112420_Info_DropShortCA=478%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.022577
Nov. 25 AU 2 lines vs 3 lines top barEdit
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1225_Info_DropAU
- Shorter 2 lines: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1225_Info_DropShortAU
ecom -s 20121125000000 -e 20121125030000 -c C12_iwpenAU_FR -g b -m --having 2
+---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+ | B12_1225_Info_DropAU | 375 | 563 | 25% | 58% | 82% | 5821.80 | 3993.42 | 4231.42 | 5500.86 | 104.57 | 15.524800 | 11.283787 | | B12_1225_Info_DropShortAU | 336 | 486 | 29% | 72% | 83% | 6357.62 | 3745.91 | 4001.91 | 5543.24 | 209.14 | 18.921488 | 11.910446 | +---------------------------+-----------+--------+-------+-------+-------+---------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1225_Info_DropAU=375%2C1000000&B12_1225_Info_DropShortAU=336%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.267284
Nov. 25 GB "If Wikipedia is useful to you.."Edit
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1125_Info_DropGB
- Useful: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1125_Info_DropUsefulGB
ecom -s 20121125110000 -e 20121125145000 -c C12_iwpenGB_FR -g b -m --having 10
+----------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_1125_Info_DropGB | 711 | 961 | 30% | 61% | 81% | 11421.98 | 8143.38 | 8377.38 | 10298.86 | 320.52 | 16.064669 | 11.782532 | | B12_1125_Info_DropUsefulGB | 826 | 1136 | 31% | 60% | 80% | 13826.33 | 9466.84 | 9782.84 | 12358.33 | 320.52 | 16.738898 | 11.843632 | +----------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_1125_Info_DropGB=711%2C1000000&B12_1125_Info_DropUsefulGB=826%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.857254
Nov. 25 US "Dear Wikipedia reader"Edit
- Control: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112516_Yellowtab_control
- Dear reader: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112516_Yellowtab_dearr
ecom -s 20121125160000 -e 201211252200000 -c C12_iwpenUS_FR -g b -m --having 20
+------------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +------------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_112516_Yellowtab_control | 2103 | 2779 | 34% | 70% | 82% | 35499.75 | 21938.91 | 23290.91 | 31099.75 | 500.00 | 16.880528 | 11.075088 | | B12_112516_Yellowtab_dearr | 2422 | 3147 | 34% | 69% | 84% | 40194.49 | 25614.00 | 27248.00 | 36393.00 | 250.00 | 16.595578 | 11.250206 | +------------------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112516_Yellowtab_control=2103%2C1000000&B12_112516_Yellowtab_dearr=2422%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.610919
Nov. 25 US Blue vs Yellow tabEdit
- Yellow tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112519_Yellowtab
- Blue tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_1125BlueTab
ecom -s 20121125193300 -e 201211252200000 -c C12_iwpenUS_FR -g b -m --having 300
+----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+---------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+---------+-----------+-----------+ | B12_112519_Yellowtab | 1264 | 1666 | 33% | 71% | 81% | 22203.67 | 13303.67 | 14153.67 | 18953.67 | 1000.00 | 17.566195 | 11.197524 | | B12_1125BlueTab | 1228 | 1616 | 34% | 69% | 84% | 19622.55 | 12854.55 | 13652.55 | 18022.55 | 200.00 | 15.979275 | 11.117712 | +----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+---------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112519_Yellowtab=1264%2C1000000&B12_1125BlueTab=1228%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.944970
Nov. 25 US Yellow tab vs Grey buttonEdit
- Yellow tab: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112520_Yellowtab2
- Grey button: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112520_NoTab2
ecom -s 20121125204000 -e 201211252200000 -c C12_iwpenUS_FR -g b -m --having 100
+-----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +-----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_112520_NoTab2 | 1121 | 1486 | 33% | 67% | 82% | 19748.67 | 11973.67 | 12763.67 | 17288.67 | 250.00 | 17.617012 | 11.385968 | | B12_112520_Yellowtab2 | 1273 | 1674 | 31% | 68% | 83% | 19771.95 | 13273.50 | 14129.50 | 18181.95 | 250.00 | 15.531775 | 11.099372 | +-----------------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112520_NoTab2=1121%2C1000000&B12_112520_Yellowtab2=1273%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true Wilcox test of means p= 0.121782
Nov. 26 "If Wikipedia is useful to you" vs "If Wikipedia is useful to you... Please help us forget fundraising and get back to Wikipedia"Edit
US, CA, GB, AU, NZ
- Useful: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112605_Control_US
- Useful + Forget fundraising: http://en.wikipedia.org/wiki/Main_Page?banner=B12_112605_Min_US
ecom -s 20121126073000 -e 20131111000000 -g b --sub B12_112605_ --left 14
+----------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | banner | donations | clicks | ccpct | cccnv | ppcnv | amount | amount18 | amount20 | amount50 | max | avg | avg20 | +----------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+ | B12_112605_Con | 4121 | 6120 | 28% | 54% | 79% | 61233.86 | 42116.06 | 44099.06 | 55607.44 | 320.52 | 14.858981 | 10.701058 | | B12_112605_Min | 4879 | 7145 | 29% | 55% | 80% | 73098.62 | 50374.75 | 52748.75 | 66466.80 | 209.14 | 14.982296 | 10.811386 | +----------------+-----------+--------+-------+-------+-------+----------+----------+----------+----------+--------+-----------+-----------+
d/bi confidence test: http://www.thumbtack.com/labs/abba/#B12_112605_Con=4121%2C1000000&B12_112605_Min=4879%2C1000000&abba%3AintervalConfidenceLevel=0.95&abba%3AuseMultipleTestCorrection=true