Research:Teahouse/Phase 2 report/Metrics

back to top

Teahouse:metrics

» findings on editor participation, engagement and retention from February 27th through October 11th, 2012

WP:Teahouse was designed to introduce new editors to the Wikipedia community in a friendly and engaging way. Between February 27th and October 11th, 1,512 editors participated in Teahouse. 73% of these participants (1,098) were new editors. Guests asked 1,381 questions on the Teahouse Q&A board, and created 420 profiles. 77 editors signed up to be Teahouse hosts.

We report on Teahouse activity and impact during this roughly seven-month period. We focus our analysis on experiments we ran and new features we added during Phase 2 of the project. Data specific to the Teahouse pilot period (February through May 2012) can be found in the pilot metrics report.

Executive summary

• Participation by new editors has increased 26%. 21% more questions are being asked and 34% more profiles created.
• 29% of Teahouse guests are women, 3 times higher than the Wikipedia average of 9%.
• Teahouse guests edit 3 times more articles. They make 4 times more edits, and edit 2 times longer than other new editors.
• Automating the invite process has increased traffic and decreased host workload.
• 80% of guests rate their Teahouse experience as above average.

Overview

Methods

Surveys

To assess editors’ perceptions of Teahouse and their experiences during the post-pilot phase of the project, we surveyed 400 Teahouse guests who visited Teahouse between July 1st and October 1st. Our response rate was fairly high: 22% (89/400) of new editors who were offered the survey responded.

Edit logs

In order to assess activity on Teahouse we tracked edits to Teahouse pages by new editors and veterans on a live mirror database of en.wikipedia.org. We tracked invitations sent to guests by HostBot and aggregated editing activities of Teahouse guests with MySQL queries and Python scripts.

Page views

We aggregated pageview data from stats.grok.se for the Teahouse main page.

Glossary of Research Terms

New editor. A registered editor who either a) had made fewer than 100 edits at the time of their invitation and/or first visit to Teahouse, or b) created their account in 2012. Similar to the E1-99 editor category used by the Wikimedia Foundation for research purposes.

Host. An experienced Wikipedian who participates in the Teahouse. Most hosts have created profiles for themselves on the Teahouse Hosts page.

Guest. An editor (registered or unregistered) who makes at least 1 edit to the Guests page, the Questions page, or the Teahouse talk page.

Visit. An edit by a non-host to the Guests page, the Questions page, the Teahouse talk page, or the Host Lounge talk page.

Automated Invites. Invitations to a specific set of new editors sent automatically on a daily basis by HostBot. First implemented on 7/23/2012.

Response rate. Percentage of editors invited through a particular mechanism (such as a personalized HostBot invite) who subsequently visited the Teahouse.

WP:Teahouse

We analyzed guest and host activity patterns over time, and asked a sample of recent guests to rate and describe their overall experience on the Teahouse. We found that participation by guests and hosts has increased over time, that most new editors enjoy their experience, and that women editors participate more than expected.

“

It was nice that it existed or else I wouldn't have created the article. I don't know how you would ever get new editors without it. It's great and necessary.

”

Findings

» More new editors are participating

The total number of new editors participating in the Teahouse Q&A board and/or the Guests profile page has increased by 26% over the pilot period, from 34 new editors per week 42. The greatest increase has been seen in the three months that automated invites have been sent out.

Total new editors participating in Teahouse/Questions and Teahouse/Guests pages, per week. Feb27 - Oct11 2012

New editor participation
total new editors participating	1098
total new editors participating per week	34.2
during pilot	32.9
since automated invites	41.5

» New editors give the Teahouse high marks

68% of new editors surveyed said that they were "Satisfied" or "Very satisfied" with their Teahouse experience, versus only 5% who said they were "Dissatisfied" or "Very dissatisfied". 78% of new editors surveyed indicated that they would return to the Teahouse.

How satisfied were you with your overall experience on Teahouse?
Answer	# responses	%
Very Dissatisfied	2	2%
Dissatisfied	3	3%
Neutral	24	27%
Satisfied	30	34%
Very Satisfied	30	34%

Do you think you will participate in the Teahouse in the future?
Answer	Response	%
Yes, frequently	15	17%
Maybe once in a while	54	61%
No, definitely not	4	4%
I'm not sure/I haven't decided	16	18%
Total	89	100%

When asked to describe what in particular they liked about their experience, new editors cited a range of factors, from the promptness and quality of the answers they received to the friendly atmosphere and the ease of use.

Was there anything in particular you liked about your Teahouse experience? Click 'Show' to view all responses
Tea House is a very good for new users because new users can learn many things at it.
It's very pretty compared to the rest of Wikipedia. I enjoy the tea house decor in real life and it was refreshing to see something like it on Wikipedia.
Quick and firm responce.
friendly helpful people
its a good way for newbies to get used to Wikipedia style and prtotcols
as of yet, nothing in particular, the welcome was warm enough, perhaps a follow up would have been nice.
It was somehwere to get soem help which is great
It's a great place to get started (for new users).
My questions were answered very quickly.
The fact that it exists and was helpful to me.
The fact that there is somebody "out there", that there is a sincere community, gives a professional and safe feeling about Wikipedia
As mentioned previously, the speed of the responses. Sometimes, particularly if you're concerned about something, it's nice to know there is a way of getting an (almost) immediate response. It makes it feel as if there IS someone to talk to about things.
balance
The editors are very friendly and patient, which is great when compared to the rest of Wikipedia in how new editors are treated.
I got my question answered. It has a nice friendly non-techie look to it.
I received answers to my questions regarding editing
Questions were answered quickly, the atmosphere felt very supportive, and most importantly, I received very helpful and clear answers to questions.
N/A
Yes, I liked that you can still edit things if you make any mistakes.
I think it is a very organized system and an improvement for the Wikipedia community.
very helpfull
All hosts were very welcoming and understanding! The Teahouse has greatly helped me to overcome being timid about editing! Thank you all sooo much! I have so much appreciation for you.
In spite of feeling pretty lost as far as how to create a Wikipedia page, everyone was very friendly and accepting.
I liked meeting other wikipedians. It's very interesting to know about them.
Senior editors like Sarah, NtheP and one other his moniker is hard to remember  :-) are all prompt in providing answers so I can move forward.
I like how I could meet more experienced editors, and I felt very welcome when somebody invited me to the Teahouse.
It was nice that it existed or else I wouldn't have created the article. I don't know how you would ever get new editors without it. Its great and necessary.
No.
got a quick and helpful answer
Yes! It is superb and very friendly. Love!
Very welcoming.
Heather was very kind and supportive after my terrible blunder. I just wish the Teahouse was more visible especially when someone creates a new account.

“

As mentioned previously, the speed of the responses. Sometimes, particularly if you're concerned about something, it's nice to know there is a way of getting an (almost) immediate response. It makes it feel as if there IS someone to talk to about things.

”

Was there anything in particular you disliked about your Teahouse experience? Click 'Show' to view all responses
The people were rude.
It was not that interesting that I could go back again. There was really nothing to do.
See comments above.
There is nothing to do except look at people's profile. I thought it would be more excited and creative.

“

It was not that interesting that I could go back again. There was really nothing to do.

”

Is there anything else you'd like to share with us? Click 'Show' to view all responses
Overall WMF projects are really a neat idea for the sharing of educational and especially free-based information. The teahouse is definately on the right track. I am sure I will return periodically to see how it develops and after this survey may even participate in question answering or other things needing mentioning.
Just that I have tons of admiration for the collaborative nature of the your hard work and want to be involved more
Nothing else, except thank you for giving me this survey! I was glad to give some feedback.
I don't function very well in team sports.
No but it was helpful.
first, thanks very much to those that helped me. The answers were not particularly helpful but they made me realize that this is a huge trial and error system in which the smallest details mean a lot. I don't even know yet how to ask a question so that it can be answered.
That last question was not very helpful. 'Female' is not a gender, it is a sex. As such, you can have technically 'female' men and 'male' women. It's also rather demeaning to refer to women as 'females' because it is the way you'd speak about an animal.
I don't understand why the photos are on a site called Wikimedia Commons, rather than Wikipedia Commons. I can't see any link from the Wikipedia site to the Commons site.
this is an amazing, frustrating site. it has all of the disadvantages of being totally anarchic. and all of the advantages of nice people helping other nice people. figure out the solution to that and you make it heavenly. thanks bpolk
Thanks for your help. Wikipedia is very hard for beginners and your help was vital for me, So THANKS guys!
This survey is all about nothing...
Wiki is a terrific resource and I would like to think by doing minor edits when I come across for instance typos, citations I can add or extend an article in some way, I'm able to put something back in return.
I have found tagging images daunting. So many editors with different views. Could use an experienced editor to assist and counsel.
Thank you, Teahouse folks!
My responses should be considered null because I don't really have any valid experience with Teahouse nor really know it's purpose Things are very complex - overly so - on Wikipedia.
Although I was ultimately satisfied with the Teahouse, I initially thought it was a mistake to try it. The first contact from an Editor was confrontational and referred to editing techniques not known to me as a beginner. Editors should remember that, most likely, a person's asking a question is less knowledgeable of Wiki's functionality (ie editing) and rules.
Nthep answered my question, and my question got answered perfectly!
I think a little edit I did needs attention but I couldn't get anyone's attention so I gave up.
where do we show the proof that the page we created is True to keep the page alive
I would like to share about our articles. There should be a Wikipedia project that is creative and expresses Wikipedians.
See above. (I despair of getting anything helpful. I expect to be told of a response somewhere locked in your world, but that won't help me.)
Nothing, except maybe you should change the part "encyclopedia anyone can edit", because all sorts of idiots are misinforming us with the power of words.
just grateful you guys are there so that I know I have somewhere to go in times of need and that the response is fast and friendly. Thank you.
I would like to share about my editing. I have only edited twice because of all your rules, but I've figured out EVERYTHING thanks to Trypofish. He was very helpful to me and my edits. If you wish to contact me, I'm RAIDENRULES123
Thanks for organizing the Teahouse - it's been a great help to me!
I'm just glad to see there are people willing to help, especially if you can't pay someone to help you. The article I created had to exist and was overdue and without help Wikipedia would be worse off for not having it, same with any other article contribution. Just very happy to see people are there to help people like me who are inexperienced so they too can contribute.
Well, nothing. Thanks!

“

just grateful you guys are there so that I know I have somewhere to go in times of need and that the response is fast and friendly. Thank you.

”

» Female newcomers participate in Teahouse at a higher rate

30% of respondents to our most recent survey identified themselves as women. This figure is in line with the results from our two previous surveys, in which 22% (13/59) and 35% (17/48) respondents identified as women. The overall rate of female participation, as measured by these survey responses, was 29%.

If you're comfortable telling us, what is your gender?
I am female	27	30%
I am male	51	57%
I'd rather not say	11	12%
Total	89	100%

Cumulative survey results: Female participation
Survey	Date	Female Respondents	Total Responents	% Female
Survey 1	April	13	59	22%
Survey 2	June	17	48	35%
Survey 3	October	27	89	30%
	Total	57	196	29%

This is a noticeably higher rate than figures for female participation in Wikipedia as a whole, which ranges between 8% and 13%. This finding strongly suggests that the Teahouse is making progress toward one of the project's original goals of creating an appealing space for new female editors to get support, by offering a new approach to help on Wikipedia.

These survey numbers are likely to accurately reflect the true proportion of female Teahouse participants. New editors who received one survey were not asked to participate in subsequent surveys, whether or not they had chosen to fill out the first. And although it is possible that during the pilot period some female editors were invited to participate by Teahouse hosts specifically because they self-identified as women on their Wikipedia, the majority of respondents to the third survey were new editors who had been invited automatically by HostBot, with no consideration for their gender.

» Most guests either ask a question or create a profile

We were surprised to find that most new editors who visit the Teahouse only participate in one activity. While 70% of new editors ask a question and 40% create a profile, only 10% do both.

Relative proportion of newcomers participating in Teahouse/Questions vs Teahouse/Guests pages, Feb27 - Oct11 2012

New editor participation by page
% of new editors who participated in Teahouse/Questions	70%
% of new editors who created a Teahouse/Guests profile	40%
% of new editors who participated in both /Questions and /Guests	10%

Since most guests visit the Teahouse for particular reasons, suggesting that one strategy for increasing participation beyond the current level would be to offer additional calls to action or new ways to participate. Some possibilities might be new calls to action around suggesting editing tasks for newcomers, creating new discussion forums not focused on Q&A, and better surfacing of WikiProjects that would welcome new members.

» More hosts are participating

The total number of hosts participating in the Teahouse (by answering questions on the Q&A board, building pages, and participating in talk page discussions) has increased by 19% over the pilot period weekly average, from 21 hosts per week to 25.

Hosts participating per week on WP:Teahouse, February - October 2012

Host participation
total hosts participating	77
average hosts participating per week	21.4
during pilot	20.7
during phase 2	24.7

Q&A board

The Teahouse passed the 1,000 questions milestone during Phase 2, and the Q&A board continues to be the most active area on the Teahouse. Most questions are asked by new editors, and most are answered by Teahouse hosts, although many experienced Wikipedians also participate in Q&A by answering questions as 'unofficial hosts'. Questions span a wide variety of topics: and while some are relatively simple and easily 'resolved', many spark detailed, animated discussions.

Teahouse/Questions
total questions asked	1381
average questions per guest	1.6
% of questioners who asked more than one question	23%
% of questioners who respond in their own question thread	48.6%
average responses per question	2.9
median time to first response	33:20 minutes
questions per day	6
during pilot	5.6
since automated invites	6.8
questions per week	42.2
during pilot	39.5
since automated invites	47.7

Findings

The Q&A board seems to be a very stimulating and interactive place for new editors. 80% of respondents stated that they were 'satisfied' or 'very satisfied' with the quality of the answers on the Q&A board. 88% of editors stated that they received a follow up message from a Teahouse host after they asked their question (generally a Teahouse talkback template, and/or a personal message). However, most guests are still hesitant to answer questions.

» Question volume has increased

The Q&A board was experiencing a lull in activity at the start of phase 2, but activity increase noticeably after we implemented automated invites on July 23rd and have remained high ever since. Current Q&A activity (48 questions/week) is 20% higher than during the pilot period (40/week) and 45% higher than during the transition period (33/week) between the end of the pilot and the deployment of automated invites.

Questions per week on WP:Teahouse/Questions, February - October 2012

» Many questions receive a response in 30 minutes or less

Teahouse hosts answer guests' questions rapidly. The median time between when a question is asked and its first response is 33 minutes and 20 seconds. Average response time has been shrinking week by week, possibly as a result of increasing host participation.

Median response time to questions on WP:Teahouse/Questions, by week, February - October 2012

» Each questions receives around 3 answers

Teahouse guests can expect several answers to their question. Each question on the Q&A board receives an average of 2.93 replies from one or more other editors. However, the number of answers per question has decreased since the end of the pilot period.

Mean number of answers to questions on WP:Teahouse/Questions, by week, February - October 2012

» New editors were very satisfied with the answers they received

Did you ask a question on the Teahouse Q&A board?
Answer	# responses	%
Yes	69	74%
No	24	26%
Total	93	100%

How satisfied were you with the quality of the answer you received?
Answer	Response	%
Very Dissatisfied	2	3%
Dissatisfied	3	5%
Neutral	8	12%
Satisfied	26	39%
Very Satisfied	27	41%
Total	66	100%

When asked to describe what in particular they they liked about the answers they received, new editors cited a range of factors, from the promptness and number of answers they received to the friendly tone and the level of detail.

Was there anything in particular you liked about the answer your received? Click 'Show' to view all responses
Charles was great - other responses were less informative. Specific answers were the best.
Gentle tone to a new user
The two who answered my questions were clearly into helping people learn the ropes on Wikipedia
It was moderately informative
A warmest and friendly answering the question
They just seemed like they wanted me (a rank beginner) to succeed.
I have asked several questions and the answers ranged from mediocre to excellent. I especially appreciated learning that mostly all editors are equal
The person who answered my question was really helpful. They gave me some specific pointers, but more than that they made me feel welcome and like I had someone to turn to for help.
It partially fixed the problem
well, it still took me a few hours/days/pulling the last remnants of my hair out but i eventually persevered to figure out how to do what i needed. the results i got totally were determined by the patience and insight of the individual i encountered. sarah stierch was wonderful. but she was one part of a long chain and, had she not been the one to give me a hint or an answer, I would have been stymied. answers on wiki seem to totally depend on the nurturing nature of the individual who responds. that's somewhat scary.
My submission was declined twice, I then updated it again and submitted it but didn't hear anything for days. I then posted a message on tealeaf and had an almost immediate reply letting me know that he/she will have a look at my page and also told me that it was looking good for submission. The page was then allowed to go through.
There were two answers and one proposed to help me set up the fact box, which was a great help
I really appreciated the speed of the responses and their advice helped ease some of my concerns.
how do you put pics and videos on here.
It was comprehensive and addressed both the abstract, general answer to my question and the more personal, specific-to-me answer.
Exactly what I was looking for.
clear and consice
When the editor understood I was a beginner, she was helpful with specifics
Concise but very specific and helpful!
They explained it very well and were specific. Specific is the key to me.
Extremely detailed. Plainly a lot of effort had been put into answering my question.
nothing like that. but it made me easy to get
I have visited twice with a total of 3 different questions. All answers were informative, easy to understand, and thoroughly explained. I particularly appreciated that one host's answer built upon the previous one's - giving me a complete picture. I'd like to mention the host dalahäst who went above and beyond in giving much additional quidence on his talk page when I asked him questions there.
Message was very friendly and positive.
as I'm a novice any help is appreciated. I found the response fairly simple to understand and resolved the issue I had.
Sarah's input help tremendously in making a direct translation of a person's military career into a life of a real person. Interesting indeed. Am motivated to do more of this type of writing. Hope Sarah will continue to provide feedback for continuous improvement. Much, much obliged and mucho gracias indeed. CHHistory
I like how descriptive and detailed the editors were, and one even went the extra mile and advised me about something on my user page.
It was accurate and onpoint
I liked that it was answered, but, it took months to answer, and I am unclear of the answer and where it is,
I initially asked if I could pay someone to help me create the page but was told that is a no-no, so I took whatever help I could get and that help ended up being very good and came from a few different people which was great.
No.
to the point with helpful links and encouraging
Direct Links
Quick, multiple replies gave me a lot of needed information.
After my blunder as a Newbie I got slammed by some other editors. I was kindly taken in by Heather at the Teahouse

“

They just seemed like they wanted me (a rank beginner) to succeed.

”

Was there anything in particular you disliked about the answer your received? Click 'Show' to view all responses
I have found becoming a new editor pretty difficult. Worse yet, i find when asking questions people tend to not be too helpful simply linking to generic pages. I read the pages, thats why i was requesting more details :)
The person who responded to me became condescending and aggressive.
Yes, I am completely lost at trying to be involved with Wikipedia. I admit that I am not a young geek, but I am reasonably sophisticated (Ph.D. in math from Duke, J.D. from Yale, Fortran programmer). But I find everything about Wikipedia completely inaccessible.
I didn't like the way the person explained. I have heard from better Wikipedians. They needed to explain properly.

“

I didn't like the way the person explained. I have heard from better Wikipedians. They needed to explain properly.

”

» New editors are still hesitant to answer questions

Our findings here are similar to those from the pilot metrics report. While 74% of new editors surveyed indicated that they had asked a question on the Q&A board, only 7% said they had answered one. If the Q&A board is meant to be a peer-support space, this is still room for improvement in this area.

Why haven't you answered any questions on the Q&A board?
Answer	Response	%
I couldn't figure out how to answer a question	6	7%
I didn't know I was allowed to answer questions	23	28%
I didn't see any questions I knew the answer to	20	24%
I didn't feel like it	5	6%
some other reason (please describe)	29	35%
Total	83	100%

When asked to describe in their own words why they had not answered a question on the Q&A board, many guests said that they did not feel they had the expertise to answer a question, or that they were concerned about making mistakes or giving incorrect answers.

Why haven't you answered any questions on the Q&A board? Click 'Show' to view all responses
I'm very new and I'm just going to be quiet and learn.
I've programmed for 25 years but wiki'd for only 25 hours
I'm a total Wikipedia "newbie" .
I am less active on Tea House. In last days I was busy in my first two wikipedia articles i.e. Lottery short story by premchand and Gajar ka Halwa. That's the reason. Thank you.
I have been mostly very busy trying to update the few pages that I have become involved in. I do not feel at all qualified yet to answer a question so mostly I don't even read them
I'm confused about what is the Q&A board and the Teahouse
i was fighting my own battles as a newbie
I have been busy trying to set up my company's own page and do not feel "expert" enough to help others
See Above
Didn't know it was there.
I didn't know I was allowed to answer questions as given above but I also don't feel I'm sufficiently experienced with the intricacies of Wikipedia!
I am new to Wikipedia
I had such a bad experience, I don't think that this is the best thing to do.
I haven't looked at other questions and doubt that I would be of any help to anyone since I am new at editing Wikipedia.
all
Have not really gone there
I'm new and doing more learning than teaching
I do not have such answers or knowledge
I'm not yet competent enough to help someone else
I have no interest in doing so. Editing articles that are in my interest is enough for me right now. Plus, I don't feel I am qualified to answer any questions yet.
Some day I hope to! But I feel I'm too new right now.
wikipedia is not an easy site to understand, let alone quickly and easy do stuff (create pages, reply to questions) unless you have done some serious learning (and that learn curve is steep)
I was going to visit it, but i hadn't enough time.
Am to new to this. Once I have completed an article that is good enough for posting live, than I will help others the best I know how.
I don't really like to answer questions that other people ask because I might get it wrong, and most Wikipedians are very rude.
There are others who are much more qualified to do that than I am.
I am still a noob myself and senior WP authors will know better answers
I haven't had a chance to answer the questions (not logged in for a while)
I don't feel like I know enough about Wikipedia and all the formatting

“

I didn't know I was allowed to answer questions as given above but I also don't feel I'm sufficiently experienced with the intricacies of Wikipedia!

”

What made you decide to answer a question? Click 'Show' to view all responses
I was just lurking but I saw a question that was obvious and that I knew with strong certainty
The popup (after writing a profile) told me to check the questions page, and I saw a few questions that I thought I could answer.
A statement was made with which I - very mildly - disagreed.
Think I said THANKS!

“

I was just lurking but I saw a question that was obvious and that I knew with strong certainty

”

Guests page

Findings

» New editors enjoyed browsing other guests' profiles and creating their own

Around 40% of survey respondents stated that they had created a profile. Guests' levels of satisfaction with the profile creation process was comparable to the level of satisfaction reported during the pilot period, with 68% of respondents saying they were 'satisfied' or 'very satisfied'.

Did you introduce yourself on the Guests page?
Answer	# responses	%
Yes	38	43%
No	51	57%
Total	89	100%

How satisfied were you with the experience of creating your guest page introduction?
Answer	# responses	%
Very Dissatisfied	0	0%
Dissatisfied	1	3%
Neutral	11	29%
Satisfied	19	50%
Very Satisfied	7	18%
Total	38	100%

Guests highlighted the ease of creating a profile and the ability to browse other profiles as particularly beneficial. No guests identified any aspects of Guest profiles that they especially disliked. The majority (55%) of guests who did not create a profile said that they didn't know it was there. Others said that they could not figure out how to create a profile and cited lack of familiarity with markup. One guest said they weren't comfortable personal information on the internet, and another said "I am only interesting in posting information that should/could be of interest to others. my mission here had nothing to do with me."

Was there anything in particular you liked about the experience of creating an introduction? Click 'Show' to view all responses
To be quite honest, my initial immersion into Wikipedia was so mind boggling, I can't remember if I really did an intro or not.
I get to put myself out there as a part of a thing far greater than myself, a thing that all of us share, because it's not just people's Wikipedia, but it's also mine.
Picking my own image from commons
Remembering my favorite quote by Mark Twain
Not really anything in particular.
This question has made more aware of the possible value of the introduction. I never really occured to me that someone would actually take the time to read it.
read the intros, found new people with interesting stories to tell on how they started with wiki
Yes, now people know about me.
I should go to the Teahouse pages because I'm a bit confused about this and the past 2 questions. I'm unsure whether or not I did a "guest introduction" - I simply posted my questions under a new section.
I mainly liked telling about myself.
I liked how you can explain about yourself and discover about others. This is what I liked in particular.
I just like typing, because it is like writing (my best school subject), only without the hand-aches (not headaches).
I like how we got to choose a picture from Wikimedia Commons.
Yes! I loved it.
I liked the Syntax method of writing and uploading pictures.

“

I get to put myself out there as a part of a thing far greater than myself, a thing that all of us share, because it's not just people's Wikipedia, but it's also mine

”

Why didn't you introduce yourself?
Answer	# responses	%
didn't feel like it	5	10%
couldn't figure out how	6	12%
didn't know it was there	28	55%
didn't want to be visible to others	2	4%
didn't want to be associated with the Teahouse	0	0%
some other reason (please describe)	10	20%
Total	51	100%

Why didn't you introduce yourself? Click 'Show' to view all responses
Not that I can remember but you may be able to clarify
i guess (and this, i suppose, could be taken as a criticism - but isn't meant to be) that i was and am only interesting in posting information that should/could be of interest to others. my mission here had nothing to do with me.
As I'm unsure of the etiquette within Wiki, which I feel is the reason I ended up embroiled in the situation I was asking advice about, I'm fairly weary of drawing undue attention; does that sound right or just plain silly?
I had no idea that I was supposed to do thtat.
I will in the near future.. when i get the time
Didn't know I was a guest.
a) couldnt find it, b) couldnt figure out how, c) didnt know it was there - again i state, wikipedia is not an easy system to learn/understand
too new, let me gain some experience first.
do not like to give personal information on the internet
As stated earlier feel I don't know enough about formatting etc.

“

As I'm unsure of the etiquette within Wiki, which I feel is the reason I ended up embroiled in the situation I was asking advice about, I'm fairly weary of drawing undue attention; does that sound right or just plain silly?

”

» More new editors are creating profiles

Although profile creation declined over the course of the pilot period, it increased substantially over the course of Phase 2. Between July 23rd and October 11, an average of 19.3 new profiles were created per week, compared with 14.4 new profiles per week during the pilot period, a 34% increase.

Number of new editors creating profiles on Teahouse/Guests page, per week. Feb27 - Oct11 2012

Teahouse/Guests
total profiles created	482
profiles per day	2
during pilot	2.1
since automated invites	2.76
profiles per week	14.2
during pilot	14.4
since automated invites	19.3

We highlight two factors in particular which likely contributed to this increase:

• The implementation of an 'archiving' script that limited the number of profiles on the main Guests page to 10-20 of the most recent profiles. Profiles of previous guests were archived to the Teahouse 'guestbook'. This change was intended to increase the perceived value of creating a profile by making new profiles more visible.
• The implementation of automated invites allowed us to reach out to more new editors each day. These new editors visited the Teahouse at the same rate as those guests who were invited manually during the pilot period. However, where many manual invites were sent to guests who a host had judged as needing help, automated invites were sent to guests regardless of whether or not they currently needed assistance. We believe that since fewer of these guests had a questions in mind when they visited, many more of them created profiles instead.

Invitations

New editors find out about the Teahouse in a variety of ways. During the pilot period, manual invitations were the primary mechanism. During Phase 2, several other mechanisms were developed.

Manual Invitations

During the pilot period, all invitations to visit the Teahouse were sent manually by Teahouse hosts, with two hosts (Rosiestep and SarahStierch) sending 80% of all tracked invitations.

While this process was effective at bringing new editors to the Teahouse, the response rate was quite low: only around 4.5% of those who received an invite subsequently visited Teahouse. Anecdotally, hosts who sent many invites expressed frustration at this low rate of return, considering how labor intensive the process of manually inviting editors was. Many editors continued to manually invite new editors to the Teahouse during Phase 2.

External links

Because invitations are crucial for driving traffic to the Teahouse, but manual invitations alone was not a sustainable strategy, we decided to experiment with two other mechanisms for making new editors aware of the Teahouse: automated invitations and a custom Article Feedback call to action.

The Wikipedia Help Desk also added a link to the Teahouse, and a special message to new editors, to their page header. The community has also added Teahouse links to many other help spaces, such as:

• en:Wikipedia:Mentorship
• en:Wikipedia:Questions
• en:Wikipedia:Noticeboards
• en:Wikipedia:Adopt-a-user
• en:Help:Contents/The_Wikipedia_community

We have not explicitly tracked inbound traffic or the response rate from these sources.

Automated invites

We developed a Python script that automatically invited new editors who met our basic invitee criteria.

On July 23rd, HostBot was approved by the Bot Approvals Group to begin sending automated invitations to all new editors listed on the invitee reports. While the goal of the Teahouse is to be as inclusive as possible, after conversation with community members we decided to program HostBot to not invite users who had received level 4 warnings, or who had been accused of sockpuppetry.

In evaluating the long-term effectiveness of automated invites, we were particularly interested in answering the following questions:

• Are new editors more or less likely to respond to an invitation sent by a bot than an invite sent by human?
• Are new editors more likely to respond to a personalized invitation or a generic one?

Article Feedback v5 call to action

We worked with the Wikimedia Foundation's Editor Engagement team to implement a Teahouse Call to Action (CtA) for the article feedback tool, which would be displayed to 10% of registered editors who submitted article feedback. The Teahouse CtA was rolled out on August 21st, and has (as of October 14th) been displayed to 881 editors who filled out Article Feedback.

Findings

» Direct invitations continue to drive traffic to Teahouse

During the pilot period, we found that the majority of Teahouse guests were invited guests. This pattern continued in Phase 2, with 44% of survey respondents stating that they were invited to the Teahouse via a talk page invitation.

How did you find out about Teahouse?
Answer	# response	%
Someone invited me on my own talk page.	41	44%
Someone emailed me an invitation.	6	6%
Another editor referred me to Teahouse.	17	18%
I am a member of a class, and my instructor told me about Teahouse	0	0%
I read about Teahouse on Wikipedia.	22	23%
Other (please describe below)	8	9%
Total	94	100%

How did you find out about the Teahouse? Click 'Show' to view all responses
I don't know. I think I arrived there by hit and miss
A bot invited me on my own talk page.
I was looking for help/advice and reached it through the help pages.
I found it
asked to join
i asked a question and Teahouse forum was among the solutions
I was looking to discover all of the features of Wikepedia and found it.

“

I was looking for help/advice and reached it through the help pages.

”

» Automated invites yield the same response rate as manual invites

HostBot sent out a total of 4268 invites between July 23rd and October 11th, and 165 (3.9%) of invitees subsequently participated in the Teahouse (asked or answered a question and/or created a profile). This response rate is not statistically different from the response rate (4.27%) for manual invitations sent to invitee report candidates during the pilot period, as determined by a 2-tailed t-test (t=0.68, p > .05, 2-tailed).

Despite some initial concerns that removing editorial discretion from the invitation process might lead to an increase in vandalism or disruptive editing on the Teahouse, we have not observed any such increase in the time that automated invites have been running.

Given the findings above, and the facts that automated invitations are much less labor-intensive than manual invites and have contributed to an increase in new editor participation, we view automated invitations as a key component of the Teahouse's continued success.

» Personalized messages did not yield significantly different results from generic invitations

Previous research has determined that adding a degree of personalization to templated talk page messages (for instance, by using active voice and calling out the messager as a fellow editor) can yield better responses in some scenarios than generic messages. We tested the effectiveness of personalization in automated invites by including a personal signature from a Teahouse host in roughly 50% of messages, while leaving the other 50% generic.

We found that adding personalization to an automated invitation does not affect response rate. A sample of 2435 automated invites yielded response rates of 3.8% for both personalized and generic invitations. Findings and a full discussion of these results can be read on the invites experiment page.

Given this result, we decided to make all invitations personalized. A personalized invitation gives the invitee a personal point of contact that they can message directly if they choose to, and may help new users feel like they are a part of a community. We currently have no plans to continue these experiments.

» Few AFT respondents subsequently visited the Teahouse

Out of the 881 registered editors who received the Teahouse call to action after submitting their article feedback, 7 (0.08%) have visited the Teahouse so far.

Page traffic

The main Teahouse page was viewed 39,678 times between February 27th and October 13th, for an average of 173 (non-unique) pageviews per day. The graph at right shows page views per day during that period.

Unique page views, by day, for WP:Teahouse. February 27th - October 14th 2012.

» Page traffic has increased over time

The graph above shows that page traffic has increased in recent months. There were an average of 155 pageviews per day during the pilot period, versus 204 views per day since the introduction of automated invites. The dramatic spike in pageviews around July 15th corresponds to Wikimania 2012, during which the Teahouse was discussed in several presentations and panels.

Teahouse pageviews
total page views	39,678
page views per day	155
during pilot	154
since automated invites	204

» Most page requests come from the US and UK

An analysis of a 10% sample of all pageviews for WP:Teahouse and its subpages shows visitors from 101 different countries. Over 65% of all page requests come from within the United States, followed by the United Kingdom with 10% of all page requests. Not surprisingly, this distribution of page requests by country closely parallels that of en.wikipedia.org.

pageviews of WP:Teahouse and its sub-pages between 4/05/2012 and 10/01/2012, by country of origin

WP:Teahouse/* pageviews by country (click 'show' to view)
United States	7179
United Kingdom	1079
Canada	413
India	287
Netherlands	225
Germany	222
Australia	216
France	195
China	148
Russian Federation	124
Spain	47
Japan	42
Poland	38
Sweden	38
Philippines	37
Denmark	35
Ukraine	34
New Zealand	31
United Arab Emirates	28
Indonesia	27
Ireland	26
Israel	26
Italy	25
Turkey	23
Iran Islamic Republic of	20
Singapore	20
Finland	18
Malaysia	18
Switzerland	18
Brazil	17
Argentina	16
Pakistan	16
Hong Kong	15
Belgium	14
South Africa	14
Czech Republic	13
Korea Republic of	13
Romania	13
Saudi Arabia	13
Austria	12
Norway	11
Europe	10
Greece	10
Thailand	10
Vietnam	9
Ghana	8
Latvia	8
Bangladesh	7
Iceland	7
Taiwan	7
Estonia	6
Bulgaria	5
Egypt	5
Qatar	5
Slovenia	5
Syrian Arab Republic	5
Zimbabwe	5
Croatia	4
Georgia	4
Hungary	4
Isle of Man	4
Venezuela	4
Brunei Darussalam	3
Colombia	3
Guernsey	3
Jamaica	3
Lebanon	3
Macedonia	3
Mexico	3
Portugal	3
Puerto Rico	3
Sri Lanka	3
Uganda	3
Uruguay	3
Albania	2
Anonymous Proxy	2
Armenia	2
Costa Rica	2
Cuba	2
Cyprus	2
Jordan	2
Kazakstan	2
Kenya	2
Morocco	2
Satellite Provider	2
Serbia	2
Slovakia	2
Algeria	1
Angola	1
Belarus	1
Bermuda	1
Fiji	1
Jersey	1
Luxembourg	1
Moldova Republic of	1
Montenegro	1
Nepal	1
Nigeria	1
Peru	1
Sudan	1
Vanuatu	1

New editor retention

Sample: 149 Teahouse visitors and 1954 non-vistors invited between 2/27 and 7/25. Edits to Teahouse pages not included in counts. Only non-visitors who made at least 1 edit to Wikipedia after the date of their invite are included.

In order to determine whether participating in the Teahouse has a positive effect on an editor's likelihood of continuing to edit Wikipedia and on their level of editing activity, we analyzed the editing behaviors of a sample of editors who were invited to the Teahouse by a host between February 27th and July 25th. Editors in this sample were invited (manually) to the Teahouse because their names appeared in the daily Invitee report or their submission was rejected by Articles for Creation.

To control for the possibility that new editors who did not visit had simply not seen the invitation, we excluded from our sample those who did not make at least one edit to Wikipedia after the date of invitation. This left us with a sample of 1,948 editors, 149 of these editors visited the Teahouse. For the editors who visited Teahouse, any edits they made to WP:Teahouse (and its sub-pages) were ignored in this analysis. We also ignored any edits made by editors in our sample prior to their invitation.

Our main findings from this analysis are highlighted below. Because our edit data are not normally distributed (in both groups, a small number of editors made hundreds or thousands of edits while most made from a few to a few dozen) we present both mean and median averages, where appropriate.

Editing activity by Teahouse visitors vs. non-visitors

	visitors	non-visitors
mean post-invite edits	241	75
median post-invite edits	29	12
mean weeks with edits	5.75	3.65
median weeks with edits	4	2
mean articles edited	76.5	21.2
median articles edited	5	2
% with article edits	74%	81%
% edits to talk namespaces	35%	20%

These findings show that Teahouse visitors are more active than new editors who do not visit Teahouse, and that they exhibit certain positive behavioral characteristics (such as editing more than one article, or participating in talk page discussions) to a greater degree than editors in the control group. While the findings presented below do not prove definitively that participating in the Teahouse causes editors to exhibit these behaviors, that is a plausible interpretation, given our findings on editor activity and satisfaction, presented above. For instance, it is easy to imagine how getting a helpful answer from a friendly Wikipedian at the Teahouse could make a new editor feel more comfortable and confident participating in talk page discussions. However, even taking the position that the Teahouse tends to attract editors who are most likely to become active, productive contributors anyway, we believe that the Teahouse provides these editors with better and earlier assistance and more positive community interaction than they would likely have received otherwise.

“

They gave me some specific pointers, but more than that they made me feel welcome and like I had someone to turn to for help.

”

Findings

» Teahouse guests make more edits to more articles

On average, Teahouse guests made more edits (241 versus 75), and edited more articles (76.5 versus 21.2). This is encouraging since many new editors start out with the intent to create or contribute to a single article. Encouraging new editors to branch out and edit more articles may lead to higher long-term retention.

Average cumulative edits by Teahouse visitors vs. invited non-visitors, February - October 2012.

Average number of articles edited by Teahouse visitors vs. invited non-visitors, February - October 2012.

» Teahouse guests edit more each week, for more weeks

The Teahouse visitors in our sample edited roughly twice as long on average as non-visitors, measured by the number of weeks after being invited in which they made at least 1 edit to Wikipedia. They also edited more on average during weeks when they did edit. We analyzed the subsequent editing activity of 1,655 editors (133 invited visitors, 1,522 invited non-visitors) from the pilot period (February 27th - May 27th).

Teahouse visitors made 7 edits per week on average, while invited non-visitors made 2.1 Teahouse visitors also made more mean edits in 21/21 (100%) of all weeks, and more median edits in 17/21 (81%) of weeks.* The first chart below (left) shows mean and median edits-per-week for the two groups during the subsequent 21 weeks (May 28th - October 18th).

The second chart below (right) shows the average increase in cumulative edits over time for these two groups. Teahouse visitors invited during the pilot period had over twice as many edits on October 21st as they had on May 28th (an 115% increase), while the non-visitor group increased their total edit counts by only 77% over that 5 month period.

Mean and median edits per week by Teahouse visitors vs. non-visitors who were invited during pilot, between May 28 - October 18 2012.

Increase in average total edits per editor between May 28 - October 21, for Teahouse visitors and non-visitors invited during the pilot period.

*The per-week medians above only count editors who made at least 1 edit in that week. Because a high proportion of editors in both groups will have made no edits in each given week, the traditional per-week median for both groups would be 0 for all weeks.

» Teahouse guests communicate more with other editors

Teahouse guests also made roughly 75% more edits to the primary talk namespaces (User_talk, Article_talk and Wikipedia_talk) than other new editors. This suggests that Teahouse guests participate more in talk page discussions and may be more involved in community processes than new editors who chose not to not participate.