User talk:LilaTretikov (WMF)/Archive 12

Please participate in the anti-harassment consultation

Please take a part in a discussion on harassment. Harassment effects all of Wikipedians, but it may have a disproportionate affect on some of our most underrepresented voices - for example those of women and editors from developing countries. Help us find better ways to support them.

Could you please perhaps summarize the recent steps the Wikimedia Foundation has undertaken for action on this topic? I believe I saw something about a staff person being assigned in part to this area. It might help to give an overview. That could help people to avoid suggesting things which are already in process. Or alternately, possibly provide some approaches that can be built upon. -- Seth Finkelstein (talk) 11:39, 18 November 2015 (UTC)
Seth Finkelstein, it is our team, Community Advocacy, that has taken on this work. It has been ongoing since the summer. The initial stages were working with community members and academics in compiling research on the topic, and making it accessible - which can be seen here (this work is ongoing). We also solicited input from the community at Wikimania, and started an informal working group of volunteers and staff to identify areas where work could be done. One of the needs that came out of those discussions was better data on how the issue affects Wikimedians specifically. This lead to the Harassment survey, which just closed on Monday. We hope to have the data from that compiled and presented by late January. Another need was the importance of consulting with the community on specific problems and desired solutions, which we've started with the current consultation. This is where we are now, but much work still lies ahead. Patrick Earley (WMF) (talk) 21:14, 18 November 2015 (UTC)
Patrick, I just want to say thank you for doing this. It's much needed, and I really hope we're able to build on it. It's very encouraging to see this amount of work going into it. Sarah talk 22:47, 18 November 2015 (UTC)

Visual editor for wikitalk

It seems that nearly all the benefits of Flow could be implemented with a visual editor for wikitalk pages. Have a "new topic" button at the top that auto-generates the heading. Have a "new post" button on each topic that auto-generates a post and signature at the bottom of the topic. Have a "reply" button for each post, which the software code use a magic word like __REPLYBUTTON__ to mark where it belongs. Put each topic on a separate page to eliminate the scourge of broken archive links. It's all doable in my view. Oiyarbepsy (talk) 05:15, 2 November 2015 (UTC)

  • I'm boldly moving this to its own high-level section because the "Flow update -- action requested (arbitrary section break)" section on this talk page is rather difficult to follow, and I've wondered the same thing: would it make sense to enable VisualEditor on talk pages? I'm thinking that this would be a change that could be adapted into the existing community workflows with a minimum of disruption. (Whether Flow is a good or bad disruptive change, I'm not sure, but my hunch is that allowing Flow and wikimarkup sections to exist on the same page would be preferable to having talk pages be exclusively Flow or wikimarkup. And I agree that human-unintelligible strings of text for Flow topic links like could use some design thinking.) --Pine 19:57, 3 November 2015 (UTC)
Talk pages should be wiki pages, with exactly the same interface and capabilities as mainspace pages. Just as the long-term health of the wikis requires wiki markup accessed directly (not hidden via WYSIWYG), it requires the same of talk pages. I note it should be possible to achieve buttons such as described above via a carefully crafted minimal tweak to wiki markup (cf. yonder). --Pi zero (talk) 00:41, 4 November 2015 (UTC)
What about personal talk pages? Are there other places on-wiki where forums-like functionality is a better fit? LilaTretikov (WMF) (talk) 07:56, 18 November 2015 (UTC)
I am currently "testing" Flow on my Wikidata talkpage. I suspect it will be several years before I'd be willing to consider deploying it to my dewiki talkpage. I've done peer reviews there in the past, currently I am using it as a notepad. --HHill (talk) 09:02, 18 November 2015 (UTC)
Lila, tbh the question of whether forums-like functionality "is a better fit" in some contexts is the wrong question. On one hand, wiki markup can be enhanced — made to be even more what it already is, rather than less so — in ways that allow it to behave in a more forum-like way if that's what one wants it to do (and if one actually does the enhancing well); so the idea that one would have any legitimate motive to depart from the wiki markup is an illusion. While on the other hand, the most effective way for wikis to be is truly uniform — that is, everything is wiki markup. There is nothing other than pure uniformity that has the property of being purely uniform. It can't be counterfeited. You can't, for example, expect to have some other "structured data" system underneath and then create an interface that makes it look like wiki markup; you'd be forever playing a losing game of catch-up trying to improve the illusion created by the interface, for no reason because you could have actually had wiki markup as the ground representation.

This is closely related to a phenomenon in programming-language design (remember, that's my particular area of expertise) that is widely recognized but not as well understood as it might be. If you're at all familiar with various programming languages, there's a famous quote about APL and Lisp, roughly, "APL is like a diamond; it has a beautiful crystalline structure, but if you add anything to it, even another diamond, what you get is an ugly kludge. Lisp is like a ball of mud; you can add any amount of mud to it, and it still looks like a ball of mud." Lisp enthusiasts like that saying, which suggests there must be something to it since it sounds as if it ought to be somehow insulting to Lisp. Lisp is an immensely extensible programming language because of its mud-like nature. The key thing to understand about Lisp — and about wikis, which is why I bring it up — is that the lack of structure is at the very heart of its success. That's because an ordinary programmer, when working with a typical "modern" programming language, is intensely engaged in a detailed conversation with the computer (more precisely, with the programming platform software); and by trying to "think like a computer" the programmer loses the advantages of doing what human beings are best at, namely interacting with other human beings. Lisp has the programmer not fussing with a complicated type system. There is (to a first approximation) just one data structure in Lisp, that has to be used for everything, and the very fact that everything uses that one structure is a major asset because it turns the programmer away from fiddling with types. That strategy isn't intuitively obvious: you might think that giving a language more data types would be more flexible, or at the very least that giving a language the ability to declare new structured data types would be more flexible, but in practice those language features just draw the programmer down into more of a detailed conversation with the computer. And wikis are the apotheosis of this mud-oriented strategy, providing a single simple, very sparse syntax for specifying extremely unstructured data and strongly encouraging conversation primarily with other human beings. Just as Lisp would be worse off with more rigid data structures, wikis would also be worse off with more rigid data structures. --Pi zero (talk) 16:07, 18 November 2015 (UTC)

I'm fine with not everything being a wiki page, but at the least it needs to be plain text, and not hide its workings behind invisible computer code. I'm fine with Lua and Javascript and the like since anyone can see the source and how it's put together. Flow hides all that stuff. Some flow-like system that can be viewed as plaintext would be okay for me, since if would still be accessible to bots and scripts and the like, without needing a fancy API. And it could be converted to some other form by a script created by ordinary users without relying on WMF developers. Oiyarbepsy (talk) 04:51, 10 November 2015 (UTC)

Talked to Risker about this topic today too and will trial balloon this with James Forrester (have you chatted with him directly). Definitely worth exploring. LilaTretikov (WMF) (talk) 05:21, 18 November 2015 (UTC)

Annual Plan

I want to acknowledge an issue we had with the last annual planning cycle. During the 2015-16 planning we had internal delays that resulted in a shortened community review cycle. While this did not affect reliability of the plan, it is always beneficial to have more community feedback on our priorities. We are planning to correct this issue for the 2016-17 year and provide a 30 day community review. We are also looking at other changes to improve this process even further. We welcome your suggestions.

LilaTretikov (WMF) (talk) 20:58, 22 November 2015 (UTC)

One thing I would suggest is giving some insight into how the plan is used within the day by day work at the WMF. Not everybody in our community might understand the value and the function of an annual plan, so it might be helpful for the discussion if you could share some thoughts on the impact the plan has for your (=WMF) work.--Schreibvieh (talk) 13:26, 23 November 2015 (UTC)
There are two main components we track in the annual plan: financials (revenue/spend/reserve) that are monitored by the finance and advancement teams on monthly basis and team goals, anchored in the interim "call to action" strategy. These goals may change throughout the year as we learn and may need to make changes each quarter. So far you can see some of the goals already completed or in process and they can be found in the quarterly review reports. LilaTretikov (WMF) (talk) 16:46, 23 November 2015 (UTC)

Some accounting changes in FY 2015-16:
Completed this year to set up for financial controls:
  1. Implemented KPIs across the organizations.
  2. Implemented quarterly goals and reviews across organization.
  3. Reduced book close to 15 days.
  4. Catalogued projects to set up project-based accounting.
  5. Created business cases to evaluate cost/benefit analysis as an evaluation tool for new projects.
  6. Accounting/analytics software updates.

Here is what is currently considered the rest of the fiscal year:
  1. 3 year forward revenue/spend forecast (may slip due to staffing constraints).
  2. A consultation with community about strategic goals.
  3. A 30 day review period for the annual plan.
  4. More detailed annual plan, project based accounting where possible. Impact goals.
  5. Gap analysis of the annual plan vs. FDC.
  6. Wikidata integration into the annual plan.

Here is what is under advisement:
  1. 3rd party review of the annual plan.
  2. FDC process alignment.

LilaTretikov (talk) 00:07, 25 November 2015 (UTC)

Happy Thanksgiving

Happy Thanksgiving, Lila.

A couple of videos which are appropriate for the occasion and which I hope you enjoy (you may have seen them already):

Have a good holiday, --Pine 19:38, 26 November 2015 (UTC)

Thank you, Pine. These are wonderful and are good as repeat views as well. LilaTretikov (WMF) (talk) 17:53, 28 November 2015 (UTC)

Lean thinking

Hi Lila, I've been thinking for awhile about a paradox. I hear from WMF staff with some frequency that individually they feel buried in work, yet at the same time WMF's staff growth has far outpaced the (negative) growth in contributors since 2007. Trying to unpack this further, I'm wondering if there is low-value-added and no-value-added work that WMF is doing that could be cut so that resources could be refocused on places where WMF is well positioned to add value. I'm sure that you're familiar with Lean thinking, which I am thinking is compatible with your interest in using surveys and other tools to focus WMF's work. It's also compatible with your background in design thinking, and the Evaluation team's interest in learning patterns.

Awhile back, Sue Gardner had an initiative that she called "Narrowing focus". While I'm skeptical that the initiative did much good at the time, I think that a combination of the tools that I mention that I mention above could yield some benefits in the here and now. Hopefully this would reduce the low-value-added work while increasing impact, ideally at zero net cost. I'm wondering if the Team Practices group would have the ability to seed Lean throughout WMF, such that instead of the top-down approach that's so common, there can be some ground-up improvements.

The downside of this approach is that there is some up-front cost. Getting leaders trained in Lean has cost. Investing in rethinking and redesigning processes has cost. Staff need to feel that they can step away from doing urgent tasks so that they can do the training and the redesign work. First-level supervisors need to feel that they can let subordinates speak up without threatening the authority of the supervisors or making the supervisors look bad in front of 2nd-level supervisors. Lean takes some up-front cost and some courage.

I'm wondering if you would be interested in taking an incremental approach, piloting Lean in a couple of teams and getting a few leaders trained (perhaps in combination with use of surveys and other tools to help them to focus their goals), see if the system works for those teams, and then consider making a larger investment with the long-term goal of zero net cost while improving impact (and, hopefully, employee morale).

Of the goals, improving impact is the most important of the group. If there is some incremental cost that yields high ROI, then the investment may well be worth doing. However, WMF's budget should be *far* more transparent than it is, so before increasing WMF's top-line spending I would also like to see major improvements in budget transparency. Reprioritizations and narrowing focus, small-scale pilot implementations of Lean, and large-scale improvements in budget transparency can happen in parallel. If the pilots are successful and after the budget is far more transparent, larger-scale rollouts of Lean that may include some incremental topline costs can be considered. Hopefully, with these changes, WMF and the community will both become healthier in the long run.

Comments? --Pine 18:03, 6 December 2015 (UTC)

Hi Pine, if you caught last metrics meeting you may have heard me say that we need to focus our efforts better. While we have a lot of opportunities and possibilities, and while the WMF grew significantly, we are still a small organization for the size of our reader and editor community. I'd like to see more focus in both of our strategic and annual plan this coming year. That said, we will always drive to continually increase the impact of our work.
With respect to training our teams on Lean method, this is something we need to discuss internally. Our staff brought up the importance of management training and a big part of it is learning to manage priorities effectively. So it is possible that some aspect may figure into those programs. And as we anything new, a "pilot" approach would be an appropriate way to start. LilaTretikov (WMF) (talk) 21:35, 6 December 2015 (UTC)
OK, thank you. I just came across this article and think that it might be of interest, both in terms of how the Wikipedia community works and in terms of blue-sky thinking about WMF. --Pine 21:45, 7 December 2015 (UTC)

issues with Russian Wikipedia and government interference

Privet Lila! I just wanted to make sure you saw this: Russian Wikipedia Suspends Editor Who Cut Deal With Authorities, Russian media watchdog bans four Wikipedia articles on drugs. Does the WMF have any kind of opinion about this government interference? Thank you. Wikimandia (talk) 03:07, 9 December 2015 (UTC)

Hello, @Wikimandia:. Thanks for the question. I asked WMF legal, and the WMF opinion is this:

Given the communities’ rigorous editorial review, any outside interference (including from governments) is unhelpful for sharing free knowledge. The Wikimedia projects are neutral platforms where you should be able to speak and share freely. We understand that the Russian Wikipedia community is facing a tremendous challenge, but we trust that the community will be able to find the best path based on the policies set by the Russian Wikipedia community.

Best, Patrick Earley (WMF) (talk) 22:26, 14 December 2015 (UTC)

Survey fatigue

Hi Lila, there have been a lot of WMF surveys and consultations (in both the technical and social domains) in the past couple of months. Another one launched today for IEG, and an additional survey is coming for Wikimania. Much as I appreciate WMF consulting the community, I am concerned about the sheer number of surveys leading to survey fatigue. Can WMF work on consolidating surveys, targeting surveys using sampling, calendaring surveys to have fewer of them within narrow time ranges, and/or using other tools like talk pages and Phabricator tasks instead of surveys? Thanks, --Pine 00:03, 5 December 2015 (UTC)

Pine thanks for the suggestion, I will discuss with my team. I believe the surveys are meant to be targeted and small (that is why there are a lot of them vs. a few long ones). Surveys are really effective for prioritizing work, much more definitive than talk pages, make it easier to reach non-English communities and usually less time consuming for the volunteers. Just want to explain the the reasoning behind them.LilaTretikov (WMF) (talk) 01:02, 5 December 2015 (UTC)
Thank you. I understand that there are tradeoffs involved. For some of us on mailing lists, we get lots of survey requests, while I wonder if some of us who subscribe to few or no mailing lists are missing out on surveys. I think that use of sampling and targeting techniques might be helpful. Have a nice weekend, --Pine 01:26, 5 December 2015 (UTC)
Mailing lists are inside jobs, they don't reach normal editors. Meta here is just a wee bit better, for real input you have to leave these in-groups and go to the places, where the action is happening, the projects. A survey without a message in at least the top 20 projects in the right language and the right place(s) there is no real survey. Perhaps even make them there, so that the users don't have to leave their home environment. Shouldn't be that problem now, you have grown that huge and have enormous amounts of money to use for such worthy purposes. This should nearly always be bottom->up projects, top-down should be the tiny exception. This is after all a community project, nit a centralized one. The central entity is just for the convenience of the communities, it's their servant. Grüße vom Sänger ♫(Reden) 09:40, 5 December 2015 (UTC)
Hi Sänger ♫ - thanks for bringing up these issues - I've grouped them in my answers below:
* Using mailing lists for surveys: As one best practice that I'm trying to promote as survey specialist, is the use of sending out surveys to a random sample of talk pages, rather than always relying on banners or mailing lists. This is not the only or the best solution in all cases, but its a great one that has been working. As with any project, how you reach your users depends heavily on what the goal is. For example, if your goal is to learn how much people are learning from a mailing list, then a mailing list would be the place to post a survey!
* Surveys & projects (& languages): Not all surveys should be done across many languages; some surveys must align to specific goals. I agree with you - as a global organization we should be much better at getting things into many languages as possible. However, as an organization, we are learning to get better at translating. It's quite the process to translate and get a survey together in the right form. One example of a survey that was done across many languages and using talk page messages in the recent community tech satisfaction poll. Another is the recent Research:Harassment_survey_2015
<squeeze> As long as you don't use the highly selective meta, and only use enWP in less then half of the cases, just because English is such a convenient language for you, it's fine. But if you do too many of these surveys in meta or the lazy solution enWP, you got no corre3ct input. You should at least use all of the biggest projects (big in number of editors, not bot-generated articles). Grüße vom Sänger ♫(Reden) 05:35, 8 December 2015 (UTC)
* Decentralized/bottom-up: I am here to support surveys in the community, based on what your needs and your goals. You can visit the Survey Support Desk to find resources to help you plan out your next survey. While its still a bit a skeleton, I'm working with teams across the foundation to share how the do a survey the right way on the projects. Of course, you are free to reach out to me on my talk page if you'd like to do a survey, or if you have other questions survey-related. Thanks so much! --EGalvez (WMF) (talk) 22:23, 7 December 2015 (UTC)
Pine: I wonder if some of us who subscribe to few or no mailing lists are missing out on surveys - What surveys?? Alsee (talk) 14:40, 5 December 2015 (UTC)
  • Alsee thank you for further demonstrating my point about sampling and targeting. Some of us get loads of surveys while others get few, or perhaps in your case, none. --Pine 18:22, 5 December 2015 (UTC)
Hi Pine - curious to know what you mean by "loads of surveys"? How many have you received? Also, I'm hearing you say that its not just about surveys, but consultations and other requests as well. Do they all come from the WMF? About how many come from other places? In my role, I'm loosely thinking about how to best coordinate events like this across the movement. Its a very difficult thing to do, but still curious about ideas you might have. Feel free to talk more about survey on my talk page. Thanks! --EGalvez (WMF) (talk) 22:23, 7 December 2015 (UTC)
  • @EGalvez (WMF): 100% are from WMF. I am loosely including "consultations" in scope when I say surveys. In principle I appreciate that WMF is consulting the community about lots of different subjects; in practice the methodology could use some further tinkering. Thanks for working on this, --Pine 22:39, 7 December 2015 (UTC)
These are important points Sänger ♫ and Alsee -- I will take this back to our team. I understand the problem: if the target group is already pre-filtered based on the channel this can make the survey data biased. We are trying to do a lot of learning right now to make sure we focus on most important projects; the teams have the right intent for improvement. LilaTretikov (WMF) (talk) 18:39, 5 December 2015 (UTC)
I just received a survey banner on mediawiki. The survey was irrelevant to me, but at least now I have a clue what's going on. Suggestion: Banner text for surveys aimed at a special group should perhaps begin with "who" rather than "what". That is a much easier way to determine relevance. Alsee (talk) 17:00, 7 December 2015 (UTC)
Hi Alsee - what made you think the survey was not for you? Typically banner surveys want to have the most people take them as possible. I am thinking that your idea about clearly stating who should open the survey may help to show this better in the future. If you want to talk more about surveys, happy to discuss on my talk page --EGalvez (WMF) (talk) 22:23, 7 December 2015 (UTC)
EGalvez (WMF) the survey was for 3rd party re-users of Wikipedia content. I looked over the questions, there weren't any that I could remotely give any sort of answer. Alsee (talk) 04:22, 11 December 2015 (UTC)
Probably this one: Survey: Understanding third-party use of Wikipedia content. It's not on the Calendar of Consultations & Surveys by EGalvez (WMF). Also it is hosted on, which i think is bad. My experience see T111403 Set up Google Form to collect user feedback on Wikidata Query Service:
  • "There is some community push for not using google forms."
(indeed, @wikidata "google? You can't be serious?! Why did WMF investigate user feedback surveys/Community input channels this year - so you can completely ignore this effort and use a data-hoarding google service?! Unusable tinyurl and google-forms??")
  • "Google Forms require minimal maintenance/overhead for us whilst also making the feedback loop easier for non-Wikimedians, so they're the best option right now. If Wikimedians don't want to use the Google Form for whatever reason, that's absolutely fine. They can email the Discovery list, or me directly."
  • "Yeah it is not a deal-breaker from my side. Just be aware that this might give you push-back."
  • "Thanks for letting me know. :-)
  • "This was done using a wiki page in the end, for legal reasons."
So, never mind community push-back, the minimal maintenance for devs is what counts, they decide. Let them eat cake! Unless "legal reasons" intervene... --Atlasowa (talk) 00:26, 13 December 2015 (UTC)
Google? WMF is using Google? The data mining, privacy ignoring, company decent people avoid as much as possible? While there is the whole wikiverse to use and no need exists to go to such unreliable enterprises? If you really go to google, you definitely are not serious. Grüße vom Sänger ♫(Reden) 11:05, 13 December 2015 (UTC)
Thanks so much for calling this to my attention Atlasowa and Sänger ♫. I'm checking with the staff member running the mediawiki survey so they can add it to the survey calendar. We do use google from time to time. The foundation does not have reliable open-source survey software that we can use (we tried using Limesurvey a while back but there were severe security issues). Surveys are also very tricky to create, and other companies are experts at surveys which really helps in the quality of the results (like qualtrics, which we also use). Google works well for shorter surveys, but its not the best. We do try to avoid it where possible. We also make sure these services are vetted by the legal department, and all surveys go through legal review at the foundation. Surveys are an essential tool to help gather input from users and we just don't have good wiki tools at the moment. Thanks for the interest on this topic! feel free to ask anything else on my talk page.--EGalvez (WMF) (talk) 23:19, 16 December 2015 (UTC)

Do you have any opinion about the disgraced new board member?

The board has appointed a disgraced member to its ranks, that was involved in illegal acts against employees and thus is completely unsuitable as a board member. OK, the board is officially something like your boss, though in reality the community should be, but nevertheless: Do you have any opinion to offer about this disgusting appointment? Grüße vom Sänger ♫(Reden) 10:13, 17 January 2016 (UTC)

To make one thing clear: I'm not for ostracising Arnnon completely, he has just proven beyond any doubt that he is unsuitable for any leading position in any business, that claims to be even remotely ethical. He may still be employed as a rank-and-file employee, with lots of oversight, he's just completely discredited for any leading position because of his illegal and unethical deeds. Grüße vom Sänger ♫(Reden) 11:31, 17 January 2016 (UTC)

See en:Wikipedia:Copyright problems/2016 January 17. Multiple Copy vios by WMF. Respect!?! Funny that you on the bottom of the history page state: "Images are freely licensed with attribution." but don't mention any attribution itself. and by the way with cc-by-sa attribtion is not enought at all. maybe some seminar about basic Wikipedia knowledege for WMF? .oO ...Sicherlich Post 12:25, 17 January 2016 (UTC)

Dear Lila, I just wanted to bring to your attention that the site provides a very special birthday gift from WMF to the Wikipedians' community: The use of user-generated content without any attribution or mention, disregarding the licenses and the core principle of Wikipedia to cite the source where one takes other's content from. By a long research I managed to identify those (ab)used files and text:

I would appreciate to get to know your personal thoughts on this case and how to solve these issues in the fastest possible way (preferrably with an honest apology to the original contributors). Thank you. --.js[democracy needed] 12:39, 17 January 2016 (UTC)

Thank you @Sicherlich and .js:, this was already noticed by the team and is part of an update. We did not intend to use content without attritbution, this was my mistake. I appreciate your taking the time to let us know. heather walls (talk) 19:50, 17 January 2016 (UTC)
@Sicherlich and .js: Thanks for bringing this to our attention, the attributions can be found in full. heather walls (talk) 03:17, 18 January 2016 (UTC)
Thank you Heather, looks very good now :) --.js[democracy needed] 09:14, 18 January 2016 (UTC)
@Sicherlich and .js: Thank you guys for bringing this up. I've passed the issue on to the Legal Department to make sure we get some eyes on it. Kbrown (WMF) (talk) 16:43, 17 January 2016 (UTC)
Thanks for brining up and KBrown for making sure we address any issues. LilaTretikov (WMF) (talk) 18:46, 17 January 2016 (UTC)
Thank you Heather, Karen and Lila for your answers; I did not at all expect a single person being pushed to take full responsibility alone, really sorry for that, Heather! And I regard that neither justful nor as professional management.
Hello @.js:, I'm not sure what you mean, but I was not pushed to take responsibility. I was pointed toward these comments by someone who hasn't participated in the conversation. I felt it was important to accelerate the updates, and some of my colleagues (@Yuvipanda, Slaporte (WMF), and JSutherland (WMF):) were kind enough to help on our day off. Heather Walls (WMF) (talk) 00:56, 19 January 2016 (UTC)
@Lila could you please tell me if there is any procedure that ensures WMF staff and contractors get educated in the Wikipedians' core principles prior to any content publishing contributions they will have to perform?
(I'm asking that because since I started editing Wikipedia somewhen in 2005 I had to witness over the years very many instances of WMF staff dealing with Wikipedia & Wikimedia Commons contents without even having any basic understanding of the mandatory licencing requirements. I could provide links to found my claim, but I do not want to persons being spotlighted but address a general problem and I am sure that this must be very well known by the WMF managent and the Board of Trustees.) --.js[democracy needed] 09:14, 18 January 2016 (UTC)

FYI: Follow-up question from the metrics meeting today

See Talk:WMF Metrics and activities meetings/2016-01#Follow-up from live question re. engagement survey. It looks like Boryana is not on Meta (I couldn't find an account on any wiki for her actually). If you could help me make her aware of the follow-up question, I'd appreciate it. Thanks! --EpochFail (talk) 20:24, 14 January 2016 (UTC)

Thanks for the comment, I will ask to follow up. LilaTretikov (WMF) (talk) 22:56, 14 January 2016 (UTC)
  • I too am interested in this, although perhaps for slightly different reasons. This email from HR implies to me that the situation with staff turnover and morale is generally OK. Apparently that answer omitted data suggesting a high level of discontent among staff with WMF senior management. I don't appreciate being misled, and I would like to ask for an explanation of why important information which contradicts the general tone of the email that was sent to Wikimedia-l was completely omitted from that email. --Pine 20:07, 17 January 2016 (UTC)
  • Pine, I'm sorry if you felt misled. That wasn't my intention when I answered the turnover question. It was, of course, already on schedule to talk about the results of the engagement survey during metrics as HR had done in the past. In retrospect, I see that I might have mentioned that we were going to be sharing the survey results in about a week. EpochFail, I will defer to LilaTretikov (WMF) to address the Signpost article pertaining to the leadership development needs we have. — The preceding unsigned comment was added by 2600:1010:b02c:60b7:a46c:8c2d:d073:ce42 (talk) January 22, 2016 (UTC)
  • I've been pondering this statement, and regrettably I still have concerns. WMF would certainly know that Wikimedia-l and the Metrics Meeting are used by different audiences. Also, the delays in releasing this information and the general lack of transparency are problematic. Also, we are still waiting to hear comments from Lila about the staff surveys. My confidence is pretty shaken regarding the handling of this situation. I am taking this set of circumstances into account in considering what strategic courses of action to discuss with the community. --Pine 20:56, 26 January 2016 (UTC)

Employee communications

Hi Lila, I've heard rumors indicating that WMF staff are feeling intimidated into keeping silent in public about matters that bother them.

While a certain amount of organizational cohesion is necessary in WMF, the existence of workplace intimidation and threats of retaliation against staff who voice their opinions about the governance of the organization would be problematic, if these rumors are true (and I hope that they aren't).

  • We are collectively strong proponents of free speech in the public sphere, as demonstrated by the SOPA blackout and the ongoing lawsuit against the US Government.
  • We are an open-source educational movement, so if these practices are occurring, they would be misaligned with the nature of our work.
  • Many staff could easily find employment elsewhere, and have little reason to work for an organization where they do not feel valued or feel intimidated.
  • Turnover of high quality staff would be a problem. My understanding is that staff generally cite the Wikimedia mission and their enjoyment of working with intelligent peers as being among their reasons for choosing to work for WMF.

So, while I would expect staff to conduct themselves professionally and in a way that as aligned with the Wikimedia mission, I find these reports disturbing. I hope that you will comment on this situation and make clear what your expectations are for staff communications in public.

Thank you, --Pine 23:25, 28 January 2016 (UTC)

This is not mere rumor; there are multiple explicit public statements on this. -Pete F (talk) 02:41, 29 January 2016 (UTC)
Hi Pine, retaliation is strictly prohibited by WMF policy and will not be tolerated. A healthy discourse is important in any organization, especially within our movement which values free speech. We support feedback in our office. I ask anyone who believes they are experiencing retaliation to report it to myself, HR, or Legal. Allegations of retaliation will be investigated and we will take action.
What we are working on is improving our performance, conflict resolution and civil communication within the organization. We are making these initiatives based on what our own employees told us. I personally receive plenty of criticism from staff members and I try to integrate this criticism into my work as much as possible. I believe it is very valuable, especially when coming from a place of good faith. We do ask all to practice civility and etiquette and to avoid being disruptive. LilaTretikov (WMF) (talk) 19:43, 29 January 2016 (UTC)
Thank you. I hope that this is sufficient clarification for staff who seemed to be feeling like their jobs were threatened if they communicated certain varieties of their opinions in public. I realize that there is ambiguity about what is appropriate criticism and protected expression of personal opinions, and when the line is crossed into inappropriate commenary. I hope that staff who still have doubts in this area are able to clarify the situation with you, HR, or Legal in a manner that respects the autonomy and rights of all concerned, particularly when employees wish to communicate in public "off the clock" in their personal capacities. --Pine 00:51, 31 January 2016 (UTC)
I hate to seem like I am assuming bad faith here but I asked several people (6 if you need the actual number) and 2 that formerly work at the WMF and every one of the ones I asked said they feel/or felt intimidated at work. So, even if Lila herself feels that retaliation is strictly prohibited by WMF policy, then perhaps many of the other junior leadership feel differently. These sentiments are also vocal on virtually every message board out there that deals with employee ratings of their companies. Additionally, the WMF doesn't listen or seem to want to work with the majority of the volunteer community and most of those communities feel they can't trust the WMF and the WMF doesn't seem to care. So why would they care about the paid employees when they can just get another one and have people lining up to work there. Take a look at one for yourself:
  • [1] Glass Door reviews
Its easy to say that something is against policy but if you don't enforce it or ignore it when its convenient, which is the case with the volunteer communities admins and functionaries that the WMF chooses not to do anything about, then it doesn't really mean much to say its against policy. Reguyla (talk) 00:08, 1 February 2016 (UTC)
@Pine: I have taken great care to speak with civility during these months of conflict, particularly when I have spoken in public. I have also expressed my concerns about the potential for retaliation to my manager and to HR. I have been repeatedly assured that I have nothing to worry about due to the care I take with my words, but the specific standards that are being used to define "aggressive", "unprofessional", and "uncivil" are still unclear to me. I hear my colleagues' concerns and see some of them being censured for speaking in ways that I have found sharply critical but still fundamentally honest and civil, and I worry that someday I will be the one who is suddenly found to have stepped over lines which were previously invisible or unspoken. I fear that even making this reply with my volunteer account will be considered "unprofessional": it is both critical and public, and no clarification has been given yet on the question of what constitutes "professional" usage of our staff and volunteer accounts. Fhocutt (talk) 09:06, 1 February 2016 (UTC)
A few comments here in my volunteer capacity as a Wikimedian and as a software engineer who has been watching dysfunctional power dynamics in the larger tech and free software communities for years now:
  • I would not expect English Wikipedia's policies (en:WP:CIVIL) or guidelines (en:WP:ETIQ) to apply to WMF staff at large in a WMF context, as these policies are tailored to the specific needs and social dynamics of English Wikipedia. These projects have different community standards for interacting and contributing to the projects--for instance, types of communication that are normal on one project may be considered quite hostile on another. Any organization needs clear standards for behavior that fit its specific context, and I believe the WMF is no different from other Wikimedia communities in that regard.
  • I have heard quiet stories of retaliation, discrimination, harassment, and other forms of poor treatment from members of a number of tech companies and open source organizations. In most of these cases some combination of HR, Legal, and senior management (or its equivalent, perhaps the BDFL) have either been the source of that treatment or have acted as enablers, often in the name of minimizing liability to the larger organization, and usually in a way that causes severe damage to the targets' health and careers. I know that some of those organizations had policies against retaliation, and claimed to support constructive criticism and diverse points of view. So if a friend working at such a company told me that she had been given similar reassurances to the ones given above, I would tell her to be very, very careful--especially if she had been publicly critical in a way that might be perceived to cause embarrassment or liability to the company, regardless how polite, civil, or professional she had been in doing so. Please note--I am not in any way diminishing the good work done in HR and Legal at the WMF! My observations over the years, however, mean that I can't help being worried when I see warning signs or antipatterns, and--as I would advise that hypothetical friend to be--I am still not reassured. A clear, written policy would help allay these worries. Fhocutt (talk) 10:23, 1 February 2016 (UTC)
For further information about how harassment and intimidation are a part of daily life on the WMF projects and how the WMF tolerates it, all we need to do is look at the Harassment Survey 2015 Results report posted by the WMF. This survey outlines a lot of data points about how harassment and intimidation are carried out on the projects and its getting worse because the WMF doesn't care as long as the donation dollars keep coming in. Of particular interest to me was page 18, that outlines in detail what I have been saying for a couple years, that a lot of people in the communities feel harassed by admins misusing the tools. I personally have been the target of this multiple times including Abusive language, having someone contact my employer to try to get me fired after it was advocated by an Arbcom member and unjustified use of the admin tools for no other reason than an intimidation tactic. Several admins and arbitrators have been using me as a warning message to the community of what happens when you criticize abusive admin behavior for a couple years now and the WMF has done nothing about it because it doesn't care about me or any other non admin editor. This is the culture that the WMF has fostered within the projects by ignoring problematic admins and this is characteristic of how editors are treated. Even for long term, dedicated and established editors with an extremely high output and with hundreds of thousands of edits and a documented desire to see the projects improve, they still ignore it as not a problem because admins are virtually exempt from policy. That is why this harassment occurs, because its been allowed to go on for over a decade and "tell it to Arbcom" has become a mantra of the most abusive of them. Why would the WMF care about new editors or even employees? They are just expendable commodities that can be thrown away when no longer useful and I have seen nothing to show me or anyone else that the WMF has any desire or plan to change this. Reguyla (talk) 19:39, 1 February 2016 (UTC)

  • Fhocutt Thanks for the comments. In response:
  • In the community we have our own difficulty defining what exactly we mean by "civil", and enforcement of civility has been inconsistent. We continue to have a tension between the value of free speech with the virtue of hospitality (as Sumanah put it).
  • In my brief research into employment and 1st amendment laws that apply to this set of circumstances, my impression as a non-lawyer is that when an employee is on the job a private-sector employer has considerable discretion in what the employer allows the employee to say or do. However, best practice in most cases is to address underlying causes of employee discontent rather than invoking policies or the employer's rights to take action against an employee; I think this is especially true in WMF's case due to the organization's mission, its support of civil liberties, and that many of its employees can easily find employment elsewhere. When an employee is not on the job they have greater liberty in what they say and do; for example an employee's announcement in public that they dislike their boss may be protected under the 1st Amendment or under labor laws. An employee announcing in public that they dislike their boss may cause upset at work and I wouldn't recommend doing that on a regular basis, but as far as I can tell that speech is protected and retaliation for it would be prohibited. Perhaps Lila can ask Boryana and Geoff to clarify the situation and publish a policy (by "publish" I mean "post in public here on Meta as well as Office Wiki") that is consistent with the applicable laws. (I am sure that they will keep in mind that if these policies go too far or are applied inconsistently, this will create the risk of an employee lawsuit against WMF if something goes wrong, so it's in WMF's interest to write a legally sound policy and to apply it consistently.) --Pine 19:47, 1 February 2016 (UTC)

Lila, does the Wikimedia Foundation still maintain a list of individuals with whom employees are forbidden to communicate? If so, how many individuals are recorded on that list, and how are names added to the list, or removed (if ever)? - Thekohser (talk) 21:57, 31 January 2016 (UTC)

If such list exists, it's certainly not well enforced, as in my year's tenure in WMF I never heard about it and as far as I remember I was never told I'm forbidden to speak to anyone (I can be forgetful but I think I'd remember such thing). I've also never heard of this list being part of anybody else's onboarding procedure. Maybe I didn't pay attention though, it happens sometimes :) --Smalyshev (WMF) (talk) 20:40, 1 February 2016 (UTC)
I probably overstated it. I was once told that I am "one of exactly two people that the Wikimedia Foundation is never obliged to engage", or something very similar to that. I suppose "never obliged to engage" is not the same as "not permitted to engage". So, I withdraw my question, which wasn't likely to be answered by Lila, anyway. - Thekohser (talk) 21:16, 1 February 2016 (UTC)
Found it. [13:10 <@StevenW> Killiondude: There is a list of exactly two people that the WMF feels obliged never to engage with. ... Kohs and an IRL stalker that the office has had.] - Thekohser (talk) 22:28, 1 February 2016 (UTC)
There was also the following nugget in the Arbcom-l mailing list leaks: "On the press committee mailing list, we have decided (based on past, profoundly negative experiences) to treat Cade Metz as a special case. Specifically, nobody is to respond to any inquiries from him -- just forward them to Jay, and let Jay handle them. I think non-engagement is the best thing to practice in this case.". Note that's the press committee, not the Wikimedia Foundation as a whole. I suspect these designations are informal, rather than big "Wanted"-style posters on an office wall. I would expect anyone at high levels to profess ignorance, or at least lack of recall (Oh, people gripe all the time, I just can't remember such petty details - it's not important, and let's move on, lovingly and thoughtfully ...). -- Seth Finkelstein (talk) 04:19, 2 February 2016 (UTC)

Minutes of meetings

Why have the slides for WMF Metrics and activities meetings/Quarterly reviews/Discovery, January 2016 not been uploaded to Commons? Currently there are on Google's cloud service requiring one to log in to view them. This is contrary to usual practice, include the previous quarterly review for discovery, and will only make it look like you're hiding something (which, to be fair...) BethNaught (talk) 13:34, 1 February 2016 (UTC)

FYI: Follow-up question from the metrics meeting today

See Talk:WMF Metrics and activities meetings/2016-01#Follow-up from live question re. engagement survey. It looks like Boryana is not on Meta (I couldn't find an account on any wiki for her actually). If you could help me make her aware of the follow-up question, I'd appreciate it. Thanks! --EpochFail (talk) 20:24, 14 January 2016 (UTC)

Thanks for the comment, I will ask to follow up. LilaTretikov (WMF) (talk) 22:56, 14 January 2016 (UTC)
  • I too am interested in this, although perhaps for slightly different reasons. This email from HR implies to me that the situation with staff turnover and morale is generally OK. Apparently that answer omitted data suggesting a high level of discontent among staff with WMF senior management. I don't appreciate being misled, and I would like to ask for an explanation of why important information which contradicts the general tone of the email that was sent to Wikimedia-l was completely omitted from that email. --Pine 20:07, 17 January 2016 (UTC)
  • Pine, I'm sorry if you felt misled. That wasn't my intention when I answered the turnover question. It was, of course, already on schedule to talk about the results of the engagement survey during metrics as HR had done in the past. In retrospect, I see that I might have mentioned that we were going to be sharing the survey results in about a week. EpochFail, I will defer to LilaTretikov (WMF) to address the Signpost article pertaining to the leadership development needs we have. — The preceding unsigned comment was added by 2600:1010:b02c:60b7:a46c:8c2d:d073:ce42 (talk) January 22, 2016 (UTC)
  • I've been pondering this statement, and regrettably I still have concerns. WMF would certainly know that Wikimedia-l and the Metrics Meeting are used by different audiences. Also, the delays in releasing this information and the general lack of transparency are problematic. Also, we are still waiting to hear comments from Lila about the staff surveys. My confidence is pretty shaken regarding the handling of this situation. I am taking this set of circumstances into account in considering what strategic courses of action to discuss with the community. --Pine 20:56, 26 January 2016 (UTC)
  • Lila, these comments are from a few weeks ago. Will you respond, please? --Pine 05:27, 17 February 2016 (UTC)

Innovation III

Dear Lila,

Last June I posted a message to you about Innovation [2] but unfortunately it aged off before you had a chance to answer it [3]. I raised the subject again [4] and again you allowed it to age off without answering [5]. That was disappointing. I started a page here on Meta for Innovation and one of your staff was kind enough to spare the time to post some comments [6]. Since then nothing has happened, as far as I can tell, to engage with the community on this subject at a strategic level. That's more than disappointing, it is disturbing.

Now, however, I discover that at the very time that you were ignoring my attempts to start a discussion between the WMF and the volunteer community, you were "ideating" your "Knowledge Engine" proposal. Nothing about this was shared with the community at that time. This sends me a clear, unambiguous and deeply insulting message to me -- my help, my input, my ideas are of such little value to you that you ignore them completely -- multiplied across the whole community, this is a disastrous attitude. There are people in the community who know more than you do about the topic you initiated (I do not claim to be one of them), the community collectively can engage far more widely and effectively with cutting-edge researchers and innovators in knowledge management and discovery than you can as an individual, and if you and the Board do not make use of that huge wealth of engagement, experience and expertise then you are failing the movement that, to be blunt, pays your salary. This is not to denigrate you or your staff, it is a matter of weight of numbers. If you can use the resources of the 100,000-strong volunteer community, you will surely do better than you can with 280 staff. We are trying to help you and yet you persistently rebuff those efforts: why?

Please consider what steps you can take urgently and effectively to reverse the growing gap between the volunteer community, the Board and the staff. I ask not because I am offended personally (although I am) but because it is necessary for the sake of the project as a whole.

Rogol Domedonfors (talk) 11:51, 7 February 2016 (UTC)

PS: Should you find time to respond to this, as I very much hope you will, I hope you will frame your response in terms that will be maximally helpful to the community going forward. For example, it would be of benefit for you to state who your strategic lead for Innovation is, and ask that person to publish their strategy for innovation within the WMF and across the community; who you have and wish to have as your strategic partners; a list of your major innovation projects; what activities are under way or planned to develop innovation; oh, and of course, I am also asking you to publish the details of your Knowledge Engine project, as proposed in your grant application to the Knight Foundation. Rogol Domedonfors (talk) 07:34, 8 February 2016 (UTC)

PPS: The Knight grant application states that the Wikimedia Foundation has a proven track record of leadership and innovation. Perhaps your strategic lead for innovation could take a moment when updating the Innovation page to include a celebration of the successes that enabled you to make that claim. Rogol Domedonfors (talk) 21:43, 11 February 2016 (UTC)

Dear Rogol, I am sorry I missed your previous posts. I have read through your Meta Innovation page and it is indeed full of solid analysis and ideas. I hear your message and I am working to make sure we are sharing and soliciting input on ideas early and often. You are correct, during my early days, I lacked comprehensive understanding of how to best work in public and to collaborate effectively in our community, especially during the early phases of idea development. Having a conversation with 100,000 people is still a challenge, but we have been working on it through different methods recently.
So going forward our early ideas will be published. You probably see that they are nowadays. We have portals for many teams and we are making plans to connect them together.
We do not have an "innovation lead". On the product side, each team does some innovation within their areas. Our senior team works to guide our internal work, but specific decisions usually rest with the team. We are wrapping up the strategy consultation, which is giving us signals about what is important to the community and directions to follow. Coming out of the strategy consultation we will be making choices on specific strategies as the senior team. The board plays a big part in guiding this. The annual plan will reflect these decisions.
As you know we made the grant public (although it is not our common practice due to typical donor privacy -- we have to ask each donor if we can release their grant individually).
We seek input to our innovation from many areas. Community is definetely a big part of this and I am learning on how to best gather the feedback and ideas. We have done some great collaboration last year with many ideas coming directly from the community. To see more on what we have done in product, you can read here. Other partners are like-minded organizations such as our own chapters but also other organizations such as the Internet Archive or Creative Commons. We also consult with donors and experts in the filed.
Our challenge with any type of innovation is focus. While we can do many things, and we should definetely do everything we are doing better, we must choose what we do very carefully, because our resources are limited. We are funded by donors, so we have to focus and measure impact, in other words to bring more knowledge to more people. LilaTretikov (WMF) (talk) 09:01, 17 February 2016 (UTC)
Lila, Thank you for your full response. If you do not have a strategic lead for innovation then I suggest that you should appoint one, and task them with building up a coherent internal innovation programme, embedding innovation into all your activities, celebrating your successes and with building partnerships with other complementary groups. Of course the volunteer community will be your key resource here. This is a demanding job and you need the right person in it. Rogol Domedonfors (talk) 22:03, 18 February 2016 (UTC)

"I would hope that for staff, the answer to this question is clear."

Hello, I am afraid the answer to this question is not clear to me. The question being "Why should the community and staff support this decision of our board and leadership?". Could you please clarify it ? Thank you, Cenarium (talk) 16:52, 11 February 2016 (UTC)

+1. Doing one's job and supporting an idea are not the same thing. BethNaught (talk) 13:46, 18 February 2016 (UTC)

Why we’ve changed.

Copied to Lila Tretikov's statement on Why we've changed

I want to address some of the many questions that are coming up in this forum. From the general to the very concrete, they all touch on the fact that many things about the WMF have been changing. We are in the thick of transformation, and you all have the right to know more about how and why this is occurring. This is not a statement of strategy, which will come out of the community consultation next week. This is the ED’s perspective.
After 15 years since the birth of Wikipedia, the WMF needs to rethink itself to ensure our editor work expands into the next decade. Recently we kicked-off some initiatives to this end, including aligning community support functions, focus on mobile and innovative technology, seeding the Wikimedia Endowment, re-organizing our internal structure, exploring partnerships and focusing on the most critical aspects of our mission: community and technology. We started this transformation, but as we move forward we are facing a crisis that is rooted in our choice of direction.
The choice in front the WMF is that of our core identity. Our mission can be served in many ways, but we cannot do them all. We could either fully focus on building our content and educational programs. Or we can get great at technology as the force multiplier for our movement. I believe the the former belongs to our volunteers and affiliates and that the role of the WMF is in providing global support and coordination of this work. I believe in -- and the board hired me to -- focus on the latter. To transform our organization into a high-tech NGO, focused on the needs of our editors and readers and rapidly moving to update our aged technology to support those needs. To this end we have made many significant changes. But the challenge in front of us is hard to underestimate: technology moves faster than any other field and meeting expectations of editors and readers will require undistracted focus.
What changed?
When Jimmy started Wikipedia, the early editors took a century-old encyclopedia page and allowed anyone to create or edit its content. At the time when creating knowledge was still limited to the chosen few, openly collaborating online gave us power to create and update knowledge at a much faster rate than anyone else. This was our innovation.
As we matured, we encountered two fundamental, existential challenges. One is of our own doing: driving away those who would otherwise join our mission through complex policies, confusing user experiences, and a caustic community culture. The other is external and is emerging from our own value of freely licensed content: Many companies copy our knowledge into their own databases and present it inside their interfaces. While this supports wider dissemination, it also separates our readers from our community. Wikipedia is more than the raw content, repurposed by anyone as they like. It is a platform for knowledge and learning, but if we don't meet the needs of users, we will lose them and ultimately fail in our mission.
Meanwhile, in the last 15 years revolutionary changes have taken hold. The rate of knowledge creation around the world is unprecedented and is increasing exponentially. User interfaces are becoming more adaptive to how users learn. This means we have a huge opportunity to accelerate human understanding. But to do so requires some significant change in technology and community interaction.
So let’s begin with technology: Many at the WMF and in our community believe that we should not be a high-tech organization. I believe we should. With over half of our staff fully committed to delivering product and technology, it is already our primary vehicle for impacting our mission and our community. In fact we constantly see additional technology needs emerging from our Community department to help amplify theirs and our community work.
What do we need to do in light of the changes I described above? We need to focus on increasing productivity of our editors and bringing more readers to Wikipedia (directly on mobile, and from 3rd party reusers back to our sites).
When we started, the open knowledge on Wikipedia was a large piece of the internet. Today, we have an opportunity to be the door into the whole ecosystem of open knowledge by:
  1. scaling knowledge (by building smart editing tools that structurally connect open sources)
  2. expanding the entry point to knowledge (by improving our search portal)
There are many ways to alleviate the manual burdens of compiling and maintaining knowledge currently taken on by our editing community, while quickly expanding new editing. We made significant strides this year with our first steps to leverage artificial intelligence to remove grunt work from editing. But that is just a start. Connecting sources through structured data would go much further and allow our editors to easily choose the best media for their article and for our readers to recieve content at their depth of understanding or language comprehension.
Wikipedia is the trusted place where people learn. Early indicators show that if we choose to improve the search function more people will use our site. We are seeing early results in use of Wikipedia in our A/B testing of search , but we have a long way to go. We want people to come directly to our sites -- and be known as the destination for learning -- so that eventually we can bring our readers into our editing community. And without community support none of this will be remotely possible.
Which brings me to the community. Over time the WMF has grown, with an opportunity of becoming a complementary, mutually empowering partner with the community. We need each other and we share one focus: humanity. Reaching and sharing with people across the world is our common goal.
In the past year we managed -- for the first time since 2007 -- to finally stem the editor decline. But that will not be enough. We need to find ways to re-open and embrace new members instead of the hazing we conduct at least in some parts of the site today. We must treat each other with kindness and respect. Technology is not the main reasons for rampant new editor attrition. It is how we talk to each other that makes all the difference.
Without tackling these issues we artificially limit our growth and scalability. And we will continue to reject those whose ideas are new or different, the most vulnerable members of our community. In this, the Gender Gap is the “canary in the coal mine”. Women are the first to leave contentious and aggressive environments and are less likely to remain when they encounter it. They are less likely to run in elections because of rude and aggressive treatment. Yet in editor surveys and in our latest strategy consultation, Gender Gap has been considered a low priority. I disagree.
Over the past two years I have actively pushed funding to improve anti-harassment, child protection and safety programs; work in these areas is ongoing. We are actively exploring some tangible approaches that -- I hope -- will turn into concrete outcomes. In the latest research this year the number of female editors shown some growth.
What does this mean for the WMF?
In the past 18 months -- and thanks to hard work of the people at the WMF and our community supporters -- we have made significant structural changes. We have organized around two core areas: technology and community. We have made changes with an eye on improving our relationships between the volunteer community, the chapters and the WMF, including the creation of structures that should vastly improve the WMF's responsiveness to volunteers. We began adopting best industry practices in the organization, such as setting and measuring goals and KPIs. We’ve given managers a lot of responsibilities and demanded results. We’ve asked for adjustment in attitude towards work, our responsibilities and professional relationships. We prioritised impact and performance so that we can provide more value to our communities and the world.
This has not been easy.
In practice this means I demanded that we set standards for staff communication with our community to be professional and respectful. It meant transitioning people, shutting down pet projects, promoting some but not others, demanding goals and results to get funding. This level of change is necessary to set up our organization to address the challenges of the next decade.
All of this means stepping away from our comfort zones to create capacity for building programs and technologies that will support us in the future. It is a demanding and difficult task to perform an organizational change at this scale and speed.
I believe that in order to successfully serve our community and humanity, the WMF has deliver best-of class technology and professional support for community. This will ensure we are delivering significant impact to volunteer editors and opening avenues for new types of contributions. This requires that we choose the route of technical excellence for the WMF with support and encouragement from our community partners. Without this empowerment, the WMF will not succeed.
The world is not standing still. It will not wait for us to finish our internal battles and struggles. Time is our most precious commodity.

LilaTretikov (WMF) (talk) 00:19, 22 February 2016 (UTC)

Thanks for this Lila. You didn't ask for responses, but hey this is a dialogic place. In any case it was good to hear this kind of high level vision thing from you.
You have made some things very clear. Your vision is "To transform our organization into a high-tech NGO, focused on the needs of our editors and readers and rapidly moving to update our aged technology to support those needs" and you make it clear that the Board backs this vision. You explain the turmoil among staff as being due to starting to make the organizational changes that are required to achieve that vision. You explain where the Discovery project is going somewhat - making content on WM/WP sites, as well as other sources of open content, available through Wikipedia, and about automation to bear on improving/accelerating that process. That is clear.
With regard to your comments about the editing community; to be frank I don't think you understand two key things about being an editor, and about the editing community. The first is what it is like to build and maintain high-quality content in an environment when "everyone can edit" , and the second is the fact that the very complexity of the content and behavioral policies and guidelines, which the community itself has put in place over the years to guide itself, is a sign of the maturity of the project in grappling with the challenge of building and maintaining high quality content in a community where everyone can edit. The environment created by the policies and guidelines can be beautiful. They save this place from being a wild west. Talking about the "everyone can edit" thing a bit, you can roughly classify editors on four axes. 1) competence in knowledge: loons to world-class experts; 2) reason for being here: to serve some outside interest (a cause, a company, a client) or to serve the mission of WP; 3) competence in the policies and guidelines (newbies to the most clueful - which means understanding the heart of things); and 4) self-awareness/willingness to learn (from the most closed/arrogant, to the most open and aware) We get people all over that four-dimensional space (and of course all the variations within any one of those axes, and all the variations of good and bad character tendencies). And we get people who are self-deluded across all of them. You really deal with humanity in its all its glory and horribleness here. And it makes it very, very hard to deal with things like civility or, to the extent that incivility is what drives the gender gap, the gender gap. A lot of smart and well-intentioned people have tried for a long time. (I'm not saying we shouldn't try, but I don't see how this is in the domain of WMF can do, and I don't see that you understand how hard it actually is to address)
But I don't get a sense that you understand these things and I urge you to be more careful in making claims about the editing community until you do. And I hope you start editing seriously if you haven't already, to understand what content creation and maintenance is actually like. Unless you are planning on throwing manual content creation out the window, your chances of success are low if you do not understand what its like to be an editor - you won't be able to communicate well to us, and you won't understand how technology can help, hinder, or complement what we actually do. I hope that makes some sense to you. Happy to discuss any of that, if you like.
And I don't understand how you think the WMF can intervene (or has intervened - you claim some credit for things that I don't see you can claim credit for) with regard to behavioral issues within the editing community (e.g the gender gap or editor retention). That makes no sense structurally to me, as the WMF has no 'reach' into intra-project governance. Maybe you are envisioning some major changes in governance or process between the WMF and the projects, but as it stands, your words there sound just.. confused or empty. Well-intentioned, sure. For sure. But also kind of alarming, as you should know better than anyone what the relationship is between the WMF and the communities in the various projects.
Also, would you please tell me, where is information about the "consultation" that will begin next week?
And while I understand that this is meant to be a very forward-looking statement, I want to ask while I have you - are you going to come back to the FAQ you started so we can continue figuring out what has transpired in the past year or so? You haven't commented there since the 18th. Thanks. Jytdog (talk) 22:37, 22 February 2016 (UTC)
I also want to say that I find the whole mailing list thing to really bizarre. This seems to be some way that WMF communicates among itself, but it is just really foreign to me as a way to communicate in community. I guess that is something I have to adjust to. :) Jytdog (talk) 22:41, 22 February 2016 (UTC)
Mailing lists are always just semi-open discussions, they are some kind of hidden back-room. They are a) not in a proper wiki, where us wikimedians usually discuss, and b) slow for answers, that have to be moderated first. They are far better than google-docs or facebook or such, where you have to leave your privacy to those privacy-raping companies that run those ventures, but an ordinary on-wiki discussion is to be preferred. Grüße vom Sänger ♫(Reden) 17:05, 23 February 2016 (UTC)
Thanks! at least it is public - I am watching it unfold now. Jytdog (talk) 19:06, 23 February 2016 (UTC)

Some background on the Knowledge Engine grant

Hi Everyone,

As some of you know, I have been making concerted efforts to engage deeper on-wikis and to provide more insights into my thought process. As a demonstration of this commitment, I would like to share my thoughts on the Knight Foundation grant which has been called out for clarification.

What are the new WMF initiatives which this grant supported?

The text (after the bullet points) below is from the actual grant paperwork, and is duplicated a bit further down. Here I’d just like to highlight the functional areas of our WMF strategy which these initiatives touch.


  • Test results from search and user testing
  • An improved search engine and API for Wikipedia searches
  • Measure user satisfaction (by analyzing rate at which queries surface relevant content)
  • Can the Wikimedia Foundation get Wikipedia embedded via carriers and Original Equipment Manufacturers?


  • A public-facing dashboard of core metrics used in product development
  • A sample prototype on a small dataset to showcase possibilities
  • Create a public-facing dashboard of key KPIs
  • Use Key Performance (KPIs) to inform product iteration, and establish key understanding and feature development for the prototypes
  • Measure application Programming Interface (API) usage


  • Test results exploring relevance of content surfaced
  • Measure no results rate

Knowledge + Reach:

  • Would users go to Wikipedia if it were an open channel beyond an encyclopedia?

Reach + Community:

  • Conduct tests with potential users
  • Measure user-perceived load time

Why didn’t you discuss these ideas with the community sooner?

It was my mistake to not initiate this ideation on-wiki. Quite honestly, I really wish I could start this discussion over in a more collaborative way, knowing what I know today. Of course, that’s retrospecting with a firmer understanding of what the ideas are, and what is worthy of actually discussing. In the staff June Metrics meeting in 2015, the ideation was beginning to form in my mind from what I was learning through various conversations with staff. I had begun visualizing open knowledge existing in the shape of a universe. I saw the Wikimedia movement as the most motivated and sincere group of beings, united in their mission to build a rocket to explore Universal Free Knowledge. The words “search” and “discovery” and “knowledge” swam around in my mind with some rocket to navigate it. However, “rocket” didn’t seem to work, but in my mind, the rocket was really just an engine, or a portal, a TARDIS, that transports people on their journey through Universal Free Knowledge.

From the perspective I had in June, however, I was unprepared for the impact uttering the words “Knowledge Engine” would have. Can we all just take a moment and mercifully admit: it’s a catchy name. Perhaps not a great one or entirely appropriate in our context (hence we don’t use it any more). I was motivated. I didn’t yet know exactly what we needed to build, or how we would end up building it. I could’ve really used your insight and guidance to help shape the ideas, and model the improvements, and test and verify the impacts.

However, I was too afraid of engaging the community early on.

Why do you think that was?

I have a few thoughts, and would like to share them with you separately, as a wider topic. Either way, this was a mistake I have learned enormously from.

Was the $250,000 Knowledge Engine grant a restricted grant?

Yes. Let's talk about restricted grants and the WMF. The Foundation has taken restricted grants in the past, per our policy, especially when we were a much younger organization. As our most recent audit report shows, we have received restricted grants more recently for Visual Editor and Wikipedia Zero and Mobile (in 2014-15). With this grant we brought the idea to the funder and they supported our work with this grant. To be clear, this is not an instance of a funder driving WMF's agenda. They provided financial support to the plans we presented to them.

I’d like to take this moment to call out the requirement for Board of Trustees approval for 1) grants over $100,000 2) grants which do not conform to Foundation policies, and 3) grants which create financially instability for the Foundation. The Knowledge Engine grant was unanimously approved by the WMF Board of Trustees. All members voted and approved (( see minutes )) the grant on November 7th. The motion was made by James, and seconded by Denny.

Why did the WMF Board of Trustees vote to accept this grant?

While I cannot speak on behalf of the board, I can share my take: Restricted grants not only benefit budding organizations, but they also aid existing organizations with new initiatives. In the early stages of discovering what would eventually become “Discovery”, we decided to apply for this grant. Our aim was to begin exploring new initiatives that could help address the challenges that Wikipedia is facing, especially as other sources and methods arise for people to acquire knowledge. If you haven’t yet, please have a look at the recent data and metrics which illustrate the downward trajectory our movement faces with readership decline (since 2013), editor decline (since 2007, which we stabilized for English Wikipedia in 2015), and our long standing struggle with conversion from reading to editing. These risks rank very high on my list of priorities, because they threaten the very core of our mission. The time for investigating new approaches at the foundation to address these risks is absolutely now. Thus, the Board approved this grant to help us fund investigating and developing new ways of reaching and serving our readers.

Why should the community and staff support this decision of our board and leadership?

I would hope that for staff, the answer to this question is clear.

That brings this to a discussion more centered around, why should the community support the decisions of the WMF board and leadership? Well the honest truth is, the community has no obligation at all to support the WMF board. It is very much the other way around, the Foundation is accountable to our readers, contributors, and donors. The Board and the Foundation both act, in their own capacities and to the best of their limited abilities, to further the mission of the movement. Sometimes we get it wrong, sometimes we get it right. But it is entirely up to each individual contributor to decide whether they want to support a given initiative or not.

I’d like to have an open discussion, on a later date, on how values, policies and duty tie into interpersonal relationships and transparency, organizational productivity, technological innovation and long term relevancy of our movement and our projets.

Some reading refresh I did for this discussion:

Why did the board not publish this grant paperwork?

Generally we do not post donor documents without advance agreement, because doing so breaks donor privacy required in maintaining sustainable donor relations. In practice, I am told we have not actually published grant paperwork since 2010, but we do publicize grants in blogs when requested and agreed to by donors. A portion of the KF Knowledge Engine grant document that outlines the actual commitments we’ve made I quoted below.

What do we want to investigate that was covered by this grant?

We hypothesize that if we help our users discover more Wikimedia content through search, more users will come to us and perhaps more will engage in editing. Our basic search brings zero results roughly 30% of the time - a problem which, once fixed, extends our reach. We also want to learn if exposing sister-project content and other open sources (like open street maps) through our search interface will help our readers find and read more of our content. At the time we called this concept “Knowledge Engine”. Today, we call this “Discovery” because that is the phase the team is in. The Discovery team is actively working with Wikidata, open maps and APIs; and you can read more about it on the Discovery FAQ page.

What are the expected outcomes of this grant? (quoted text from the grant)

At the conclusion of the first stage, the results will include:

Test results exploring relevance of content surfaced

Test results from search and user testing
An improved search engine and API for Wikipedia searches
A public-facing dashboard of core metrics used in product development
A sample prototype on a small dataset to showcase possibilities

What are the activities this grant supports? (quoted text from the grant)

Answer key questions:

Would users go to Wikipedia if it were an open channel beyond an encyclopedia?
Can the Wikimedia Foundation get Wikipedia embedded via carriers and Original Equipment Manufacturers?
Use Key Performance (KPIs) to inform product iteration, and establish key understanding and feature development for the prototypes
Conduct tests with potential users
Create a public-facing dashboard of key KPIs
User satisfaction (by analyzing rate at which queries surface relevant content)
User-perceived load time
No results rate
Application Programming Interface (API) usage

What is an example of discoveries we made so far?

Portal First A/B/C test report (2 tests ran at the same time) .

With the first test group, we did not find reliable improvement as the delta was between -0.9% and 2.8%. Since it is not consistently positive, it is not reliable. With the second test group, 1.7% to 5.5% more sessions were likely to end in a clickthrough compared to the control. This represents between 300 [thousandRF] and 1.3 million more people every day likely to go through to read an article our editors wrote. These people “bounced” from our site before. This is a tangible improvement, and I’d like to thank Discovery for the great work they have been doing.

Please help us shape these ideas and validate them.

Please help provide insight on Discovery team work on the product portal pages. You can read and comment on our tests and help submit ideas in Phabricator. I am confident you can help us with both observations, opinions, ideas and safeguards. Ultimately, we’ve just started the Discovery process, and I’m hoping we can give it a clean start when it comes to ideating together, please don’t bite the newbie. I assume good faith, and I hope you do too. Let’s all treat each other with civility and etiquette, and see if we can collaborate to build a consensus on the WMF’s project direction to help readers discover the high quality content and knowledge our editors are creating.

A fellow humble child of knowledge, LilaTretikov (WMF) (talk) 22:46, 29 January 2016 (UTC)


Thanks Lila, that's interesting context. I believe the phrase "knowledge engine" has been used to describe WP or aspects of WM since at least 2014, when it showed up in the annual report. I've heard it used with various other connotations since then, but this seems as good a description as any of a constellation of tools, living reference works, and communications channels devoted to synthesizing and organizing information into usable knowledge. SJ talk  06:01, 30 January 2016 (UTC)
  • I have a question which I do not see covered above: whose employment is dependent on this restricted grant?
  • I work on that team. The work you report as supported by this grant, is my work. I wrote the dashboards, my team performs the A/B tests. I had no idea that there was a restricted grant covering this work until now. And quite honestly it scares me quite a bit to hear that a one-off donation is the thing resourcing my work - because I feel like my work is valuable to the movement's progress - and my employment, because I like making rent. So that's my question; what specifically is this being spent on? Not, what are the goals of the project, but whose existence is supported by it? And what is going to happen when that grant expires? Ironholds (talk) 15:15, 30 January 2016 (UTC)
I don't work at the Wikimedia Foundation as a staff member and so I can't answer anything specific about your employment, but I wanted to comment on the general principle of restricted grants as I see it as a board member. In general, I think it is very unhealthy for most nonprofits to "chase funding" by looking for ideas that funders will like just in order to get funding. I've seen some sad cases of distraction in nonprofits which did that. We don't do that and I would oppose any grant which led us around in that way. (As an aside, grant makers tend to hate it as well.)
So, this work is fundamental and core to our longterm progress. We need to improve the website over time, and we need to modernize and adapt. Discovery on the site isn't very good. Tools for editors to find things aren't very good. We'll want to invest in that whether it is funded by an external grant or not.
Why do some funders prefer to give restricted grants? That should be pretty obvious. They've been engaged in a dialogue about what a nonprofit working on and what they plan to do and they come to a decision to support that. They want to make sure that the money is spent in line with what they were told. The fact that a grant is restricted does not imply - at all - that the work is a whim which will stop when the grant is finished.--Jimbo Wales (talk) 20:40, 30 January 2016 (UTC)
To add to Jimmy's point, Discovery work was part of the 2015-16 Annual Plan and was funded regardless of any specific grants. The KF Knowledge Engine grant is an additional validation of our thinking and efforts. While it does not change who works on what, it helps support evolution of our projects not just financially, but from thought leadership perspective from a well-respected grantor. LilaTretikov (WMF) (talk) 03:03, 1 February 2016 (UTC)

I simply don't understand what is being said here. Lila, you say that the Wikipedia search engine should be improved. Alright. But what's that got to do with a "vision" of "free knowledge as a universe"? I don't understand even the mere meaning of this phrase. I don't understand what the question means whether "the Wikimedia Foundation can get Wikipedia embedded via carriers and Original Equipment Manufacturers". What does embedding mean here, what do you mean by carriers, which OEMs? You are asking: "Would users go to Wikipedia if it were an open channel beyond an encyclopedia?" What's that got to mean? What is "an open channel beyond an encyclopedia"? It all sounds very high-profile, not at all like the rather understandable task of improving a search engine. Isn't it possible to put in clear words what this Knight grant is good for?--Mautpreller (talk) 19:16, 30 January 2016 (UTC)

Hi Mautpreller, I realize that the grant language can be abstract and markety and I am sorry that my metaphor was unclear. Let me clarify some of the terms: we have many project besides Wikipedia itself, we also have Wikipedias many languages. Those are all knowledge projects. In addition to this, other knowledge is often linked into our projects, let's say maps or graphs that have been recently implemented by the Discovery team. In other words, we want to test if we start surfacing content from other projects (like Commons) through our search box would people find that useful? The second question about OEMs has to do with working with manufacturers that produce mobile phones to put Wikipedia on them at manufacturing time, before those devices are sold. This is especially helpful to raise awareness in emerging markets where Wikipedia is not well known. We have been working with a manufacturer in South America to attempt this. I hope this clarifies this a bit. LilaTretikov (WMF) (talk) 16:17, 1 February 2016 (UTC)
+1. Morover I have certain fears raised by this explanation (and the WMF has no credit left anymore).
First option: The lofty words about universe and stuff were a mean to lure the Foundation into funding the WMF. Well played, but then communicate at least honestly to us volunteers and drop the PR speech that is just a balloon full of warm air and nothing more. And in the end we have a better search function, ok.
Second option: The WMF tries to pull off one of the big visonary projects that it will fuck up again like all other bgger projects in the past, and tries to assure us and lure us into thinking it would be just an improvement of the search function (while blowing it up with visionary wording for the world outside the movement).
What I find most problematic about all this? The WMF recieves more donations than you can spend wisely (that'S why so much money gets burned on crap and efficiency in the WMF is a joke). The search function of our projects is a core area of your technical responsibility. Is it just me who think you could have pulled this off by the funds the WMF is already hoarding? And improvements on thos part of the system got requested by volunteers countless times over the years. Why can't you work on this issue in all seriousness and pull off a real improvement of our search engine without all this lofty PR crap? We are not a Tec Comapny, you don't ahve to sell shares to investors, you neither have to revolutionize knowledge (on which you would fail anyway). Get your business done, simple. --Julius1990 (talk) 00:03, 31 January 2016 (UTC)
  • This seems like an exciting piece of Innovation and I look forward to seeing the detailed proposal which will presumably be contained in the published grant application. Unfortunately it seems hard to get a view of all the exciting areas of innovation that the WMF is engaged in. The Phabricator tasks linked from the Innovation page seem mainly inactive and there have been no major contributions to that page by WMF staff for six months. Perhaps the scope of the WMF innovation activities are being documented and discussed at some other location, and if so it would be good to see the links. It would also be good to know who are the WMF's innovation partners (I made some suggestions last June, around the time you were ideating the open knowledge universe, but I am sure that the movement can give you a much wider range than I can). Where would you like to see that aspect of the community engagement, which is much wider than the so-called Knowledge Engine project, take place? Or are you satisfied that you already know all you need to know to take forward the WMF innovation programme? Rogol Domedonfors (talk) 22:33, 31 January 2016 (UTC)
Hi Lila, thank you for the clarification so far. Please give more detail about what is "an open channel beyond an encyclopedia"? --Sebastian Wallroth (talk) 17:17, 1 February 2016 (UTC)
  • I agree our search engine needs improvement and the majority of the ideas and KPIs above have obvious merit, and sympathize with the need to trot out the bullshit for grant applications. There are two problems here: (1) we volunteers appreciate straight talk and (2) something like this is concerning -- Wikipedia is an encyclopedia that has a search engine, not the other way around. Placing a box beside existing search results that says "hey, Wikivoyage has a travel guide on X", or selecting a language automatically based on IP (with ability to disable this if not wanted) is likely to be non-controversial. Finally, this new search function needs to be for every project, not just Wikipedia. MER-C (talk) 12:36, 9 February 2016 (UTC)

What is being requested

What has been requested is the "grant application". This is a document prepared by the WMF and submitted to the Knight Foundation rather than a document from the donor.

All other moment entities, including chapters and those applying for individual engagement grants publicly post proposals for funding. I do not understanding the reasons the WMF cannot also? Doc James (talk · contribs · email) 16:56, 30 January 2016 (UTC)

I would like to see a citation for the claim that all other movement entities publicly post proposals for funding. In particular, I would be rather surprised if all of the EU research project submissions to FP6, FP7, and Horizon2020 are being published for the chapters that participate in them. Do you have any support for this statement, James? --denny (talk) 17:39, 30 January 2016 (UTC)
I think that is certainly not the case, as you say, Denny. My guess is Doc James meant "proposals to WMF for funding". Those have indeed been publicly documented every time (with only one exception I'm aware of in my tenure (since 2011)). Asaf (WMF) (talk) 22:34, 30 January 2016 (UTC)
To clarify yes I am referring to proposals to the WMF for funding. And thank you User:Asaf (WMF) for answering. Doc James (talk · contribs · email) 09:28, 31 January 2016 (UTC)
Grants:APG/FDC recommendations/2015-2016 round 1#WMF: "...the FDC laments that the Wikimedia Foundation’s own planning process does not meet the minimum standards of transparency and planning detail that it requires of affiliates applying for its own Annual Plan Grant (APG) process. The FDC is appalled by the closed way that the WMF has undertaken both strategic and annual planning, and the WMF’s approach to budget transparency (or lack thereof)."
Speaking of transparency, denny: Do i remember correctly you writing somewhere, that you were obligated as Wikidata project lead to write reports for funding institutions of wikidata (google etc.)? Is this true? Did you write reports, and if so, where can the community read them? --Atlasowa (talk) 23:47, 30 January 2016 (UTC)
Slightly off-topic, no? Yes, the Wikidata reports (both the intermediary and the final one) were all published, just as was the working plan for the Wikidata grant before work started. I am too lazy to find out the links right now. Why asking for possible issues in completely unrelated topics? --denny (talk) 04:59, 31 January 2016 (UTC)
Comparing the degree of transparency shown by various components of the movement seems entirely appropriate for this discussion. Rogol Domedonfors (talk) 13:54, 31 January 2016 (UTC)
The funder has agreed to share the grant agreement publicly. It is available here. --Lgruwell-WMF (talk) 19:33, 11 February 2016 (UTC)
Lisa, many thanks for that. Rogol Domedonfors (talk) 21:29, 11 February 2016 (UTC)

Lila, Sj, Ironholds, Doc James, Denny: Having a document like this published on a user talk page seems sub-optimal. Those who are interested in this topic (as opposed to those interested in this user's thoughts generally) can't watchlist it here; among other issues. Any problem if I move this elsewhere for continued discussion -- e.g., Lila Tretikov's remarks on the Knowledge Engine, January 2016? -Pete F (talk) 17:56, 30 January 2016 (UTC)

Or better yet, simply Knowledge Engine -- with a section reserved for Lila's comments. -Pete F (talk) 17:59, 30 January 2016 (UTC)
I reverted the move because although I agree there should be a centralized place for discussion, this is a good discussion here separate from that.--Jimbo Wales (talk) 20:32, 30 January 2016 (UTC)
If you'd like to discuss the topic in more depth, I recommend the discovery page talk page. LilaTretikov (WMF) (talk) 17:59, 31 January 2016 (UTC)
For the record, I was not involved with commissioning that report. I was, however, interviewed at some length by its author. Asaf (WMF) (talk) 22:16, 30 January 2016 (UTC)

Questions on the grant proposal

Lila, the proposal uses the phrases "openly curated", "Public curation mechanisms", "curation of that data". May we know who you envisage undertaking this curation? Are you by any chance assuming that the Knowledge Engine by Wikipedia will be curated by the current Wikipedia volunteer community? Since you did not put this down as a risk, it seems that you are reasonably sure that the people you expect to carry out this curation will indeed do so -- what is the basis for that assessment? Rogol Domedonfors (talk) 21:39, 11 February 2016 (UTC)

I'll take this as having been answered here with the assurance I am not aware of any actual plans for Discovery work that would rely on any human curation beyond what is already being done. Rogol Domedonfors (talk) 21:53, 18 February 2016 (UTC)

Results of A/B/C test

@Ironholds: The paper says "With the first test group we found no trustworthy improvement; the difference for the test group as between -0.9% and 2.8%. Since it isn’t even consistently positive, it is not reliable."

This needs explanation. How can the results of comparing two populations be a range? How are the results "unreliable" becasue they contain a negative number? What are the raw click-through percentages for each group? What are the search click-through percentages for each group? What statistical methods have been applied to measure significance?

Given that there is more clickthough, how do we measure the value of this? Are we distracting readers from the primary content onto sidetracks? If so is this a good thing or a bad thing?

Rich Farmbrough 14:32 24 February 2016 (GMT).

Return to the user page of "LilaTretikov (WMF)/Archive 12".