Talk:Community Tech/Pageview stats tool

This project is complete -- for further discussion and bug reports, please use Talk:Pageviews Analysis.

This is the link between experts and Wikipedia

The single most important thing that Wikimedia projects can do to attract expert attention is to provide metrics.

Twitter and Facebook became popular among established organizations because they provided metrics reports that allowed organizations to quantify the value of using them for information distribution. Likewise, if Wikimedia projects could ever consistently provide pageview feedback, then that would meet one of the most important prerequisites for getting expert organizations to invest in content development on Wikipedia.

I work for an organization which employees me to use Wikipedia to share information. A fundamental part of my employment is proving that when content is posted to Wikipedia, the pages hosting that content get pageviews. This is a scalable outreach model. The lack of ability to provide viewer feedback is a major cause for the poor reputation of Wikimedia projects in academia society generally.

Please develop this project as quickly as possible. The sooner this is developed, the sooner institutional partnerships can begin to mature. Blue Rasberry (talk) 16:19, 2 February 2016 (UTC)

We're working on it! :) There's a group of volunteers who are going to work on this tool for the next couple months, and then our team will pick it back up, and either make some improvements or just help to get it deployed.

There's a list of possible features on the project page, under the 2016-1-26 heading. The list is broken down into Basic features, Nice to have features and requests from Community Engagement. Are there items on the "nice to have" or "requests" part of the list that would be really important for your work? -- DannyH (WMF) (talk) 20:35, 2 February 2016 (UTC)

DannyH (WMF) I added a community request section that says, "Given a list of Wikipedia articles and a language, present pageviews for each of them over any give set of months "

I am not sure who is developing this or what information would be useful, so for context, I am going to describe a use case.

All organizations which have expert staff develop Wikipedia (as a Wikipedian in Residence or otherwise) need pageview data for sets of articles. Pageview data over time for sets of ~20 articles is required and persuasive evidence for telling any organization that it is necessary that they develop Wikipedia. This includes all GLAMs, research institutes, universities, and medical organizations.

Sometimes I think WMF take it for granted that the outside world thinks Wikipedia is popular. This is not the case - the almost universal perception is that Wikipedia is a fringe source of information and hardly read by anyone. Anyone who is able to see quarterly traffic of ~20 Wikipedia articles in their field gets a changed perspective. I need a quick way to get pageview reports from multiple articles at once.

Note that "Ability to see page views by category" is different. Categories are community managed and hidden categories are controversial. It is nice if the pageview tool can report page views by category, but the option for other input is critical. One nice input might be "Ability to see page views of all categories and articles listed on a userpage" that way individuals can create their own list of interest in their own space and not have anyone disturb it, as would happen with categories. The tool at http://wikipediaviews.org/ allows anyone to copypaste the text of titles into a box, and that works. Blue Rasberry (talk) 15:26, 7 February 2016 (UTC)

@Bluerasberry: This is already possible with toollabs:musikanimal/pageviews. There is currently a limit of 5 articles at once, but I guess I could increase that, just the chart might not look as pretty. I might be able to address that issue with different chart options. Anyway, I guess I want to offer any help with this project, since I'm already working on one of my own. I'm not sure about showing all pages in a category. That could be very, very expensive (e.g. en:Category:Living people). We'll need a limit or something, and probably throttling of the API requests. I do like your idea of providing an on-wiki page list of pages to view, I can try to add something like that — MusikAnimal ^talk 20:46, 7 February 2016 (UTC)

Actually, you could just bookmark a link to the tool with all the articles you want to view. E.g. https://tools.wmflabs.org/musikanimal/pageviews#start=2016-01-18&end=2016-02-06&project=en.wikipedia.org&pages=Google|Bing|Yahoo!|Baidu, where you just append |article_name to the end. Leave the date range out to get the most recent data, as with https://tools.wmflabs.org/musikanimal/pageviews#project=en.wikipedia.org&pages=Google|Bing|Yahoo!|Baidu. Not that user-friendly for your purposes, but it works — MusikAnimal ^talk 20:52, 7 February 2016 (UTC)

BTW: Could you add a feature, so that it works with canonical name space names? At the moment you need to know, if a user is a "Benutzer" or "Benutzerin" to access the stats for the user page. Even worse, if the user has changed gender information at some time in the past, there will be different stats for Benutzer and Benutzerin. --° (Gradzeichen) 23:08, 7 February 2016 (UTC)

Wow, I had no idea this was a thing on some wikis! If you use the search built into the tool it will correctly resolve to Benutzer. For direct links, I'm not sure of a graceful way to map Benutzerin to Benutzer without hard coding it – which I can do. Expect a fix soon — MusikAnimal ^talk 00:05, 8 February 2016 (UTC)

@°: Fixed (I think), let me know if you have any issues — MusikAnimal ^talk 01:19, 8 February 2016 (UTC)

@MusikAnimal: I will check the benutzer/in thing later, but some additional remarks: stats.grok.se also shows statistics for inexistent pages (redlinks), could this be added? If not, there is a problem: If you query for more than one page and one of them does not exist, no information at all is displayed. A number of german users reported problems with content blockers (see de:Wikipedia_Diskussion:Kurier#Abrufzahlen): the page is displayed, but the graphic never arrives, so it looks like a problem with toollabs, not content blocking. Is there a way to inform the user, if this happens? (and a nice to have: the display date format m/d/y is confusing to germans, could you add an url parameter &dateformat=dd._mmm._yyyy to the tool?) --° (Gradzeichen) 10:55, 8 February 2016 (UTC)

All fantastic ideas. I want to add full i18n support at some point, but I suppose in the meantime I could go by a dateformat parameter. Thanks for the suggestions! — MusikAnimal ^talk 16:05, 8 February 2016 (UTC)

:-) - have you looked at the Kurier-talk-page? There are to more requests: scaling of the x-axis and a question if the numbers are shown at the correct position. May be you could comment there yourself? --° (Gradzeichen) 17:11, 8 February 2016 (UTC)

┌─────────────────────────────────┘
Well, can't say I know any German so I probably won't get very far trying to communicate there =P Scaling of the x-axis is definitely needed. That's an issue with the chart plugin I'm using. I'm going to explore other plugins, but also provide options for different types of charts. So maybe when it gets too cluttered another type of chart might look better. I hope the numbers are at the correct position! The tooltips that show when hovering over a point sometimes overlap if it's too tight of a graph, maybe that's what they're talking about. If you hover over the dates on the x-axis the tooltips will not overlap — MusikAnimal ^talk 18:27, 8 February 2016 (UTC)

While german wikipedia is picky about language, it is ok to write on talk-pages in english. For the meaning of the german text: as there is no complex language used, babelfish should give you an idea, what they are talking about. --° (Gradzeichen) 19:12, 8 February 2016 (UTC)

@°: I have added locale support for the date formats. It should just work, going by the language your computer is set to. It uses the default format for your country, which in the US is M/D/YYYY. I actually prefer YYYY-MM-DD, but alas, I should conform to the default. Maybe I'll implement some kind of "settings" panel, where you can change things like this, and the colours, etc. — MusikAnimal ^talk 21:20, 13 February 2016 (UTC)

@MusikAnimal: Great! It works. But I have noted something strange: When I click the url, I get the cat|dog example page at first, only at reload the correct page. Maybe this is because of the use of # instead of & or ? in the Url? And while I am at it: It would be really nice if you could add three more things. 1) optional display of canonical name space names (useful when browsing foreign projects), 2) If one of the selected pages does not exist, display at least the data for the other selected pages, 3) If the accumulated number of views is high, you get in the footer an output like "1052863 (8774/day)", it would be nicer, if it was "1.052.863 (8.774/day)". --° (Gradzeichen) 21:43, 13 February 2016 (UTC)

@°: Hmm I'm not able to reproduce the issue you are seeing. Could you give me a specific link? For (1) I can add this to the to-dos, but low priority, to be honest. I'm going to need to introduce a whole "settings" panel in order to get all the features people want on the same. Doable, just will take a bit :) For (2) this I plan to fix very soon. (3) I've kind of wanted this myself, but there's the issue of people copying/pasting the totals to somewhere else. Many times the delimiters are not desired. Furthermore I'll need to do a little i18n for it as well, e.g. in the US we use a comma and not a period. Keep the feedback coming! :) — MusikAnimal ^talk 00:55, 14 February 2016 (UTC)

@MusikAnimal: I just send you a mail on reproducing this. (3) I am working on a project with this difficulty and I use the draggable/ondragstart-attributes to provide the user with an unformatted version of the selected text. --° (Gradzeichen) 06:54, 14 February 2016 (UTC)

Pageview Comparison by MusikAnimal on Toollabs

MusikAnimal has created a tool on toollabs, that uses the api and shows article statistics. I have created a script, that includes a link to this tool in the "More"-menu on every page in every name space on every wikimedia project. The date range can be configured individually. To use this, add the following to common.js or global.js:

// mwASdays = 30;
mw.loader.load('//de.wikipedia.org/w/index.php?title=User:°/mwArticleStatistics.js&action=raw&ctype=text/javascript');

--° (Gradzeichen) 10:37, 7 February 2016 (UTC)

If you are using a blocker like uBlock or AddBlock Plus, you might need to whitelist /pageviews/* on toollabs for the tool to work. --° (Gradzeichen) 16:47, 7 February 2016 (UTC)

Making Pageviews Analysis multilingual

@DannyH (WMF): Saw your update :) First off, if it's not already clear, I'm completely open to making my work a joint effort with Community Tech (and anyone who wants to help, really). I've been working hard to get the code in a more collaborative-friendly state. I could talk more about code in particular, but I'll save that for later.

Kaldari has updated the CSV format and we've deployed that change. Many of the other non-feature issues I imagine we'll get through quickly, but one thing I personally am incapable of is translations. If anyone is able to help with that it would be fantastic. I don't really know the best way of going about this. With my Ruby projects, I had a predefined set of dictionaries that defines each and every phrase into a given language. I feel like there must be an easier way... any advice is appreciated. I'm aware of translatewiki.net, which is what brought toollabs:xtools to each language, but my impression is it works much in the same way as the dictionaries. — MusikAnimal ^talk 05:39, 18 February 2016 (UTC)

@MusikAnimal: That's awesome, I'm really glad we're working with you on this. A big part of our team's mission is supporting developers who are working on these projects, filling in gaps where we can be helpful. Kaldari said he was having a good time working with you. :)

I don't know much about translations. I'm sure Kaldari has ideas -- he's traveling today, but I know he'll get back in touch soon. I'll ask the other folks on the team if they have any thoughts. -- DannyH (WMF) (talk) 18:37, 18 February 2016 (UTC)

Hi MusikAnimal! There are a few tools that exist, which we can look into for doing the translations. I looked into some of these here: phab:T112307 but that's for Python tools. Intuition works with PHP tools only right now. We'll have to look for solutions for Ruby. If not, we can look into porting Intuition to Ruby. I think we have a ticket for doing this next sprint. NiharikaKohli (talk) 18:48, 18 February 2016 (UTC)

New documentation page

I started a documentation page for users to consider their options for traffic reporting. This tool is first, but I listed some other options as well. I also presented some discussion about how and why traffic reporting is used.

• Traffic reporting

If anyone knows of other places discussing pageviews then please link those pages with this one. Thanks. Blue Rasberry (talk) 16:33, 1 June 2016 (UTC)