Research talk:WikiCV

About

@Meghasharma213: is the lead in this project. Blue Rasberry (talk) 13:00, 14 December 2017 (UTC)

I'm working on this project under the mentorship of Tgr and Slaporte.
Meghasharma213 (talk) 06:05, 17 December 2017 (UTC)

Notification

In December 2017 Meghasharma213 posted to WikiProject Medicine and WikiProject Genetics asking for feedback on the proposal.

• discussion at medicine
• discussion at genetics

Blue Rasberry (talk) 13:00, 14 December 2017 (UTC)

I've posted to Village Pump (Technical) as well to gather inputs.
Meghasharma213 (talk) 06:05, 17 December 2017 (UTC)

Comments from bluerasberry

I see that in conversations at WP:MED and WP:GEN you are already getting good comments. I read those. Here are my further ideas -

Incorporate a way to report some count of how many times other people have viewed content which a person has edited.

For example, imagine that a person has created or heavily edited an article enough to trigger this audience tracking function. In that case, report all the pageviews which that article has gotten since the person triggered the tracking. This metric is Wikipedia's approximation of other social media impressions, like how many people viewed a tweet.

I talked about this concept at Development_of_the_programs_and_events_dashboard, which was a selected proposal in the recent 2017 wishlist survey. The takeaway point is that tracking audience response is a priority on all major new media platforms except Wikimedia projects, but I think this should change. We should track, report, and incentivize the development of content and contributors who are reaching a larger audience.

Anticipate a future with badging or certification

In your mockup you have a field called "achievements" which is sort of like this, but your mockup imagines this section to be self-reported. I am imagining something like this but with badges.

There is a perennial call for some way to recognize program participants with a badge or certification. For example, we have annual campaigns like Black History Month, Art+Feminism, Wiki Loves Loves in Library Week, and a calendar of regional and global Wikipedia holidays which are a big part of our outreach. Probably it is beyond the scope of your project to implement a badging system, but please be aware that (1) program organizers already give badges (2) participants like badges (3) we have a lot of trouble managing the display of badges long term (4) and we also have trouble establishing that third-parties can credibly believe a badge.

There is a Wikipedia Education Program in which professors have students edit Wikipedia. Students like completion certificates. At the high school level they use these certificates to fulfill community service obligations and at college level these kinds of certifications have other uses. Again, I see that you are working on an automated display of data which you can pull from a profile, and we do not document certificates or badges in this way, but just keep this use case in mind.

Thanks. Blue Rasberry (talk) 13:07, 14 December 2017 (UTC)

Firstly thanks a lot for your inputs and also for making this wiki page!

Now addressing your comments, adding page views data is already on my list! I got a suggestion too to take % contribution to an article and multiply it by page-views to said article over a given month to give an editor’s total impact. But yes, taking page views after an edit would be more precise, I guess.

Secondly, by badges I meant pulling out the user's data and showing the major achievements in the form of a badge, e.g user is an admin or is in top 1% editors. It might differ from what badging system Wikipedia has as of now but all I wanted was to highlight the user's achievements.

Meghasharma213 (talk) 06:05, 17 December 2017 (UTC)

@Meghasharma213: At this point for a first round an award like "top 1%" is fine as a demo. In the longer term that is problematic.

"Top 1%" can be defined in lots of ways. We do not want to incentive and celebrate only people who do automated edits, so just counting edits to calculate a top 1% is not a long term solution.

Instead, there should be a way for someone to propose awards and badges, and for other people to sign off that they support them or think a certain badge is a good metric. When lots of people upvote a badge then that should make it more respectable than other badges and help to define the community's values.

In this pilot project please do demo an award for top 1% or whatever seems natural to recognize. But as you do that, imagine a next step in this where somehow, the community can decide what to recognize as award-worthy and how a WikiCV might both showcase awards and also lead to discussions about people discussing the metrics behind the award, its significance, and how to make the award more meaningful. There probably is no single datapoint in a wiki user's profile which will tell a clear and accurate story in even 70% of users. Blue Rasberry (talk) 16:14, 17 December 2017 (UTC)

@Bluerasberry: I totally agree with your point. I'm actually doing the same. Based on the inputs of the user and discussions with my mentor, I'm making a design mockup and a document which contains definitions of all the metrics. I'll gather feedback on this and mend it based on that.

Comments from The Quixotic Potato

Please don't get discouraged by the fact that I am a negative Nancy (trust me, it is worse for me than it is for you). My userpage contains the word "fuck" twelve times. I have edited articles about the ideology of the SS and the effects of pornography. I have made edits like: 1 2 3. Those are NSFW. I don't have an employer, but I wouldn't want my customers to know that I edit Wikipedia as The Quixotic Potato. Some of them wouldn't understand. Quite a few of us Wikipedians (including myself) have edited Wikipedia while we were supposed to be working.

AFAIK people edit Wikipedia because they believe in its mission, not because they want to use it on their CV. I am a WikiGnome, not a WikiPrincess. On my userpage I have a userbox that says: "This user has made more than 1 contribution to Wikipedia.", which is a parody of userboxes proudly displaying their owners editcount, and I have created JAPAFEISC which is basically a nonsensical version of a conventional WP:BARNSTAR. I don't care about barnstars or Wikipedia:Notifications/Thanks. Editing Wikipedia does not give me bragging rights, and people's reaction (in general) is that I am crazy and wasting my time doing unpaid volunteer work. My reward is the feeling I did something good.

The relative "value" of an edit (measured by the amount of time required to make it or the amount it improved the encyclopedia), or the value of one's participation as a whole, is impossible to determine using software. The people who make the most edits spent very little time thinking about each individual edit. Some Wikipedians rewrite entire articles in a single edit (after having spent several days or weeks doing the research required) and receive no feedback whatsoever. Some people have written hundreds of low quality articles full of copyright violations, WP:POV and nonsense. Some people run bots or use tools and make many edits each day. I have edited over 2000 articles in 24 hours, making a typofix on each. To make matters even more complicated, I've written software which finds and fixes typos that are not on the typo list. Some people clearly have a case of WP:Editcountitis and make edits for the sake of making edits that don't actually improve anything, see for example WP:COSMETICBOT. Quite a few of us strongly dislike social networks (and those who use Wikipedia as if it is one) and Wikipedia:Hat collecting.

The good news is that you are clearly very skilled, and we can really use people with the skill set you have. I am not a big fan of this idea, but I know that there are many of us who are less negative than I am. My opinions are not always the same as those of the majority of Wikipedians. On this page you'll find quite a few people who are working on many different projects, and they can really use your help! It is very difficult to come up with an idea for a new tool that is useful and doesn't re-invent the wheel. My advice would be to start by helping others with their projects, instead of creating your own. This video was really interesting to me. I can see you've put a lot of work into this (because it looks beautiful!) and I think it is important to emphasize that I am impressed with your skills and thankful for your willingness to help our community. (((The Quixotic Potato))) (talk) 05:25, 18 December 2017 (UTC)

I myself don't think you are a negative nancy and I do think you are one of many who feel similar. I also think that what you say is indeed a important and a large part of why Wikipedia is both successful and different from many other things in this world. But at the same time I often feel we have taken this a bit too far. Recently I talked with several people who have done great things with organising editathons, conferences, facilitating writing contests, researchers and some chapter office employees. Some of their experiences can be summarised as: We have less time for editing now and we feel like we are being shunned by the community due to our activities. I think that is VERY bad. So i'd like to see that part of 'professionalism' being brought back into the fold somehow and I think that with a few tweaks a project like this might be able to help facilitate that. —TheDJ (talk • contribs) 14:35, 20 December 2017 (UTC)

Oh and "My advice would be to start by helping others with their projects, instead of creating your own." while indeed that is a great way to learn, i think it's a terrible way to be innovative. Build whatever you think is valuable for the community. It's how most tools on wmflabs came into existence. If it's used, then it's successful, if it's not used then it will fail. rinse, repeat. analogous to WP:BRD. Others' inability to think of something new that would be useful seems to me to be the worst reason to limit your own creativity. —TheDJ (talk • contribs) 14:48, 20 December 2017 (UTC)

You can be innovative while working on a project started by someone else. If you wanna create something totally new from scratch then having a little bit of experience won't hurt. We are talking weeks here, not years. The sentence about inability makes little sense to me in this context, because you can be the first follower in a project to create something totally new and unique. (((The Quixotic Potato))) (talk) 07:51, 21 December 2017 (UTC)

@The Quixotic Potato: thanks for your inputs! Firstly, I got this project under the Outreachy program. And the main intent of the project is to introduce the idea of quantification of performance among the editors. I do agree that many editors contribute out of intrinsic motivation and don't need such a tool. But we want to motivate the other bunch and the new editors as well!

And I also feel that it is better to be experienced when doing a project of your own but my inexperience will balance out because of my mentors' experience. Thanks! Meghasharma213 (talk) 17:16, 23 December 2017 (UTC)

┌─────────────────────────────────┘
I noticed this edit. Please see WP:NAMESPACE. (((The Quixotic Potato))) (talk) 10:23, 7 January 2018 (UTC)

I believed Meghasharma213's edit[1] was due to a misunderstanding of namespaces. The Quixotic Potato above also thought that version was a mistake. Tgr (WMF), I'm confused that you reverted[2] back to that version, and your edit summary only left me even more confused. I have been unable to find any way to connect the words "namespace" and "technology" into any kind of coherent meaning. Apparently The Quixotic Potato couldn't understand it either. Either the current text is a mistake, or the sentence desperately needs clarification. Alsee (talk) 07:23, 29 January 2018 (UTC)

Hi Alsee! The overarching design principle for this tool is to be accessible to non-Wikipedians (present user contributions in such a way that you don't need to know a lot about Wikipedia to understand them - otherwise you can already use XTools etc), and namespaces don't really meet that criteria. So the idea is to group them into clusters which are more self-explanatory - e.g. the main namespace would be "content contributions", Template/Module/MediaWiki would be "technical contributions" and so on. --Tgr (WMF) (talk) 19:28, 29 January 2018 (UTC)

Tgr (WMF), ahhh. Thanx. Grouping Template/Module/MediaWiki as "technical" was the elusive concept. I'll take a shot at editing it for more clarity. I have to note though, it's rather crude. Userspace can be anything - in particular a lot of articles are written in userspace / usersandboxes. They would presumably be excluded from "content". Edits in draftspace are heavily split between content, draft review&management, and garbage. etc. Alsee (talk) 06:24, 30 January 2018 (UTC)

Thanks! Yeah, it's a rough approximation but so'll be many other things (like contribution ratios or identifying article topics). The goal of the project (besides the standard Outreachy goals of attracting new people to the developer community and increasing diversity) is to produce an MVP that can be used to evaluate interest and provide further UX/design insight, not to create something super polished (that's not realistic for a single person in a quarter, much less for a student project).

I'm thinking of main + draft -> content, template/module/mw -> technical, all other subject namespaces -> collaboration, talk namespaces -> discussion. (Or maybe collaboration vs. discussion is not really a useful distinction and talkspaces should just be counted along with the corresponding subject namespaces.) From an employer's point of view (this is a CV tool after all) content creator vs. technical contributor vs. organizer seems like a relevant distinction, everything beyond that not so much. (And yeah, draft namespace edits can be for organizing things, and project namespace subpages can be templates, and user subpages can be whatever. Not worth putting too much effort into more reliable categorization while it might turn out later that no one actually cares about this kind of breakdown at all.) --00:20, 31 January 2018 (UTC)

On editcountis

One of the main reasons we have a dislike for counting edits and showing off that count, is that it is a very bad metric for quality of participation. It is my personal opinion that if you are thinking about measuring someones participation and reflecting this, you need to balance that by reflecting other forms of participation than the quantity of edits. You recognise how long someone has been participating, and their consistency in that participation. Or maybe a recent increase in the rate of participation. The average size of their contributions. Instead of noting all FA/GA articles by a user, you show their first Featured article and their latest. Or you have thresholds that you reflect in the year you pass them. but then forget about again in the next year. Their activity as a gnome, or their activity in supporting newcomers. Basically the wider you are able to recognise what participation is, and the less you emphasise quantity, the less problematic keeping 'score' is. Think about time, quality, rate, size, duration, type-aspects instead of just counts and then I think there shouldn't be any problems with editcountis. —TheDJ (talk • contribs) 15:06, 20 December 2017 (UTC)

Comments from Jokulhlaup

So, if we are not keen on the number of edits alone as an indicator of quantity, would a function of counts, number of articles created and number of years’ service, with a square root reduction factor for edits (to reduce editcountis) be useful, for example:

• WikiCV Score=((sqrt (counts))*articles*years/1000)
  • So Bluerasberry gets a score of (Sqrt(45,000)*428*9)/1000=817.3

It’s not perfect, you would have to include a fudge for users who have not created any articles (say 0.1) to prevent a zero score. It’s worth saying that a scheme such as this should be inclusive, and not used just to find the top 1%, as people want to know where they sit in relation to other editors.

Following on from the other comments you may also want an indicator of quality, a summary banner of icons could be used, with the number of quality articles produced, number of barnstars etc, plus something to show that they are an admin or other notable user rights.

• Template:Class/icon FA 2 :Template:Class/icon GA 10 : Template:Class/icon DYK 12 Barnstars 7

These are just suggestions to help this project along...Jokulhlaup (talk) 12:34, 21 December 2017 (UTC)

Nice for those of us who do not write articles like myself... (((The Quixotic Potato))) (talk) 12:37, 21 December 2017 (UTC)

A score of 0.04 and Barnstars 2, is not bad for a potato...Jokulhlaup (talk) 15:12, 21 December 2017 (UTC)

True, but it shows that there is no formula/algorithm that can be used to accurately determine the value of someone's contributions. Try the formula above with some random Wikipedians (e.g. someone who does not write articles and one who used a bot to post thousands of stubs, or someone who has been a net negative for over a decade vs a newbie who does excellent work) and you get all kinds of weird results. Comparing someone who collects barnstars and another who deletes everything from their talkpage after a while is also quite unfair. The formula also does not take Thanks & WikiLove into account. (((The Quixotic Potato))) (talk) 16:41, 21 December 2017 (UTC)