Grants:IdeaLab/Multidimensional EPR (Earned Public Reputation)

Multidimensional EPR (Earned Public Reputation)

The overall health of the Wikipedia community is a reflection of the health of the individual members, so it should start with a meaningful metric of the people who contribute to Wikipedia.

countryJapan

themepersonal experience

idea creator• shanen

volunteer• Gryllida• Shanen• Tong lenong

this project needs...

volunteer

contact email

give feedback

join

endorse

created on20:08, 25 July 2018 (UTC)

Project idea

What Wikimedia project(s) and specific areas will you be evaluating?

Is this project measuring a specific space on a project (e.g. deletion discussions), or the project as a whole?
The EPR metric is intended to evaluate the characteristics of contributors to any part of Wikipedia.

Describe your idea. How might it be implemented?

Provide details about the method or process of how you will evaluate your community or collect data. Does your idea involve private or personally identifying information? Take a look at the Privacy Policy for Wikimedia’s guidelines in this area.
EPR should be based on orthogonal binary dimensions that are relevant to the purposes of Wikipedia. Each metric should be biased in the positive dimension, in that it should be easier to receive or earn a positive evaluation than a negative one, but people who actually earn negative reputations should be penalized, for example by requiring all of their edits be reviewed by people with more positive reputations.

It is probably easiest to illustrate with a relatively clear and simple dimension, which might be called reliability. It would mostly be based on evidence of favorable use of a contributor's contributions, while the negative side of this dimension would mostly come from reversions or or corrections made to that contributor's contributions. If the EPR goes far enough into the negative range, then that contributor's future contributions would become subject to delay pending review, while a highly positive contributor might even be featured, perhaps with some form of highlighting to indicate that the passage came from such a person.

There are actually a number of dimensions that are relevant in the Wikipedia context, but I think the EPR should be displayed in a paired fashion against self-claimed reputation. Essentially there should be two aspects of reputation, what the person wants to say about himself, while EPR captures what the world is saying about that person. In the context of Wikipedia, EPR is seeking to answer the questions about the integrity, credibility, and reliability of the people who are contributing to the articles. I think this could best be displayed in a small radar icon, which would link to the summary page of all the components of EPR, and that page would also have links to the data. The health of the entire Wikipedia community would then be reflected in the trends of EPR statistics.

The idea does involve PII, but only information that the individual has posted in public. There should also be a mechanism by which people can appeal against negative evaluations, and all of the data used for evaluating the EPR should age over time, automatically favoring more recent contributions over older ones.

Not sure how clear this elevator pitch is, and also don't regard myself as sufficiently knowledgeable about the internals of Wikipedia. It usually seems rather close-minded to me, but there are many other aspects of this suggestion and I'm willing to answer any questions.

Are there experienced Wikimedians who can help implement this project?

If applicable, please list groups or usernames of individuals who you can work with on this project, and what kind of work they will do.
Not that I know of, even by reputation. Is Dr Ed Morbius here?

How will you know if this project is successful? What are some outcomes that you can share after the project is completed?

The main success of this project would be if it helps people become better by understanding their own EPRs so they can try to become better people. The success would also be reflected in the EPR demographics of the entire Wikipedia community. (I should have mentioned my studies of sociology in my self-description...)

How would your measurement idea help your community make better decisions?

After you are finished measuring or evaluating your Wikimedia project, how do you expect that information to be used to benefit the project?
Not sure what "your community" is supposed to mean in this context, but I want to be a better person and thus a better member of every community that I participate in.

Do you think you can implement this idea? What support do you need?

Do you need people with specific skills to complete this idea? Are there any financial needs for this project? If you can’t implement this project, can you scale down your project so it is doable?
Actually, in my programming days I did help implement something similar to this as part of a teacher evaluation system. It was one of the more successful such systems in the state where we wrote it, but I stopped programming long ago and don't know if that system or its descendants are still in use. At this point I'm more interested in the mathematics of it, especially in the symmetries between publications (including videos), the authors of the publications, and the reactions of the readers to those publications. I'm also interested in driving solutions from the publications, though I think that part is less relevant in the Wikipedia context.

Get Involved

About the idea creator

shanen: Unusually broad but shallow. Widely read, and spent many years as the technical editor and rewriter for a major research lab. Before that I was a senior referee for the IEEE Computer Society. Basically retired now (and read over 200 books last year), but I still regard myself as a pure solutions researcher. Implementation is a problem for the real engineers.

Participants

Volunteer would like to join to make sure that there is an opt-out - any attempt to quantify users is subjective (and also to encourage creating several ratings (separate rating for content edits, for talk page edits, for user talk edits, etc) several ratings may make it easier for others to know that the user's rating is subjective and not a measure of superiority, but even then I think this would be rather dangerous in such a large group as English Wikipedia)

Instead I would suggest to make it easier for users to join wikiprojects as this is where collaboration and encouragement comes from Gryllida 01:17, 26 July 2018 (UTC)

Volunteer Question: Not sure how to address a question to Gryllida in this context, but I'm trying to understand the basis of his (or her) opposition, even though I am not opposed to the idea of allowing people to opt out. In terms of the way I have suggested implementing the idea, that simply means the second icon associated with the person would have a special null ("no EPR") icon instead of the standardized EPR display. I also think there should be an option to wipe out your EPR data without erasing all of your public contributions, though perhaps you should have to provide some kind of explanation why you can't wait for your data to age and naturally lose most of its negativity.

My question is actually about anonymity in the context of Wikipedia. I think that all of the edits and contributions you make on Wikipedia are currently treated as public data, and EPR could be thought of as a way to make that data more accessible and more understandable by organizing some of it. However, I also think that anonymous contributions are allowed, but that certain restrictions apply. I guess that one way to reformulate this question is to ask if a special category is needed, or if people could just opt out of EPR by not logging into Wikipedia? What would the difference be? Shanen (talk) 07:50, 26 July 2018 (UTC)
Volunteer I don't know if I need to join my own submission, but it definitely seems that my involvement is necessary to try to clarify some things that seem too obvious to me. Shanen (talk) 07:37, 29 July 2018 (UTC)
Volunteer join. thanks Tong lenong (talk) 13:27, 2 August 2018 (UTC)

Endorsements

Support It seems like a good idea but it also needs more work. I think it would do more good than harm and help improve the quality of the articles as well as contributions to Wikipedia, encouraging more good-faith edits and discouraging vandalism or other bad behavior.

However, there are various details that are left out and possible compliance issues with Wikipedia policies, as those who do not support this idea have noted. So I think that this idea basically needs more people to contribute to the idea and help refine and improve it and solve the problems with it. But overall it is a good idea and we should try and build on it and improve it instead of just discarding it without even seriously considering it, as some here are advocating.

Yes there are some serious flaws with this idea that others have pointed out, but I think this idea could be improved from its present form to fix them. I have some experience over at Wikia.com where on many wikis people can earn various “badges” for different amounts and types of contributions and it sort of becomes like a game to try and improve your stats. But if the system is carefully designed, the incentive structure could reward good behavior and punish bad behavior in such a way that people would improve articles more and vandalize articles less. If it is badly designed this would just devolve into a popularity contest ruled by cliquish behavior and then either lead to groupthink or edit wars. But I have enough faith in the Wikipedia community to believe that Wikipedians would implement this as carefully and in as well-thought-out a way as possible, if an idea like this were implemented.

Anyway if the EPR thing were implemented in such a way that people trying to maximize their EPR score would, ipso facto, have to make a large number of positive contributions to Wikipedia and almost no negative ones in the process, this would certainly encourage good behavior and discourage bad behavior. And some people would do this simply to see how high of a score they could get, and would not need any more of a reward than that. For people with a low score, though, we would need to analyze the reasons why their scores are low and whether they actually did anything wrong or if the EPR algorithm is at fault. And assuming good faith, we would give them the benefit of the doubt unless there is evidence to the contrary. So I guess my support is not for this idea in its present form but I think this idea holds some promise if it improved upon and should not simply be discarded because of the flaws it presently has. Yetisyny (talk) 23:25, 28 July 2018 (UTC)

Oppose, see Grants talk:IdeaLab/Multidimensional EPR (Earned Public Reputation). --Havang(nl) (talk) 11:34, 30 July 2018 (UTC)

A reputation, usefulness or accuracy metric could allow better organization of the community. Tools are already implemented (thank contributors for specific contributions) and an automatic system of counting if someone has resolved controversies, added sources, made constructive criticism etc can be readily implemented. Yoconst (talk) 14:00, 7 August 2018 (UTC)

Expand your idea

Would a grant from the Wikimedia Foundation help make your idea happen? You can expand this idea into a grant proposal.

Expand into a Rapid Grant

No funding needed?

Does your idea not require funding, but you're not sure about what to do next? Not sure how to start a proposal on your local project that needs consensus? Contact Chris Schilling on-wiki at I JethroBT (WMF) (talk · contribs) or via e-mail at cschilling wikimedia.org for help!