Talk:Wikimedia Fellowships/Project Ideas/Badges for Wikipedia Community Engagement

Great idea! Badges can create pathways for new wikipedia editors - making it easier for them to jump in and get their feet wet. I can also see badges being used to guide Wikipedia Campus Ambassadors through the education program training and through their interactions with faculty members and students. Badges are much more interesting and goal oriented than a list of check boxes.

Terminology: Badges vs Barnstars

I think it might be useful to explicitly address the difference of Badges and Barnstars. Barnstars can either have explicit requirements on being awarded, or not. Badges always have explicit requirements. Barnstars are WP specific and aren't easily shared while Badges are not WP-specific and are inherently easily shared. With those contours defined it would then be easier to profer an idea of how to 'grow' badges out of barnstars as one way of implementation in the WP community. Other than that, I love this idea and fully support it! Greg G (talk) 18:31, 30 March 2012 (UTC)

I concur with Greg G, but will note that there may be some fluidity here. I mean, in some ways, Wikipedia itself was designed as a proving ground for something more formal, and there may be some way of "certifying" (badgifying?) barnstars that seem to have a common set of requirements or understandings that have built up around them. It's important that barnstars represent a kind of "appreciation" over "approval"--and that such appreciation can come from "below" in some sense. It is not a country club model. It would be good if that kind of open generation could be used to shape badges. That catalog and evaluation of existing barnstars would provide a nice starting point. -- Halavais (talk) 17:50, 2 April 2012 (UTC)

Comments and suggestions

In general I like the idea of introducing more types of recognition to users for their volunteer work on WMF wikis, so I think developing a new recognition systems with the Teahouse is a good one that is worth trying. The badge system sounds interesting, but I think we should also see if our existing recognition programs can be better utilized, too. I have several free flowing comments and suggestions.

1. Look at the type of recognition given by the Wikipedia English WikiProject Military history. Although not exactly what you are suggesting, they have developed one of the most extensive recognition processes on WP English and I think it works pretty well for them. The link to the Awards page is here. And this page discusses recognition in a more general way Academy/Barnstars, Awards, and Honorable Mentions. Previously I worked with Kirill Loksin and Roger Davies, both Military History Coordinators emeritus, on the Arbitration Committee and know them to be very approachable and helpful people. They were active in organizing the WikiProject Military History into one of the most active projects on Wikipedia English. I suggest interviewing them about how the MILHIS awards program is working. Also look at the service awards for tasks such as March_2011_backlog_reduction_drive#Awards.
2. Also look at Wikipedia English's Service awards which are self awarded badges. Could the existing Service awards be integrated into the Teahouse? Maybe they are under used by people who would like to use them because they are not introduced early on to new users. Some of the service awards are eligible to be self awarded on day one, and again after one month.
3. Although these suggestions are geared for making volunteer activities successful for people with disabilities, the approaches mentioned seem like something to consider when designing any type of feedback methods on a large virtual project.
4. I need to do more reading about open badge projects that would possibly be affiliated with the Teahouse open badge system to understand how it would work. [1] [2] One area of concern would be privacy issues. At this point because I don't fully understand how it works, I have no idea whether it would be a problem or not but it is something that we need to consider going in. Because so many of the people at the Teahouse are new users who may not be aware of the privacy concerns of some of our long time users, I think we need to think through how this system would be integrated into the larger community even if it works with the newcomers at the Teahouse. All that said, don't think that I'm writing off the idea of linking with the existing off site open badge project.

I'll do more reading and leave more comments as something comes to me. Sydney Poore/FloNight (talk) 19:27, 4 August 2012 (UTC)

Thanks for posting the resources. I'm definitely going to need to think through how some of the WikiProject Military History and Wikipedia:Service_awards ideas could be used. I'll try to reach out to the project organizers and see what worked and what didn't. If you have any other examples that you think could fit in well, please post!

Privacy concerns for participation with the open badges project are definitely something to consider. The major issue that I see is that in order to push badges to the Open Badge Backpack hosted by the Mozilla Foundation, the system would need to know the email addresses of the users who get badges. The only way I can see it working is to give users a choice of disclosing an email address if they want their badges to be compatible with the Mozilla Open Badge Backpack (if they don't then they'd still get a badge but not necessarily one that's as portable). Other privacy concerns could be an ability to identify someone based on their other badges that they earn in other contexts if they end up sharing all their badges on some other public forum. I don't think that's as much of a problem when people have an option of being able to choose if they share that information and where, but definitely something to watch out for. Anya (talk) 18:31, 20 August 2012 (UTC)

Thoughts on your neat idea...

I'm really excited about this project and it raised a lot of thoughts, questions, and ideas. In no-particular, free form order...

• How are badges different than userboxes (self-given) or barnstars (given by others)?

Excellent question. In the context of the Open Badge Initiative (OBI), badges are something specific: a package of graphic, title, description, required criteria, evidence of work, and the identity of person/entity that gives out the badge. The type of giver (self, other, or automatic) isn’t important (though different givers may be better in different situations). By that definition they aren’t really that different from barnstars or userboxes, just may have more or less data depending on the use. OBI Badges are also portable and can be shared outside of the projects where they are earned (Facebook, Twitter and G+ currently).

In the context of this project, I have been thinking about making “badges” distinct from barnstars in the sense that they are the award earned for a lower class of achievements. They are given by others and not self-given. This is useful because that way we can keep barnstars as something kind of special and not worry about cheapening them (a particular danger with automatically awarded badges).

This has been a major point of confusion though. The current thinking is to perhaps change the terminology (and perhaps focus) to thinking about awards overall instead of badges in particular. Perhaps…

I wonder if 'badges' has 'baggage' that 'awards' does not. I personally like the idea of badges, in the Boy Scouts sense of a mark of skill-achievement you receive upon completion of a task. Award has a slightly different connotation like it is something 'winners' receive. I suppose this is ultimately just a matter of semantics and whatever is more appealing, less off-putting to the community is best. It'd be interesting to do some informal polling of the teahouse community to determine which one they find more intuitive and attractive. For example, 'badges' might be unfamiliar and localized to niche tech and scouting-type communities, whereas everyone knows what an 'award' is. Still, I love the idea of badges that show off your 'skills' and accomplishments.

• Who is empowered to give a badge?

Depends on the badge. You can have self-given badges, badges given by others, or badges given automatically.

• Is there a formal userright for that?

Maybe. Would be interested in hearing your thoughts on this. Wikiproject:Military History has the project coordinators award some of the badges. I don’t think that’s a formal userright as much as a responsibility of the role. I was thinking that for something like the Teahouse there could be hosts giving out badges to new editors and to each other. And project coordinators giving out badges to hosts. But this idea is very open right now…

• What's to stop someone from self-awarding a badge?

Honestly, it’s not about stopping people, but rather discouraging them from doing so. The idea is to surface information about the award that can make it easier for others to tell if it was self-awarded or mis-awarded (awarded to someone for work not done). Requiring evidence of work toward a badge and proof of who gave the badge would go a long way to identify people who are cheating at badges. The ideal thing is to present that information in a way that’s immediately obvious and creates a negative signal about a cheating user (for example, being able to see that a user gave himself all the badges ever, makes it easy to tell that he doing something fishy). I admit that the ideal could be difficult to achieve and there are trade-offs with making this practical/easy to use. That’s kind of the point of piloting, to see if we can make it happen.

It would be fairly simple technically to create a /subpage in an editor's userspace where badges were posted. It would also be easy to see if the badge had been added by another editor or by oneself. It does make sense that only WikiProject members or Teahouse Hosts would give out WikiProject Badges or Teahouse Badges respectively. I think my bigger concern is with ease of use than with hat-collecting. If people really want to scam the badge system, that's less bothersome to me than if efforts to prevent that discourage the giving of badges in the first place. It might be technically possible to 'lock' the /badges subpage so that only an approved bot could edit it; it also might be possible to prevent an editor from changing their own /badges page. That's something I'd have to ask around about. My guess is that it will be seen as more controversial than technically difficult. If so, it might be another case where it's best to just permit some 'cheating' in the interest of avoiding resistance to overall implementation.

• How will badge-worthiness be determined? Are there consistent tests/tasks? Or is it more subjective?

I’d like to see what Teahouse project coordinators and project participants think are some badges people would like to earn. I’d like to aim for a combination of different types of badges that award both consistent tasks and more subjective situations. Personally, I’m fond of badge challenges where people can self elect to go through a pre-defined set of steps to earn a badge. So for example, we can have a “Dip your feed into editing” challenge that people earn when they’ve made their first copy edit, their first reference, and added a first sentence.

This part of everything would get flushed out more as we go along. I’m really hoping to get more people’s input/thoughts on what badges would be cool to have.

• Can badges be integrated with the WikiLove delivery mechanism?

Maybe. Honestly, I haven’t thought about this. I’ll check it out. What do you think?

I think it would be fairly straightforward, although any changes to the core interface requires a direct developer support, code review, etc. which is always a challenge. In essence a badge is just a 'template', and the WikiLove mechanism already handles a variety of templates (barnstars are templates, for example). It would just be a matter of increasing the pre-approved, pre-loaded list of templates. Because of that it might even be the case that no significant alternations to the WikiLove code would be necessary, and merely an expansion of one of its included templates list. This is something to ask some Foundation people who work/ed on WikiLove. The strength of using WikiLove is it would semi-automate the badge-giving process. Searching for the proper template and having to place it oneself on an editor's /badges subpage would be about 3 times as complicated and involved. WikiLove already has the infrastructure in place to make it easy, so it would be a natural fit. I'm not sure if it would be too cumbersome an expansion to expect during pilot stages, but it does seem at least like a good medium-term goal.

• Re: linking of evidence, who will be responsible for searching for diffs? Is diff-linking too technically off-putting that it will discourage badge recipients? Is it too annoying that it will discourage badge-granters?

Could be too difficult. Could be alright. Depends on the badge and the people involved. Linking a "first created article" may not be that bad for example. Especially if people self nominate. I've also seen people do this with Barnstars sometimes. Check out User:Worm That Turned/Awards, you can see a handful of people linking to evidence. Very important questions to keep in mind as we do this though. There is definitely tension here with making badges easy to use while difficult to cheat.

Yup, and ease of use in my opinion, again, is more importance that cheating difficulty. Something to explore conceptually is 'what would motivate an editor to cheat and self-add badges?', 'what is lost by having some editors cheat on their badges', 'if the purpose of badges is to increase editor motivation (inward facing) as much as it is to signal abilities to other editors (outward facing), then is there any loss to the main purpose of badges even if some people do cheat?'

• Research has already been done showing that receiving positive encouragement (e.g. through Barnstars) does indeed increase future editor activity. In what way would this project build on, extend, or duplicate that finding? In other words, how would it be unique distinct from prior research? link: http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2012-04-30/Recent_research

Thanks for bringing this up! That’s really good research and gives more confidence to this whole badge idea because there is reason to expect that it’s worthwhile. My intention is to surface what makes a successful award system as well as take a chance to build upon what’s already there. There’s a lot of thinking around badges and awards that’s been going on and I’d like to see if we can make it work in the Wikimedia context. For example, it is possible that giving badges for the first 5, 10, 25 edits actually does nothing to encourage people to edit, or having badges on the project at all actually discourages new female contributors. Hopefully not, but it’s important to check.

Siko mentioned that Mako (not sure on spelling) presented at Wikimania some research that actually showed a decrease in editing after receiving a barnstar. I'd have to see that research myself, but my initial hunch is that barnstars are given after editing sprees and may very well coincide with a natural ebb in an editor's activity. I personally find barnstars very motivating, but I do typically receive them after a period of intense work, after which I naturally slow down. Teasing out causality vs. mere correlation there would be a next step.

• This is relatively brilliant: "For example, barnstars don’t tend to acknowledge work that is much less than exceptional. Barnstars don’t acknowledge someone’s first edit or first full article (though these can be great accomplishments for new editors). As pinnacles of achievement barnstars don’t show a path to becoming a better contributor and poorly recognize progress. And they are not always consistent. Barnstars may not always be given out regularly to all contributors doing equally exceptional levels of work." I LOVE the idea of recognizing regular, non-exceptional achievement, which is in the context of Wikipedia's challenging editing environment, still pretty exceptional.

Thanks! One thing we can do with badges is to help surface where to start and how to move forward. They can serve both as awards and as guidance for how to become a better editor. This is why I’d like to think more through badge challenges…

I love the idea of editors having a loose map of what types of skills are 'out there' to develop. Right now a new editor not only has no idea how to edit, they don't even know what skills are involved in doing so. Badges could both incrementalize and concretize that vague and intimidating process, providing as you said, guidance as we well as motivation.

• How does this idea scale? **Can it be automated**? (e.g. A bot could recognize a person's first edit to their userspace, first edit on another person's talk page, first edit to an article talk page, first edit to an article, first edit to a help desk or noticeboard, first edit... a bot could also give badges based on edit count milestones, time as an editor milestones, addition of a reference, use of {{ }} templates, addition of a wikilink, etc.) I'm liking this more and more when it overlaps with automation.

Automatic is good for scaling but can also cause problems because it’s hard to control for quality. I think this is where some of the “hat collector” considerations are coming from. Theoretically someone could go through and get badges but do a really poor job of whatever edits they’re doing. In this way we could end up using badges to encourage bad editing. :/ Something to keep in mind… I think we can still work with automatic badges but can’t be overly reliant on them either.

Scaling is definitely a good question though. Do we create a badge system for everyone on all Wikimedia projects or do we enable projects to create their own award systems and just support their work? Personally, I'm thinking the latter and this fellowship/pilot project are a way to surface some information about what works, what doesn't, and any best practices that projects could adopt. I imagine different projects having different badge collections and ideas that are relevant to their particular project. I'd appreciate your help in helping to think this through though. How does this model work with automated badges? What would "support" look like? Is there a better way to scale?

I think there would be room for both. You could have universal badges for skills like a first edit to a talk page, responding to a comment, etc.--those ideally would be awarded automatically by a bot. But then you could also WikiProject specific skills that would be manually/semi-manually given. Because there is no technical difference between a badge (template) that's given by a bot and a badge (template) that's given by an editor (or the WikiLove mechanism), there's nothing I can see preventing a dual approach with both automatic and manually given badges.

• A fantastic proven model for badges is StackOverflow. Look closely at what they do and how they do it. It's automatic, clever, motivating, and well-executed.

Good point. I’ll take a look at StackOverflow again and refresh my knowledge of it. Would you happen to be someone that’s earned badges on it? Would be really interested in learning more about that experience. If not, I’d still be interested in learning what you think make’s their badge system design clever, motivating and well-executed?

Another good example is P2PU. They award badges for completed badge challenges.

I was very active on the http://cooking.stackexchange.com website (stackexchange is the overarching structure in which stackoverflow is just one topic-focused site). You can read stack overflow's blog post about badges here: post.

I loved several aspects of stackexchange's badge system.

1. Badges were given early and often. There were badges for all types of firsts.
2. Badges had neat and somewhat creative names. For example, rather than the 'you answered a question' badge, it was called the 'Teacher' badge. Similarly, on Wikipedia you could come up with clever names for badges.
3. Several Badges had 'levels', bronze/silver/gold, that escalated with a higher level badge replacing the lower level badger . So for the simple example of time registered on the site, you might get a bronze badge for being a 'Month-ling', a silver badge for being a 'Year-ling', and a 'Veteran' badge for being a 5-year member. There was somewhat of an exponential increase in difficulty with these levels. Bronze badges were very accessible and provided quick recognition to newcomers. Silver badges were something to shoot for after a considerable but not discouraging investment. Gold badges were significantly more of a reach than the difference between Bronze and Silver; they truly marked someone who made a distinctive and excellent accomplishment.
4. Badges were largely 'discoverable'. You were given them automatically and not immediately told what all of the available badges were. Instead, as you played around on the site, you would just 'wander' upon an achievement that was awarded with a badge. The badges themselves educated you about what was possible and what you might have to do next. This might not be ideal for Wikipedia, where the purpose of badges is partly to inform editors what they should do next. In that case, Stackexchange did have a badge overview page where all badges and their descriptions were listed. Here is the cooking stack exchange badge list: list and the stack overflow list: list.
5. A nice feature of badges was that some of them you could earn more than once. So some badges had a number next to them as in 'Teacher x6' meaning that you had accomplished the Teacher requirements 6 times.
6. Another feature of badges that was appealing is you could see other users who had those badges. A simple way to do this on Wikipedia is with the Category system. It's possible when creating a template to have a category added to the page. For example the 'Year-ling' category would be 'Category: Editors with the Year-ling badge' and clicking on the category would show you all other users with that badge. A nice benefit here would be to identify other editors who have skills that you might want to employ. If you wanted to find a 5-year editor who had achieved the gold badge of template work, you could conceivably just cross-reference two categories and come up with a list of all the veterans who had advanced template skills. So one novel use of badges beyond inward facing motivation and outward facing reputation would be interfacing discovery of editors who are well suited to collaborate on a specific task. This is a major challenge on Wikipedia, finding people who know how to do what you need help with, and a Badge category system would be one way to simplify that.

• Teahouse is an IDEAL testing ground for initiatives like this. Are Stierch and J-Mo on board? Will there be a control group 'within' the tea house or will the comparison be to non-teahouse users? If the latter, how can you distinguish between benefits of the Teahouse and benefits of the badges alone. Okay... you're planning to use pre-badge Teahouse participants as a control group. That would work, especially if Teahouse procedures are otherwise relatively consistent over the two periods. An even more rigorous trial would compare four groups. No teahouse/no badges, no teahouse/yes badges, yes Teahouse/no badges, yes Teahouse/yes badges. In other words, maybe badges only motivate the types of people who already joined the Teahouse. To get a true sample and have a valid measure of whether or not this will scale, you need to try it a bit outside of the Teahouse, out in the 'real world'

I’ve talked to Jtmorgan about testing badges on the Teahouse and he’s suggested something more-or-less similar to what you have. It seems to be the right way to go about things. We’ll need to think through this more and flush it out.

Awesome. I'd love to talk with Jtmorgan about the automation aspects here (we have a Skype scheduled for next week). Rumor has it he's becoming quite the handy coder. I also have a pretty good connection with some key members of the Bot Approval Group, and could solicit some useful technical feedback about the feasibility of automation.

• What's the Open Badge Backpack, and why haven't I ever heard of it before? You mean to tell me there's an open-source, transferable badge application? I'm intrigued.

Yup. It’s a pretty neat idea with a lively community. Check it out:

• Open Badge Backpack
• Openbadges Google Group
• Openbadges official site

The idea of transferable badges is very cool, but it might raise the level of importance in cheating-deterrence. I think the main focus of badges should be communicating within the Wikipedia community, so at this point I'd frankly be less worried about the transferable aspect.

Great work so far. I'm looking forward to seeing this happen. Ocaasi (talk) 17:36, 23 August 2012 (UTC)

Thanks! You’ve definitely got me thinking deeper in more directions. This is great! Looking forward to more of your thoughts. Anya (talk) 21:10, 24 August 2012 (UTC)

I've responded inline above. Also, I've sent you an email: would love to Skype this week. Cheers! Ocaasi (talk) 18:46, 25 August 2012 (UTC)