Research talk:Committee
Addendum to Charter
editI would like to suggest two additions/expansions to the charter.
- Set guidelines for Wikimedians creating and using statistics. I've seen things as problematic as graphs with the vertical scale truncated because an accurate graph didn't convey the message that the author wanted.
- Set guidelines for use of aggregation and research to single out wikimedians. For example User:Emijrp/List of Wikimedians by number of edits has an opt out mechanism for wikimedians who don't want to be listed.
WereSpielChequers 14:40, 26 August 2010 (UTC)
- These are good points -- be bold :-). Regarding the first point, I'd also love to see more code sharing, more centralized collecting of stats, etc. --Eloquence 17:46, 26 August 2010 (UTC)
- I still want to make my point which I initially started to discuss on the webpage. What would be the appropriate place: here or the mailing list?--Yaroslav Blanter 18:14, 26 August 2010 (UTC)
- I would suggest the list to ensure that it reaches all members.--Eloquence 18:17, 26 August 2010 (UTC)
- I agree that the list is the better place. ---- Daniel Mietchen 19:16, 26 August 2010 (UTC)
- Feeds of this talk may be useful, anyway. --Nemo 20:47, 26 August 2010 (UTC)
Nature of the committee
editWith regard to wmf:Resolution:Wikimedia Committees, I've categorized this committee as a staff committee. Correct if I'm wrong. Thank you, Nemo 20:46, 26 August 2010 (UTC)
- That's correct, thanks!--Eloquence 17:36, 27 August 2010 (UTC)
Experimental research
editIn addition to permissions for non-public data, the committee might want to consider developing policy around permissions for research involving experimental manipulation without prior participant consent (e.g. hoaxing, spamming, & baiting), limiting the nature and extent of such manipulations. I realize requiring permission is unenforceable since anyone can edit, but an informative policy statement could guide well-meaning researchers toward better experimental design. Just a thought. ~ Ningauble 17:57, 31 August 2010 (UTC)
- I agree with this--Yaroslav Blanter 19:42, 31 August 2010 (UTC)
- I agree, though as its initiator I'd question your example of "baiting". WereSpielChequers 11:42, 3 September 2010 (UTC)
- That is a fair criticism. I confess I am often guilty of inapt word choice in the pursuit of succinctness.
That particular experiment might have benefited from a more rigorous design than inviting people to invent their own test cases (or "bait", in a manner of speaking). As with all intrusive methods, one of the first questions should be whether the question can be researched by non-intrusive means instead, e.g., by surveying all new pages in a certain period of time. ~ Ningauble 13:59, 3 September 2010 (UTC)
- Hi Ningauble, I see where you are coming from with your comment about research guidelines having a preference for non-intrusive methods, but on wiki that can be seen as a breach of EN:WP:SOFIXIT. We already have tension between people who tag articles in the hope that others will fix them and those who actually fix articles, so someone who trawled through some articles and calculated how many had problems but did nothing to fix or identify them would IMHO risk annoying the community. As for using NEWT as an example of baiting, well to be fair I would accept that there were at least a couple of out of scope test cases in that project; Either the article in question did actually meet the criteria for deletion or in at least one case it was what I'd call baiting, as it arguably met the criteria for a new article but looked like a clear case for deletion. I suppose the analogy would be with policewomen and kerbcrawlers. A kerbcrawler arrested for harassing an off duty policewoman out jogging could have harassed anyone. A kerbcrawler arrested by a policewoman dressed as, and behaving as a hooker may well have been baited. I think it can sometimes be difficult to draw lines between mystery shopping, baiting and breaching experiments, and I'd caution that all three are likely to be contentious. Breaching in particular needs some sort of community authorisation, I believe the foundation ran a breaching experiment some time ago with the IT guys trying to crack admins passwords, so that sort of breaching experiment can be acceptable. I'm also aware of a breaching experiment whereby a group of unwatched BLPs were edited to add incorrect information such as about other people of the same name, I'd use that as an example of a breaching experiment that the community regarded as vandalism. WereSpielChequers 11:17, 4 September 2010 (UTC)
- That is a fair criticism. I confess I am often guilty of inapt word choice in the pursuit of succinctness.
Request page
editI think it would be helpful if we had a research requests page on similar lines to en:Wikipedia:Bot_requests. Do people agree? WereSpielChequers 18:11, 31 August 2010 (UTC)
- In principle, yes, but we need first to come up with some policies, otherwise it is not so much clear how we are going to handle the requests--Yaroslav Blanter 19:27, 6 September 2010 (UTC)
- Thanks, and yes I agree we need policies first as to what sort of research can be done and how to interact with the community. WereSpielChequers 20:13, 6 September 2010 (UTC)
Expert barriers
editShould this survey be mentioned on this page? Expert barriers to Wikipedia -- Jtneill - Talk 12:01, 13 February 2011 (UTC)
- The survey has a subpage of this one, at Research Committee/Areas of interest/Expert involvement/2011 survey, so I don't think it is necessary to put it in here, but since there is nothing wrong in having it either, I just put up a banner. -- Daniel Mietchen 12:20, 13 February 2011 (UTC)
Application
editPlease consider my application as a member of the Research Committee
Claudia Koltzenburg User:Koltzenburg is a literary and cultural studies scholar by training. Current focus of research: The governance of the Commons today and what Open Science arguments can contribute to render the effects of commodification on academic research less detrimental (Philosophy of Technology, short profile). Activities have included (in alphabetical order, most of these are current): computer science bookseller, festival co-organizer, independent researcher, international NGO joint manager, legal news writer, LGBTIQ human rights activist and author, managing editor of an international medical journal, musician, open access/open data and wiki advocacy, pipet calibration, project management. Wikipedia contributor since April 2005. --Koltzenburg 11:23, 14 April 2011 (UTC)
Exposure of the Committee
editHi, I am attending CLEF 2011 and I see people use wikimedia material for research purposes. I can see how the result(s) of such research may be of great benefit to wikimedia projects. Also people seem to be confused as to how to gather information for research purposes such as downloading a set of images in bulk or even aware of the existence of our archive of deleted images which aren't really deleted.
Also I noticed how projects like Yahoo! Research is awarding small prizes (like 500 euros) to promote research on the field relevant to them. Such small grants could be of benefit in gathering more attention to problems we could solve through research.
Userbox
editI created the userbox {{RCom member}}, which is right now on meta, and if everybody agrees with the design I will also create a similar one in enwiki. If we have a better logo, suggestions are welcome.--Ymblanter 19:24, 16 November 2011 (UTC)
- I like it! Drdee 20:00, 16 November 2011 (UTC)
- Nice one, I made a few tweaks. As for the logo, I don't like the one used on the RCom page (inherited from the former research team/network), I'd rather stick to the current one (Meta). --DarTar 21:10, 16 November 2011 (UTC)
Analysis on quality
editI've put down some analysis/recommendations on Wiki quality. Basic issues is the failure to get high quality on our high view/high importance articles.
One analysis by Gorbatai uses server data to get a weighted average to see what quality the reader experiences. Only 3% see a GA+ article, only 31% a C+. Seems basic, but I think this is the first time crunched this way. And I think % of reader views is the lens we need, not % of articles.
There are also some capsules on patterns of writing and the like that may be interesting.
Except for three cited analyses, all the work is mine. And all of the implications, recommendations, etc. (or if the language is too strong)...that all comes from me.
See here: PowerPoint: Wikipedia's poor treatment of its most important articles
Request for researcher permissions?
editHave any policies been set yet on who can request researcher permissions and how to go about doing so?
--Varnent (talk) 23:21, 16 September 2012 (UTC)
- There is no such thing as researcher permission. Research projects requiring subject recruitment usually go through the review process.--Ymblanter (talk) 18:17, 19 September 2012 (UTC)
- The question may be referring to the "Researcher" user group that was introduced on the English Wikipedia in 2010. Regards, Tbayer (WMF) (talk) 21:38, 19 September 2012 (UTC)
- Yes - that is what I was wondering about. It would be helpful for some folks who do not need or want admin level permissions to have access to things like deleted articles. --Varnent (talk) 19:16, 4 October 2012 (UTC)
- There is a user permission called researcher in EN:WP. I spoke with user:NativeForeigner and user:Shararihareswara about this. So, what are the parameters of the user group? The page is skimpy on information. Geraldshields11 (talk) 20:37, 1 June 2014 (UTC)
- Yes - that is what I was wondering about. It would be helpful for some folks who do not need or want admin level permissions to have access to things like deleted articles. --Varnent (talk) 19:16, 4 October 2012 (UTC)
- The question may be referring to the "Researcher" user group that was introduced on the English Wikipedia in 2010. Regards, Tbayer (WMF) (talk) 21:38, 19 September 2012 (UTC)
RCom
editHi. What are you guys? Has the thing died completely? --Base (talk) 17:56, 17 December 2014 (UTC)
Seconded. The page has had no content update since 2013 and the committee apparently has not met since 2011. What does the committee do now, and how does it do it, please? On a related note, is there still a wmf:Chief Research Coordinator? Rogol Domedonfors (talk) 21:03, 29 July 2015 (UTC)
Research protocol review guide for layman reviewers
editIn medical research on humans (clinical research) there is a trend in the field to get layman audiences to review the medical protocols. This is always problematic because layman audiences need a lot of training to be able to give useful comments on medical research. Because of the difficulty, various organizations continually make more resources to coach anyone to review research procedures. In the past, almost none of these resources are published online or available to be shared just because of the secretive culture of medical research.
Just in the last week an organization called Community Research Advisor's Group published the Protocol Review Companion for Activists to give layman audiences suggestions for how to review a research protocol. While medical research review is beyond the scope of most of the human subject research which happens on Wikipedia, and this guide makes presumptions of risks that will not happen with social research on Wikimedia projects, I still think that community organization to create some standards to be checked for all human subject research on the Wikipedia community would be ideal.
I have raised this in the past and gotten a bit of pushback. I want to state again that my own standards for review are extremely low, but I do advocate for minimal standards especially with regard to transparency and creating a channel for public communication in case of problems, and that I do feel that some research which has come to Wikimedia projects has created community risks and burdens. I feel that having standards in the Wikimedia community would both protect the community and make researchers respect the community more and take their work more seriously. Blue Rasberry (talk) 13:02, 16 January 2015 (UTC)
Meetup at Wikimania 2015?
editIf any R-Com members or fans are going to Wikimania in Mexico then I would like to meet them on Wednesday or Thursday during the hackathon before the conference. Blue Rasberry (talk) 18:17, 7 July 2015 (UTC)
- Good idea - would you like to make a list on the Wikimania-wiki? --Ziko (talk) 19:48, 7 July 2015 (UTC)
- Bluerasberry and Ziko +1. Will you both be at the Wikimania Hackathon? --Halfak (WMF) (talk) 20:59, 13 July 2015 (UTC)
- I got sidetracked. Yes, I wanted to schedule an event for the hackathon. The new thing is that "hackathon" is a developer event and it is part of the "preconference" so the two should not be confused, and will have separate schedules. Next year there will be an RCom meeting during the preconference, as well as other working group meetings. I had problems scheduling anything during the first two conference days this year.
- No worries - everything resolved now. Talk soon. Blue Rasberry (talk) 14:59, 4 September 2015 (UTC)
- Bluerasberry and Ziko +1. Will you both be at the Wikimania Hackathon? --Halfak (WMF) (talk) 20:59, 13 July 2015 (UTC)
Interaction with research participants at the end of a study
editWhen a researcher in a Wikimedia project is doing user testing, and requests the time of Wikimedia contributors, they should seek to use as little of the users' time as necessary, and those users should be informed first about big events like the end of the user data collection. This seems to not have happened in the case of Flow.
mw:Flow is a Mediawiki extension for discussion. Some Wikimedia communities were asked to support research by using the Flow extension. This is great - I am glad when researchers recruit human subjects to get user data. When communities agreed to test the software, it burdened them a little because the software did not work. It is nice when volunteers agree to do these things.
Development of Flow was recently halted. I am sure that was a thoughtful decision. This was reported at en:Wikipedia:Wikipedia Signpost/2015-09-02/News and notes and in the WikiTech mailing list.
What seemed less thoughtful was Flow researchers' response to volunteers who agreed to test Flow so that the developers would have user testing data. I am not aware of any notice being given to people who were testing Flow to inform them that user data is no longer requested. No announcement has been made to give instructions for un-installing Flow where it is being used, if the volunteers no longer wish to participate in the research.
In the future I would like to see more consideration given to users who make sacrifices to provide data for studies, especially when the WMF is doing the research. With development halted, it makes me wonder when the user data ceased to be important to collect. Was user data no longer useful just now, or was it recently, or was it months or a year ago? If volunteers are recruited for research, then someone needs to track them continually and let them know when the research has ended. When the research is over, if there was any intervention done like a software change, then there should be an offer to put things back the way they were before.
Failure to show appreciation for volunteer contributions decreases enthusiasm for participating in research. There are research standards and a culture of research participant management which set precedents for how human subject research should happen. When researchers recruit participants for an intervention then human subject research is happening and standards should be kept. Blue Rasberry (talk) 14:59, 4 September 2015 (UTC)
- I understand your concern, Bluerasberry, but I think you might be misinterpreting the recent announcement by DannyH (WMF) about the future of Flow, as well as the current and future status of test-deployments. But I'll leave the clarification up to the experts. Also pinging Mattflaschen-WMF, who is also qualified to provide an answer. Cheers, Jmorgan (WMF) (talk) 16:14, 4 September 2015 (UTC)
- Bluerasberry, you are right, and I really apologize. Flow is still being maintained and supported, and I've been talking with the communities that have been using Flow about what they want to do. But it's been so long since I talked to you and the other people at WikiProject:Breakfast and WikiProject:Hampshire that it slipped my mind. That was foolish and disrespectful of me. I'm really sorry about that. We can archive the Flow page on WP:Breakfast, and move the archived wikitext page back, if that's what you want to do.
- As for the user research, what we found in brief is that Flow is working well for forum-style group conversations. New users find Flow easier to use, and there are several wiki communities that have been very enthusiastic about using Flow on user talk pages, help pages and general Village-pump-style discussion spaces. But the discussion features aren't suited for the more complex Workflow processes, so the team is halting development on discussion features and focusing specifically on supporting Workflows. All of the testing and feedback we've gotten has been very helpful in the development work, and it also informs the work that we're starting on Workflows.
- But I should have talked to you about that earlier, and offered the chance to keep Flow or change back to wikitext. Again, I'm sorry. DannyH (WMF) (talk) 17:26, 4 September 2015 (UTC)
- Totally cool - consider this resolved. I am very happy with interaction between WMF developers and the community, and only want good relationships to get better. I would support future user research if requested anytime.
- The relationship is between the Flow team and the WikiProjects, and personally, for as long as the Flow team wants some user data then I support the continued placement of Flow in community spaces. I only want to avoid situations where there are hanging Flow installations which no one is watching. If someone on the Flow team is continuing to get user data from these places, then I like supporting the researchers with user data.
- Thanks a lot for the history of regular, good communication. The team should be proud. Blue Rasberry (talk) 18:23, 4 September 2015 (UTC)
- But I should have talked to you about that earlier, and offered the chance to keep Flow or change back to wikitext. Again, I'm sorry. DannyH (WMF) (talk) 17:26, 4 September 2015 (UTC)
I am requesting WMF funding to make a paper open access - also, there should be an open access policy for funding
editI need money to make a Wikipedia research paper open access.
I would appreciate comments, criticism, and endorsement of my request.
A bigger issue is deciding the circumstances under which the WMF grantfunding process should provide funding to researchers to make their Wikipedia-related publications open access. I would work with anyone to draft a policy in the future. Perhaps my grant request could help start the discussion. If anyone has comments, please share. Blue Rasberry (talk) 15:16, 3 February 2016 (UTC)
Any interest in re-establishing a research committee?
editThis group went defunct in perhaps 2012, and was marked as defunct in 2016.
Things have changed. Wikipedia research is hot. There is no particular place where researchers can connect to Wikimedia community volunteers for research support.
Does anyone have interest in re-establishing a wiki research committee? Here is a proposal for doing so:
- Get at least 5, and any number, of people to sign up to watch a research noticeboard here on Meta-wiki
- Meet periodically, for an hour twice a year at least, to maintain administrative rules
- Perhaps adopt the old rules from this group to start
- No promises or commitments, but we try out using a committee page and talk page as a place for researchers to connect with wiki community members for questions and guidance
The problems that the committee would address includes the following:
- Researchers can do good research with a little help finding best practices
- Having a dedicated channel to connect researchers to a wiki community group is helpful since so much wiki research depends on community support and consent
- Researchers get confused about the differences between the Wikimedia Foundation research staff and the Wikimedia community of users. A page for the community would be helpful, as there is none now
- The community could start making requests of researchers, such as requests to publish results with open licenses and requests to inform community stakeholders of result availability
- Community members can help prevent certain problems, like researchers excessively contacting and fatiguing certain demographics of wiki community members
Bluerasberry (talk) 22:11, 30 April 2024 (UTC)
- There is no particular place where researchers can connect to Wikimedia community volunteers for research support. - actually the Wiki-research-l mailing list still exists and has proved quite useful for that, if researchers manage find it.
- That said, there surely are improvements to be had on the goals/problems you are listing. However, before reviving RCom, I would recommend to get a better understanding of the reasons why the old RCom failed, to avoid these issues in a new iteration. I'm pretty sure it was not simply because Wikipedia research was not cool enough back then. Things have changed. Wikipedia research is hot. - "changed" and "hot" meaning what exactly? (There weren't fewer new research publications about Wikipedia back in 2011 than in recent years, for example.)
- Regards, HaeB (talk) 05:12, 2 May 2024 (UTC)
- @HaeB: Thanks for reacting.
- One difference is that now the Wikimedia Movement raises and spends US$1 billion every 6-8 years (annual budget of about US$170 million) whereas in 2011 by financial reports the annual budget was about $17 million and there was no certainty of 5-10 year planning. Another difference is that aside from any money in the Wikimedia Movement itself, AI researchers and other outsiders in the wiki movement value wiki activity and content in the billions of dollars for economic value. Most Wikipedia money is invested outside the movement now, whereas 10 years ago, Wikipedia was not formalized as part of the global tech infrastructure. I think money in the ecosystem is more indicative of changed and hot than counting papers.
- The mailing list as a certain tone and permits certain activities. Benefits of a committee include
- Asserting authority beyond that of a mailing list post
- Applying wiki values, like transparency, rule development, and democracy, to the establishment of research ethics
- Diverging Wikimedia community ethics and values from those of the Wikimedia Foundation, as the two groups have distinct interests.
- Gaining normality, because conventional and normal billion dollar projects run their research through named humans rather than through random pseudo-anonymous internet personalities through a list serv. We should be operating in a way that can be compared to a research institute, university, advocacy org with 100k members, or tech organization, and it would be challenging for any researcher to approach our current way of doing things and recognizing any of it.
- Catching up with external investments. There is no one in the Wikimedia Foundation who has ever sought to encourage research partnerships at scale, and no one who counts or recognizes how many people are paid professionally to research Wikipedia. There are 1000s of people who have been paid and consumed research budgets to do Wikipedia, and the usual experience is that they get no support or acknowledgement. This is not the best list, but for example, in d:Wikidata:WikiProject_PCC_Wikidata_Pilot/Participants 50 universities sponsored staff Wikidata engagement which was connected to research libraries and certainly brought wiki research conversations into universities. Comparison: Meta (Facebook) gets billions of dollars in revenue, but the real value of Meta is that organizations spend 10x that revenue designing content to publish in Meta platforms. Similarly, although not many orgs give money to WMF to do research, a lot of orgs spend lots of money - perhaps 5% of the annual Wikimedia Movement budget - sponsoring researchers to do wiki research with which we could be interacting.
- If we set up a zero budget research committee which offered nearly no useful services, then I still think there would be demand from researchers and consumers to have any kind of interaction with such a committee in comparison to the status quo of not achieving an interaction at all. Benefit to wiki community is promoting research quality and ethics, and benefit to researchers access to whatever conventional research services the committee chooses to provide.
- There are 100 possible services to provide. If there were a committee, it could choose to start with providing 1-3 of those. Whichever services a committee chose, that would determine how a new iteration would differ from the 2011 iteration. Bluerasberry (talk) 15:12, 2 May 2024 (UTC)