Research:Finding a Collaborator

Contact

Hyunggu Jung

University of Washington

Sungsoo Ray Hong

University of Washington

Research:Projects

This page documents a completed research project.

Project Summary

This usability study will involve introducing Wikipedia editors to a new visualization tool within Wikipedia, asking them to do a set of short, scenario-guided tasks that the tool is designed to support, and soliciting their opinions about the tool. The usability study participants will be remotely invited to the shared Internet-based video chat program for these sessions, which will last approximately 60 minutes. Observing these Wikipedia users working with our tool and gathering their opinions about its design and functionality will help us refine the tool and provide insight into how such tools can support the standard activities of Wikipedia editors. The new visualization tool is designed to enhance searches of potential collaborators within a Wikipedia community by visualizing a "credibility" in Wikipedia. We expect that the tool enable people to make precise decision. The history of edits for each wikiproject will be used to visualize the level of editors. Research questions are as follows: How will our visualization tool enhance searches of potential collaborators? How will our visualization tool be used during the search process? Would this tool support users to find potential collaborators when editing Wikipedia? What changes to the tool would make it more helpful to users?

The purpose of this study is to understand how Wikipedia editors find their collaborators, and to measure the usability of a visualization tool for Wikipedia collaborator finder that enables users to know about potential collaborators and their status. This study will help us determine the nature of Wikipedia collaborator finding, and also give us how the usability of the tool can be improved. If participants decide to participate in the study, we will start to have a session via online video. This appointment will last about one hour. During the study participants will be asked some basic questions about experiences of the participants and their feelings about our visualization tool. And the participants will perform tasks that are related to collaborator finding in Wikipedia. The participants must be an experienced and registered editor to participate in the study. At any point the participants may pause or stop the study by simply letting the researcher know of their wishes. We will do everything possible to maintain the confidentiality of information of the participants. Videos, software data and survey data will be linked with a session number. Video recordings will be kept for no longer than five years. All study data will be maintained on password-protected computers associated with the research team. Session numbers rather than user names will be used to identify data files.

This study situates on the topics of credibility/trustworthiness with respect to the literature. The originality in this study that has not been covered in past research is that our new visualization tool represents four distinguished metrics to support decision-making process to identify potential collaborators. If successful, we anticipate the we will be able to understand behavior of editors in finding collaborators. In addition, we will also learn how the new visualization tool could be used in identifying potential collaborators.

Please refer to [1] to find the screenshot of the new visualization tool.

Methods

The basic structure of the usability study is described below:

Researchers will post on Wikipedia pages and in related electronic communication forum (Internet chat, email lists related to Wikipedia) an announcement of the study with an invitation for interested and qualified individuals to contact one of the researchers via email. The announcement will include information about who will be invited to participate in user studies with the researchers. When individuals voluntarily contact the researchers to indicate interest in participating, the researchers will verify they are active Wikipedia editors via publicly available information in Wikipedia. If individual is qualified, the researchers will contact them via electronic message (email) with a description of the study, information about their right to not answer any question(s) and/or to discontinue the interview-based study at any time without penalty. They will be informed that they must be at least 18 years-of-age in order to participate in. The message will also inform potential participants that the study will be recorded, but that their actual identities will not be used in any research report. The target population for this study is experienced Wikipedians. We expect that the minimum participant sample size is 15.

Qualified, interested parties will be contacted via email with an inquiry about their availability to use the shared Internet-based video chat program for a usability study session. This message will also include a statement about participating in the study (i.e., the voluntary nature of the study, their right to discontinue the session without penalty, their right to not answer any of the questions without penalty) and will inform them if they consent to these conditions, they should move forward with scheduling the session.

After an individual has agreed to the conditions of the study, the researchers will schedule a time for the session.The researchers will conduct the sessions remotely using the shared Internet-based video chat program. Participants will be given an overview of the usability study session, asked to click “ok” if they agree to give their consent to participants [2], asked to use our tool in Wikipedia to perform a set of common editing-related tasks, and asked questions about their perceptions of the tool and its potential value.

Upon conclusion of the session, the researchers will prepare notes about the session, using researcher-assigned subject identifiers rather than real names or Wikipedia user names. Only researcher-assigned subject identifiers will be retained in the study data set.

Dissemination

We will advertise the user studies via announcement on social media sites (e.g., meetup.com. Wikipedia), related electronic message distribution channels (e.g., IGQ) and by distribution of the announcement to acquaintances. Should the study be carried out and published, we will show the data/preprint of any research output made available as open access as per Research:Subject_recruitment.

Wikimedia Policies, Ethics, and Human Subjects Protection

Interested participants will supply their Wikipedia user name and email address. None of this information will be used as part of the research data to be reported. The user name will be used to verify that an individual is qualified to participate in the study and to determine the length and breadth of their experience as a Wikipedia editor. The email address will be used only for scheduling individuals for participation in the user study. The email addresses will be discarded (deleted) after the subject has participated in the study.

During the recruiting process, prospective participants can verify the identity of the researchers posting the recruitment message by communicating with researchers via email with organization domain (uw.edu). We plan to use a talk page to verity the participant eligibility, but we still need to collect personal information by email for scheduling individuals for participation in the user study as mentioned above.

We will not retain direct identifies (legal names, email addresses, telephone numbers) after the subject has participated in the study. However, we will retain their Wikipedia user name for 24 months after completing the study for the purpose of understanding user preferences and behaviors with the software. After 24 months, Wikipedia user names will not be retained.

This study has been approved by an institutional review board (IRB) in July 2013. The detail of the IRB approval is as follows: Human Subjects application #45502, “Collaboration visualization (Re:Flex) user studies” has been approved by the University of Washington IRB in Subcommittee EB under Expedited Categories 5 and 7. The Subcommittee has determined that this research meets all the requirements for approval outlined in 45 CFR 46.111. Principal Investigator: Mark Zachry. For more information about the IRB approval, please refer to [3].

Benefits for the Wikimedia community

This research will contribute to the knowledge of researchers interested in designing social media websites wherein patterns of participation are evident to the community of participants. It thus complements existing lines of inquiry in the Computer Supported Cooperative work and Computer-Human Interaction communities of researchers.

Timeline

Jun–Aug 2013: Implementation of the visualization tool
Sep–Nov 2013: Study design
Dec 2013–Feb 2014: Data collection and analysis