Learning and Evaluation/News/Pre-conference for Wikimedia Conference 2016/Measuring and Evaluating Impact

Tools Rotations: Key tools for grants and evaluation

Survey Questions

Session Abstract: Writing survey questions is not as easy as it looks, even with tools like google forms to help you. This session will review 8 tips you can do to improve your survey questions and you will get a bit of practice as well.

Specific learning objectives:

• Learn 8 tips for writing good survey questions
• Learn resources for writing surveys related to wikimedia work

• Amount of time required: 20 minutes
• Potential participant collaborators: Jaime Anstee

Teaching Lead:
Edward Galvez

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)
2. --Mglaser (talk) 08:09, 19 April 2016 (UTC)

Logic Models

Session Abstract: Each year, volunteers and program leaders of the Wikimedia movement plan and execute a great diversity of program activities. But how do we know that our program activities are likely to achieve the results we hope to see?

Logic Models are a very valuable tool for the planning of our program activities and for developing appropriate evalution strategies. Based on the distinction between the ‘outputs’ and the ‘outcomes’ of our work, they can be used to carefully think through the links between what we are doing and what we want to change by this. In this rotation session we will shortly recap the concepts of outputs, outcomes and impact, practice to set up a simple logic model and discuss how this tool can be used and trained at the participants’ home affiliations.
Specific learning objectives:

• Participants improve/ revive their understanding of basic logic model terms like outputs, outcomes, impact.
• Participants feel more practiced in setting up a simple logic model for a concrete program activity
• Particpants get first ideas how to train and use logic models at their home affiliations

Amount of time required: 20-30 minutes
Potential participant collaborators:

Teaching Lead:
Christof Pins

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

Outcome Mapping

Capturing Social Change Through Outcome Mapping

Session Abstract: Outcome Mapping is an evaluation approach developed in 2001 for evaluation which places people as the center and looks at changes in behavior to define outcomes. Outcomes are defined as “changes in behavior, relationships, activities, or actions of the individuals, groups, and organizations with whom a program works directly” as well as indirectly with “those potential partners with whom the program anticipates opportunities for influence.” (Earl, Carden, & Smutylo, 2001, p.1)

In the Wikimedia world of programs, outcome mapping strategies can help us to gather data on the contributions that our programs make in terms of bringing about changes in our partners around the world that help to build toward greater engagement in open and free knowledge. Further, outcome mapping can also help us to evaluate both the intended as well as unintended results of our innovative Wikimedia programs. In this way, outcome mapping can help to get beyond the more direct program outcomes to deeper environmental outcomes to measure a program’s contribution to complex change processes.

• Because it allows for qualitative outcomes and stories of our projects and programs and helps to better surface our shared impact.
• Because it provides useful framework for getting beyond - to measure those intermediate and longer-term qualitative outcomes that influence higher-level systems change in the socio-political environment.
• Because our projects seek deeper changes and impact on the world than may be directly linked to our own direct project environments.

Specific learning objectives:

• Participants will learn two core outcome mapping strategies to help identify the social influences related to their programs.
• Participants will learn of next steps for learning more about how to apply outcome mapping in their evaluation work.

• Amount of time required: 30 minutes
• Potential participant collaborators:

Teaching Lead: Jaime Anstee (WMF)

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

Global Metrics Magic Button

Session Abstract: Participants will learn how to use the new Global Metrics magic button, that collects four global metrics using one report with four inputs.

Specific learning objectives:

• Participants learn how to use the "magic button"

• Amount of time required: 10 minutes
• Potential participant collaborators:

Teaching Lead(s):
Sati Houston

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

SMART targets

Session Abstract: SMART is a framework for creating targets (or "measures of success" as it's known in PEG, or "objectives" as it's known in APG) for programmatic work; it's a framework that the Wikimedia Foundation grant teams use to help grantees turn generic objectives / measures of success into focused statements, that specify the action and expected outcomes. This session will focus on clarifying terminology (such as What is an "output" vs. "outcome") and reviewing the SMART framework used by the Wikimedia Foundation grant teams.

Specific learning objectives: Participants understand SMART terminology and how to create SMART targets

• Amount of time required: 10-20 minutes
• Potential participant collaborators:

Teaching Lead(s): Sati Houston

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

Category Tools: What to use?

Session Abstract:

Will provide a quick rundown of the existing category assessment tools, and help solve project specific requests about how to use these tools:

• BaGLAMa 2
• GLAMorgan
• Treeviews
• PetScan (successor to CatScan)

Also, will briefly discuss other backup tools for generating lists and grouping of articles.

Specific learning objectives:

• Identify when and how to use different tools available for assessing content on the projects

• Amount of time required:10-20
• Potential participant collaborators:

Teaching Lead(s):

Maria Cruz
Alex Stinson

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

Programs and Events Dashboard Overview

Session Abstract:This session will present the Programs and Events Dashboard as a program management tool for Wikimedians interested in organizing, running and tracking groups of users in various contexts (education programs, writing contests, editathons and workshops, etc.). This tool is still in development, and participants can give feedback or help file bugs.

Specific learning objectives: Participants understand what the Program and Events Dashboard is and how it can be used in the future for their programs.

• Amount of time required: 10-20 minutes

Teaching Lead(s):
Tighe Flanagan

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

Phabricator: A place for communicating about tools and tech

Session Abstract: How can people let developers know about the tech issues they're having and the features they'd like to suggest? Through Phabricator is the answer. Putting things in front of the people who can actually do something about them is easier than you think.
Specific learning objectives: Participants understand what Phabricator is and how to file a basic task or to find one.

• Amount of time required: 10-20 minutes

Teaching Lead(s):
Erica Litrenta

Interested participants:
Add your signature below

1. Poco a poco (talk) 10:56, 17 April 2016 (UTC)

Data Rotations: Quick & Dirty tips and tools for data use

Quantitative Analysis Tools/Strategies

Session Abstract: Have a lot of data but not sure what to do with it? This session will review easy ways to analyze your quantitative data. First, we will review the different types of data and what kind of basic analysis works best for each. Then, we will go over how to present data in a report. This session will not review graphics (See Data Presentation below!)

Specific learning objectives:

• Learn about different types of data
• Learn how to present quantitative data in a report.

• Amount of time required: 20 minutes
• Potential participant collaborators: Jaime Anstee

Teaching Lead(s):

Edward Galvez

Interested participants:
Add your signature below

• Flixtey (talk) 20:46, 15 April 2016 (UTC)

Qualitative Analysis Tools/Strategies

Session Abstract: Learn a few quick tips and tools for making sense of comments and feedback gathered from open-ended questions and consultations through qualitative coding and word clouds. This rotation will briefly overview a few strategies to turn long form text data into meaningful information and visual transformation moving from basic to advanced word clouds in 3 basic steps:

Basic Word Cloud Generation: Use a word cloud to look at the raw word frequency of your text response data.

• Step 1. Data Cleaning: Learn some key tips and tricks to get your data in shape for sense-making.
• Step 2. Thematic Coding: Learn some quick categorization techniques to help organize related responses.
• Step 3. Affective Coding: Learn a tips for associating meaning to ambiguous response terms.

Advanced Word Cloud Generation: Use category codes to weight words and phrases in your word cloud.

Specific learning objectives:

• Participants will understand the value of processing qualitative data for evaluation and decision-making
• Participants will understand three qualitative analysis strategies and how they work together to share qualitative data more easily
• Participants will be motivated to try at least one new strategy

• Amount of time required: 20 minutes

Teaching Lead:
Jaime Anstee (WMF)

Interested participants:
Add your signature below

• Flixtey (talk) 21:06, 15 April 2016 (UTC)

Data Presentation

Session Abstract:

Specific learning objectives:

• Amount of time required:
• Potential participant collaborators:

Teaching Lead(s):

Maria Cruz

Interested participants:
Add your signature below

• Flixtey (talk) 21:06, 15 April 2016 (UTC)