Learning and Evaluation/Archive/Learning modules/3Questions order

Jump to page

Part 1: Introduction

Why Survey?
Why Surveys Are Useful
Survey instruments
Types of information
Attributes - a special case
Survey Objective and Planning

Part 2: Reliability & Validity

Reliability & Validity
Face Validity
Content Validity
Criterion Validity
Construct Validity

Part 3: Question Construction

Writing Good Questions
Questions from Existing Surveys
Constructing your own Questions
Be Specific
Be Concise
Avoid Double Negatives
Minimize Social Desirability Bias
Avoid Double-barreled questions
Avoid abbreviations, jargon, technical terms, or slang
Avoid leading questions
Avoid loaded questions
Use appropriate wording
Ask useful questions
Rely on second-hand data sparsely
Use caution when asking personal questions

Part 4: Response Options

Question types
Dichotomous pairs
Multiple choice
Check all that apply
Choosing response options

Part 5: Questionnaire structure

Important considerations
Questions order
Additional Resources

  Wikimedia Training Designing Effective Questions Menu

The order of questions matters

After deciding your questions, you must combine them into a questionnaire. A questionnaire is not a hazard collection of questions but is a carefully planned sequence of questions. The questionnaire is structured to obtain information that meets the requirements for your program goals.
  • Natural Time Sequence
  • Most familiar to least familiar
  • Sensitive questions later
  • Easy questions last
  • Open-ended questions nearer to end

Typically, surveys follow the order below:
1. Introduction
Always explain the purpose and process of the survey before participants are asked their first question.
2. Participation measures
Questions about what type of program participant is upfront is often important for easing survey burden and appropriately guiding survey participants to the survey questions that apply to them, while skipping them passed those which do not apply to their particular participation category.
3. Knowledge
Knowledge of rules, regulations, guidelines, operational standards, information sources, etc. For Wikimedia, this would be a good place to understand how much they know about Wikimedia or their experience with editing Wikipedia.
4. Opinions/attitudes
Next, a respondent might answer questions about their opinions or attitudes about the program. For example, what types of opinions did they have about an Edit-a-thon or a workshop?
5. Recommendations
Recommendations are usually open-ended essay questions. These provide opportunity for respondents to offer qualitative information that might provide insight into a program beyond the scope of the survey questions.
6. Demographics
Easiest questions come first so respondents can ease into the survey. Questions include basic demographics start first, such as one’s job or job description, length of time on the job, educational background, general location, etc. Sensitive questions typically occur in the middle of the suvey, where appropriate, while essay questions happen at the very end because they might take some time.
7. Thank you
People are bombarded by survey requests more and more. It is important to appreciate participants who take the time to share their perceptions through surveys as it allows for broad data collection that would otherwise require much more time of an evaluator.