Learning and Evaluation/Archive/Learning modules/3Construct Validity

  Wikimedia Training Designing Effective Questions Menu

Construct Validity

Does a question measure what it is really supposed to measure?

For example, does an intelligence test really measure intelligence?

Objective: A program leader wants to know what the quality level is of their images from Wiki Loves Monuments that just ended last month.

---

Very poor contruct validity: Number of images uploaded to Commons as a measure of quality for WLM implementation.

The number of images is not an accurate measure of photo quality; many of the photos might be duplicates or have poor quality. Thus this is not an accurate measure of quality.

Poor construct validity: Measuring the quality rating one week after the event to determine if the photos quality.

The process of reviewing all photos and nominating those of the highest quality for the quality photo rating takes much longer than one week after the event. Thus, this measure is not accurately measuring the number of photos uploaded.

Better construct validity: Measuring the quality rating three months after the event to determine the proportion of photos with a Quality rating.

This third option has better construct validity because obtaining the Quality rating involves not only the label but the nomination process, which takes a lot of time. Three months after the event is a more reasonable time frame for counting the number of Quality photos.