10.2 Blinding

From Sense & Sensibility & Science
Topic Icon - 10.2 Blinding.png

Blind analysis has emerged as the newest addition to the scientific process to guard against scientists' own confirmation bias, especially in high-precision experiments involving complex analyses, by preventing the scientists from seeing the result of their analysis as they refine and debug the analysis procedure. We illustrate blinding techniques and their dramatic effect through a simple measurement experiment.

The Lesson in Context

This lesson offers solutions to potential pitfalls in scientific studies raised in 10.1 Confirmation Bias and 4.2 Finding Patterns in Random Noise. We use a stick measurement activity to illustrate the effects of confirmation bias and to motivate techniques to reduce its effect, especially blind analysis. These techniques are not universally employed in all fields of science today, and students pursuing a scientific career are encouraged to introduce these techniques in their own work.

Earlier Lessons

4.2 Finding Patterns in Random NoiseTopic Icon - 4.2 Finding Patterns in Random Noise.png
  • Blind analysis is one way to prevent some forms of [math]\displaystyle{ p }[/math]-hacking. For example, choices to be made about a study, such as the exact statement of the hypothesis, definitions of terms, and statistical techniques, can be preregistered or made "blinded" to the data. The effects on the final result due to these choices may be hidden from the researcher during analysis. These techniques prevent motivated reasoning during analysis decisions.
10.1 Confirmation BiasTopic Icon - 10.1 Confirmation Bias.png
  • Blind analysis helps prevent scientists from the temptation to make choices that make the result more likely to confirm the researcher's own prediction or to match currently accepted knowledge. Otherwise, non-confirming, surprising results may be incorrectly missed.

Later Lessons

13.1 Denver Bullet StudyTopic Icon - 13.1 Denver Bullet Study.png
  • In group decision making, when factual evaluation and values evaluation are made by two different groups of people without knowledge of the other, it prevents evaluation motivated by the need to confirm a personal belief.

Takeaways

After this lesson, students should

  1. Recognize what types of blinding are useful for solving what types of errors.
  2. Be able to explain why blind analysis might be needed, by explaining the errors that can arise in its absence.
  3. Recognize when blind analysis is being used and explain what function it serves. Identify situations and decisions in which blind analysis would be useful.
  4. Be able to evaluate techniques (e.g., registered replication, adversarial collaboration, peer review)
    1. for ability to address confirmation bias, and
    2. in comparison to blind analysis.
  5. Propose how to use blind analysis for simple studies.

Additional Content

You must be logged in to see this content.