11.1 Pathological Science

Not all published scientific results are automatically trustworthy. How can we identify pathological science, pseudoscience, fraudulent science, poorly done science, or good science that just happens to get the wrong answer? We present a set of indicators of pathological science—when well-intentioned scientists have "fallen in love" with their ideas and start making excuses when experiments turn out otherwise.

The Lesson in Context

This lesson teaches students to be cautious of bad science of all kinds and aware of the signs thereof. We ask students to study a few established examples of pathological science in history dressed as science merely in form. The emphasis is on well-intentioned researchers "falling in love" with their own ideas, finding excuses to justify them even when reality has turned out to be contrary to them.

Earlier Lessons

1.2 Shared Reality and Modeling

Everyone in principle has access to the same shared reality. If a certain (amazing) result by one research group cannot seem to be replicated by many other groups, it is a good sign that it does not accurately describe the shared reality.

3.2 Calibration of Credence Levels

Overstating the confidence level of a scientific result is a sign of bad science.

4.2 Finding Patterns in Random Noise

It is expected that patterns arise from random noise. Good science can also turn out to be wrong just by chance, but stubbornly sticking with the original result would turn it into bad science.

5.2 Scientific Optimism

In the spirit of personal persistence and iterative progress, setbacks such as a wrong or disappointing result should be accepted as an inevitable part of this progress.

10.2 Blinding

Though not yet widely employed in all fields of science, various blind analysis techniques can help reduce the possibility that a scientific result may be contaminated by subtle analysis choices made by the researchers that are (often subconsciously) motivated by the desire for a certain anticipated result.

Later Lessons

11.2 When Is Science Suspect

Bad science is particularly problematic when it concerns the study of human subgroups, as it may be motivated by or may perpetuate preexisting inequitable power structures in society.

Takeaways

After this lesson, students should

Be able to distinguish roughly among the following:
1. Good science (that gets the wrong or right answer).
2. Fraudulent science.
3. Pathological science.
4. Poorly-done science.
5. Pseudo-science.
Feel comfortable using Langmuir's Pathological Science Indicators to assess scientific studies.
Be able to identify what is wrong in cases of fraudulent, pathological, poorly-done, and pseudo-science.

Spectrum of Poor Research

There are many gradations to poor research and poor results. Several approximate categories are listed below.

Good Science

This includes good well-done science that gets the wrong result. Even with a confidence interval of 95%, 5% of the time the results will be wrong. This is normal.

Poorly-done Science

Science done honestly, but not done well.

Pathological Science

This is what happens when a researcher falls a little too "in love" with their ideas. It is indicated by Langmuir's criteria.

Pseudo-science

Pseudo-science is characterized by using scientific vocabulary without aligning with the corresponding concepts or engaging in real scientific practices (i.e., science being "skin deep," not scientific below the surface).

Fraudulent Science

Research that involves intentional deception, such as deliberately fabricating data or deliberately deceiving the reader about the strength of evidence.

Langmuir's Pathological Science Indicators

A set of six indicators that can be used to flag when something might be pathological science.

The effect is produced by a barely detectable cause, and the magnitude of the effect is substantially independent of the intensity of the cause.
The effect is barely detectable, or has very low statistical significance.
Claims of great accuracy.
Involving fantastic theories contrary to experience.
Criticisms are met with ad hoc excuses.
Ratio of supporters to critics rises to near 1:1, then drops back to near zero.

There is no set number of Langmuir's criteria that determines whether something is pathological science or not. The criteria serve as a useful guide, but detecting pathological science is ultimately something of an art. That being said, whether or not a given study is a case of pathological science depends on the attitude of the researchers. Indicators 3 and 5 speak to this most directly.

Exemplary Quotes

“Finding a tarantula at that elevation, more than 14,700 feet up, was a revelation. Normally, these hairy spiders aren't too fond of arid, oxygen-deprived mountain air or subglacial terrain. But little did Seimon know, the South American hills were literally crawling with previously undescribed tiny tarantulas—including the one she'd just plucked from its burrow. This spider not only turned out to be a new species—it lives at the highest elevation at which a tarantula has ever been found. And this discovery, along with several concurrent investigations, have turned up a total of seven new tarantula species in the genus Hapalotremus, as described in a recent study published in the Journal of Natural History. After Seimon's first find, she returned to that same spot in the Andes to look for more of the spiders.”

National Geographic

This paper's result could not be replicated by others in the field, therefore the authors have committed pathological science.

There may be a genuine difference in the way the study is conducted between these papers, contributing to a difference in result. The original study may have reached the wrong conclusion from correct analysis simply due to the random noise in the data. The original study may have made an error in their analysis that has not been discovered (for example, superluminal neutrinos).

After this lesson, students should

Concept Acquisition
1. The boundaries demarcating science from non-science and distinguishing among the categories of pathological science, pseudo-science, fraudulent science, poorly-done science, and good science can often be difficult, with overlapping and fuzzy boundaries between categories.
2. Pathological Science Indicators:
  1. The effect is produced by a barely detectable cause, and the magnitude of the effect is substantially independent of the intensity of the cause.
  2. The effect is barely detectable, or has very low statistical significance. Claims of great accuracy.
  3. Involving fantastic theories contrary to experience.
  4. Criticisms are met with ad hoc excuses.
  5. Ratio of supporters to critics rises to near 50%, then drops back to near zero.
  6. Conclusion-motivated design & analysis.
  7. Pseudo-science is characterized by using scientific vocabulary without aligning with the corresponding concepts or engaging in real scientific practices (i.e., science being "skin deep," not scientific below the surface).
3. Fraudulent science involves intentional deception, such as deliberately fabricating data or deliberately deceiving the reader about the strength of evidence.
4. Poorly-done science, e.g. failure to consider confounds, failure to use best practices in terms of data collection and analysis (e.g., small sample size, look elsewhere effect).
5. Unintentional self-deception can be involved in justifying poor practices and/or interpretations in pathological, pseudo-, & poorly-done science.
6. Motivation to support a particular conclusion (i.e., science undertaken to support a given conclusion, rather than to discover the truth) can be a feature of poorly done or pathological science.
7. Good science:
  1. Will get the wrong answer some of the time, e.g., via statistical flukes.
  2. Entails good faith engagement with the alternative hypotheses through a search for evidence that you are wrong.
Concept Application
1. Distinguish science from enterprises such as religion or (perhaps) astrology where the attempt is not to find descriptive adequacy but meaning in ordinary life.
2. Identify cases of good science that gets the wrong answer, fraudulent science, pathological science, poorly-done science, and pseudo-science based on the above characteristics.
3. Identify what is wrong in cases of fraudulent, pathological science, poorly-done, and pseudo-science.

Additional Content

You must be logged in to see this content.