|
|
| Line 71: |
Line 71: |
|
| |
|
| </tabber> | | </tabber> |
| <restricted>
| |
|
| |
|
| == Useful Resources ==
| | {{#restricted:{{11.1 Pathological Science}}}}{{NavCard|prev=10.2 Blinding|next=11.2 When Is Science Suspect}} |
| | |
| <tabber>
| |
| | |
| |-|Lecture Video=
| |
| | |
| <br /><center><youtube>oaxHKHpwlxw</youtube></center><br />
| |
| | |
| |-|Discussion Slides=
| |
| | |
| {{LinkCard | |
| |url=https://docs.google.com/presentation/d/1CgRwwPNdaDf7wYtcMSQ6OhD9O56eHrE5Q0buh76DSGw/
| |
| |title=Discussion Slides Template
| |
| |description=The discussion slides for this lesson.
| |
| }}
| |
| <br />
| |
| | |
| |-|Handouts and Activities=
| |
| | |
| {{LinkCardInternal
| |
| |url=:File:Pathological Science Case Studies.pdf
| |
| |title=Pathological Science Case Studies Handout
| |
| |description=Handout for the poor science case studies activity.}}
| |
| {{LinkCardInternal
| |
| |url=:File:Arsenic Bacterium.pdf
| |
| |title=Study A: Arsenic Bacterium
| |
| |description=The first example for the poor science case studies activity.}}
| |
| {{LinkCardInternal
| |
| |url=:File:Water Memory.pdf
| |
| |title=Study B: Water Memory
| |
| |description=The second example for the poor science case studies activity.}}
| |
| {{LinkCardInternal
| |
| |url=:File:Superluminal Neutrinos.pdf
| |
| |title=Study C: Superluminal Neutrinos
| |
| |description=The third example for the poor science case studies activity.}}
| |
| {{Line}}
| |
| The following are the original papers the summaries in the [[#Poor Science Case Studies|poor science case studies activity]] were based on.
| |
| {{LinkCardInternal
| |
| |url=:File:Wolfe-Simon et al. - 2011 - A Bacterium That Can Grow by Using Arsenic Instead.pdf
| |
| |title=A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus
| |
| |description=The original paper for study A.}}
| |
| {{LinkCardInternal | |
| |url=:File:Davenas et al. - 1988 - Human basophil degranulation triggered py very dil.pdf
| |
| |title=Human Basophil Degranulation Triggered by Very Dilute Antiserum Against IgE
| |
| |description=The original paper for study B.}}
| |
| {{LinkCardInternal
| |
| |url=:File:The OPERA collaboration et al. - 2012 - Measurement of the neutrino velocity with the OPER.pdf
| |
| |title=Measurement of the Neutrino Velocity With the OPERA Detector in the CNGS Beam
| |
| |description=The original paper for study C.}}
| |
| <br />
| |
| | |
| |-|Readings and Assignments=
| |
| | |
| {{LinkCardInternal
| |
| |url=:File:Pathological Science - Langmuir.pdf
| |
| |title=Pathological Science
| |
| |description=Essay by Irving Langmuir presenting his pathological science indicators.}}
| |
| {{LinkCardInternal
| |
| |url=:File:Cargo Cult Science - Feynman.pdf
| |
| |title=Cargo Cult Science
| |
| |description=Essay by Richard Feynman about cargo cult science.}}
| |
| <br />
| |
| | |
| </tabber>
| |
| | |
| == Recommended Outline ==
| |
| | |
| === Before Class ===
| |
| | |
| Each student will have read one of the three studies that we discuss in this section, as well as an study by Langmuir. Assign each group to one of the three studies one week ahead of time, and then send another reminder to read them one or two days before the lesson. Also make sure to print the handout.
| |
| | |
| === During Class ===
| |
| | |
| {| class="wikitable" style="margin-left: 0px; margin-right: auto;"
| |
| |5 Minutes
| |
| |Introduce the lesson and go over the plan for the day. Make sure people have groups, spokespeople, etc.
| |
| |-
| |
| |15 Minutes
| |
| |Review the [[#Takeaways|definitions]] for this lesson.
| |
| |-
| |
| |60 Minutes
| |
| |Run the [[#Poor Science Case Studies|poor science case studies activity]].
| |
| |}
| |
| | |
| == Lesson Content ==
| |
| | |
| === Poor Science Case Studies ===
| |
| | |
| Students are assigned one of three scientific studies to skim through before class and asked to assess the validity of its methodology, with reference to Langmuir's indicators for pathological science. They may follow the handout.
| |
| {{LinkCardInternal
| |
| |url=:File:Pathological Science Case Studies.pdf
| |
| |title=Pathological Science Case Studies Handout
| |
| |description=Handout for the poor science case studies activity.}}
| |
| ==== Instructions ====
| |
| | |
| {| class="wikitable" style="margin-left: 0px; margin-right: auto;"
| |
| |15 Minutes
| |
| |Students review their studies in small groups and answer the questions in the handout. A note taker should be responsible for writing down ideas from the group.
| |
| |-
| |
| |20 Minutes
| |
| |Since each group has been assigned one of three studies, we now bring three groups with different studies together into larger pods of groups. Each group's spokesperson will spend 3-5 min presenting their group's study to the other groups using their answers to [[#Small-group Questions|the small-group questions on the handout]] as a guide.
| |
| |-
| |
| |25 Minutes
| |
| |Call the whole class together and go over the [[#Final Discussion Questions|final discussion questions]]. Try to come up with a class consensus for each study.
| |
| |}
| |
| | |
| === Small-group Questions ===
| |
| | |
| These are the questions on the handout that small groups will use to guide their discussion. Only after taking notes that answer these will they go over the [[#Final Discussion Questions|final discussion questions]]. These questions are the same for each group no matter what study they were assigned.
| |
| | |
| # What was the research question and hypothesis?
| |
| # What variables were tested?
| |
| # Do you have a sense that there were other confounding factors that were controlled for?
| |
| # How did the scientific community respond? How did the scientists who conducted the research respond to the scientific community?
| |
| # What did the scientists conclude? What did the scientific community conclude?
| |
| | |
| === Final Discussion Questions ===
| |
| | |
| [[File:Poorly-done Science Scale - Light.png|thumb|The poor science scale (or spectrum).]]
| |
| These questions are used for each study. They are initially discussed in small groups and then answered collectively by the entire class. The third question is intended mainly for small groups so as to serve as fodder for the larger discussion.
| |
| # Where does this study fall on the poor science scale?
| |
| # Which of Langmuir's criteria (if any) apply to this study?
| |
| # Where, if anywhere, did this science go wrong?
| |
| | |
| ==== Study A: Arsenic Bacterium ====
| |
| | |
| This paper discussed a novel substitution of arsenic for phosphorus in bacterial DNA, which would have profound implications on requirements for life in any form. They reported the discovery of an unusual microbe that could vary elemental composition of its basic biomolecules by substituting arsenic for phosphorus. Mechanisms for such substitution were unknown. Many criticisms were published as technical comments in ''Science'' and online, such as accusing the original team of failing to "meticulously clean up the DNA first". An Israeli team worked out the mechanism by which the bacterium in question discriminated between nearly identical molecules of phosphate and arsenate. ''Science'' also published an official statement saying some of the findings in the original paper were incorrect in 2012.
| |
| {{LinkCardInternal
| |
| |url=:File:Arsenic Bacterium.pdf
| |
| |title=Study A: Arsenic Bacterium
| |
| |description=The first example for the poor science case studies activity.}}
| |
| {{LinkCardInternal
| |
| |url=:File:Wolfe-Simon et al. - 2011 - A Bacterium That Can Grow by Using Arsenic Instead.pdf
| |
| |title=A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus
| |
| |description=The original paper for study A.}}
| |
| ===== Answers =====
| |
| | |
| <ol start=1><li>Where does this study fall on the poor science scale?</li></ol>
| |
| {{BoxAnswer|Poorly-done science. Wrong results.}}
| |
| <ol start=2><li>Which of Langmuir's criteria (if any) apply to this study?
| |
| <ol style="list-style-type:decimal">
| |
| <li>{{Correct|The effect is produced by a barely detectable cause, and the magnitude of the effect is substantially independent of the intensity of the cause.}}</li>
| |
| <li>The effect is barely detectable, or has very low statistical significance.</li>
| |
| <li>Claims of great accuracy.</li>
| |
| <li>{{Correct|Involving fantastic theories contrary to experience.}}</li>
| |
| <li>{{Correct|Criticisms are met with ad hoc excuses.}}</li>
| |
| <li>Ratio of supporters to critics rises to near 1:1, then drops back to near zero.</li>
| |
| </ol>
| |
| </li></ol>
| |
| <ol start=3><li>Where, if anywhere, did this science go wrong?</li></ol>
| |
| {{BoxAnswer|The experiment was not done carefully enough.}}
| |
| ==== Study B: Water Memory ====
| |
| | |
| This paper provided evidence for the theory that water retains memory of previously dissolved substances, which has been claimed to be a mechanism by which homeopathic remedies work, but runs contradictory to our current understanding of chemistry and physics. However, since the paper had no apparent methodological problems, it was published in ''Nature'' accompanied with an editorial that noted "There are good and particular reasons why prudent people should, for the time being, suspend judgment" and described some of the fundamental laws of chemistry and physics which it would violate, if shown to be true. Additionally, the editor of ''Nature'' demanded that the experiments be re-run under supervision. In the supervised double-blind experiments, no memory effect was observed. ''Nature'' published a follow-up report of these findings, and their editor said: "We believe the laboratory has fostered and then cherished a delusion about the interpretation of its data." and pointed out that two of the researchers were being paid for by a homeopathic company. The PI also published a follow-up letter where he compared his treatment by the ''Nature'' team to "Salem witchhunts or McCarthy-like prosecutions". More drama ensued, and the results of the paper are still used to claim that the experiments "prove" that homeopathy works.
| |
| {{LinkCardInternal
| |
| |url=:File:Water Memory.pdf
| |
| |title=Study B: Water Memory
| |
| |description=The second example for the poor science case studies activity.}}
| |
| {{LinkCardInternal
| |
| |url=:File:Davenas et al. - 1988 - Human basophil degranulation triggered py very dil.pdf
| |
| |title=Human Basophil Degranulation Triggered by Very Dilute Antiserum Against IgE
| |
| |description=The original paper for study B.}}
| |
| ===== Answers =====
| |
| | |
| <ol start=1><li>Where does this study fall on the poor science scale?</li></ol>
| |
| {{BoxAnswer|Pathological science.}}
| |
| <ol start=2><li>Which of Langmuir's criteria (if any) apply to this study?
| |
| <ol style="list-style-type:decimal">
| |
| <li>The effect is produced by a barely detectable cause, and the magnitude of the effect is substantially independent of the intensity of the cause.</li>
| |
| <li>{{Correct|The effect is barely detectable, or has very low statistical significance.}}</li>
| |
| <li>Claims of great accuracy.</li>
| |
| <li>{{Correct|Involving fantastic theories contrary to experience.}}</li>
| |
| <li>{{Correct|Criticisms are met with ad hoc excuses.}}</li>
| |
| <li>{{Correct|Ratio of supporters to critics rises to near 1:1, then drops back to near zero.}}</li>
| |
| </ol>
| |
| </li></ol>
| |
| <ol start=3><li>Where, if anywhere, did this science go wrong?</li></ol>
| |
| {{BoxAnswer|The researchers were very attached to their scientific findings and criticized the supervised experiments.}}
| |
| ==== Study C: Superluminal Neutrinos ====
| |
| | |
| Neutrinos are subatomic pstudies with extremely small masses, which means that they should move at very nearly (but slightly slower than) the speed of light. In this paper, neutrinos created at CERN were detected at another site in Italy to measure the precise arrival time and distance, and therefore the speed at which neutrinos travel. They found that neutrinos traveled slightly faster than the speed of light with high significance (over 15,000 separate neutrino events). If this is correct, it would overturn a cornerstone of modern physics. The team published their preliminary data to the public to enlist the help of other scientists for searching for an explanation, and doubled down on their efforts to find an explanation for this result. The team's in-depth investigation of their experiment turned up a few errors, which when corrected found that the speed was consistent with the speed of light. Other independent experiments also found that neutrinos traveled at the speed of light, but showed no evidence for faster than the speed of light.
| |
| {{LinkCardInternal
| |
| |url=:File:Superluminal Neutrinos.pdf
| |
| |title=Study C: Superluminal Neutrinos
| |
| |description=The third example for the poor science case studies activity.}}
| |
| {{LinkCardInternal
| |
| |url=:File:The OPERA collaboration et al. - 2012 - Measurement of the neutrino velocity with the OPER.pdf
| |
| |title=Measurement of the Neutrino Velocity With the OPERA Detector in the CNGS Beam
| |
| |description=The original paper for study C.}}
| |
| ===== Answers =====
| |
| | |
| <ol start=1><li>Where does this study fall on the poor science scale?</li></ol>
| |
| {{BoxAnswer|Good scientific practice. Wrong results.}}
| |
| <ol start=2><li>Which of Langmuir's criteria (if any) apply to this study?
| |
| <ol style="list-style-type:decimal">
| |
| <li>The effect is produced by a barely detectable cause, and the magnitude of the effect is substantially independent of the intensity of the cause.</li>
| |
| <li>{{Correct|The effect is barely detectable, or has very low statistical significance.}}</li>
| |
| <li>Claims of great accuracy.</li>
| |
| <li>{{Correct|Involving fantastic theories contrary to experience.}}</li>
| |
| <li>Criticisms are met with ad hoc excuses.</li>
| |
| <li>Ratio of supporters to critics rises to near 1:1, then drops back to near zero.</li>
| |
| </ol>
| |
| </li></ol>
| |
| <ol start=3><li>Where, if anywhere, did this science go wrong?</li></ol>
| |
| {{BoxAnswer|Even very carefully done science may inevitably contain errors. The team recognized the implications of the results and openly invited other scientists to critique their procedure, leading to the discovery and correction of the relevant errors.}}
| |
| {{BoxTip|This example illustrates that a couple of Langmuir's criteria in-and-of themselves do not mean that a certain study is a case of pathological science. If the researchers fully acknowledge the implications and limitations of their results and do everything in their power to prove themselves wrong then it is likely still an example of good scientific practice.}}</restricted>{{NavCard|prev=10.2 Blinding|next=11.2 When Is Science Suspect}}
| |
| [[Category:Lesson plans]] | | [[Category:Lesson plans]] |
