This page is written as a rubric to be read from the perspective of a student.
You will design a poster that analyzes and proposes an answer to some practical question. The question could be at the level of national decision making, personal choice, or anywhere in between. The poster should show how four major course topics (see below) ought to inform rational decision making on this question. In particular, it should assess the bearing of scientific evidence.
Good topics are not excessively broad or open-ended (e.g. “Should California institute a carbon tax?” is more manageable than “How do we solve climate change?”).
Please choose a question you are passionate about, but be sure to use strategies for avoiding confirmation bias if you already feel strongly one way or the other. Make sure to assess all evidence fairly and present arguments on all sides.
Recommend a course of action while clearly indicating the degree of strength with which you make that recommendation.
Key Dates, Components, and Deadlines
Project Proposal
Due YYYY-MM-DD at 11:59 PM on Gradescope.
Reminder
Make sure to update due date when sharing with students.
The purpose of the proposal is to make sure you have a question that's within scope for this project as well as an idea of how it may be addressed using course concepts. This is not graded harshly. Instead, it's to make sure that you get as much from this project as possible and provide preliminary feedback on how you can best implement your project idea.
The proposal should clearly state your proposed question as well as why you think it might benefit from analysis using the concepts in this course. You should list two course topics you think are relevant. You will be allowed to change your topic or course concepts later (by submitting a new proposal). The proposal only needs to be a couple sentences long, and should be no more than two short paragraphs.
Poster Submission
Due YYYY-MM-DD at 11:59 PM on Gradescope.
Reminder
Make sure to update due date when sharing with students.
Please upload a PDF of your poster that includes your name, SID, and poster title in your submission, as well as whether you'd be comfortable using your poster as an example of an exemplary project in the future.
Poster Session
Date YYYY-MM-DD in person.
Reminder
Make sure to update due date when sharing with students.
You have the option to get extra credit by presenting your poster in person during RRR week. We'll award four extra credit points (equivalent to 17% of the poster grade) to students who present their posters to the professors, teaching staff, and fellow classmates. We highly recommend you present your poster at the poster session. If you cannot make the poster session, you can still get extra credit by presenting to a member of the teaching team if this is the case). The extra credit points apply to your grade for the whole course, not just for this project.
Poster sign-ups will be posted and announced when ready.
Which course concepts to consider?
Below is a list of the major topics we cover in this class. We will have covered all topics by the due date of the project. By "topic" we mean the top-level bullet points. You can include any of the sub-topics as part of your discussion of that topic.
- 1.1 Introduction and When Is Science Relevant
- Democracy vs. epistocracy
- Facts vs. values
- 1.2 Shared Reality and Modeling
- Shared reality
- Raft vs. pyramid
- Evaluation of models
- Science vs. decree
- Scientific realism vs. anti-realism
- Operationalism, conventionalism, and realism
- 2.1 Senses and Instrumentation
- Validation of instruments through interactive exploration, triangulation with other instruments, and comparison with direct senses
- 2.2 Systematic and Statistical Uncertainty
- Measurement proxies as sources of systematic and statistical uncertainty
- 3.1 Probabilistic Reasoning & 3.2 Calibration of Credence Levels
- The value of partial and probabilistic information
- Words of estimative probability (probably, likely, definitely, etc.)
- [math]\displaystyle{ p }[/math]-values and statistical significance
- Error bars and confidence intervals
- Strategies to improve calibration of credence levels (feedback, AOT, growth mindset, etc.)
- 4.1 Signal and Noise & 4.2 Finding Patterns in Random Noise
- Signal-to-noise ratio
- [math]\displaystyle{ p }[/math]-hacking
- Look elsewhere effect
- Gambler's fallacy
- Hot-hand fallacy
- File drawer effect
- HARKing (hypothesizing after results are known)
- Effect size (as distinct from statistical significance)
- 5.1 False Positives and Negatives
- Thresholds between positive and negative detections
- Trade offs between false positives and negatives
- 5.2 Scientific Optimism
- Iterative progress
- 6.1 Correlation and Causation
- Causation as correlation under intervention
- Randomized controlled trials
- Different directions of causation
- Spurious correlations
- 6.2 Hill's Criteria (Causation in the Messy Real World)
- Natural experiments
- Hill's criteria for causation
- Causal networks
- 7.1 Causation, Blame, and Policy
- Singular and general causation
- Acts of omission vs. commission (and the omission bias)
- Status quo bias
- 7.2 Emergent Phenomena
- Global effects that arise through the interaction of small pieces (rather than general causation)
- Explanation at different scales
- Scientific reductionism
- 8.1 Orders of Understanding
- Orders of magnitude
- Multiple causes of comparable importance
- Orders of importance of causes
- Refinement of models using higher order descriptions
- Scale of impact of policies
- 8.2 Fermi Problems
- Fermi problems
- 9.1 Heuristics
- Base rate neglect
- Representativeness heuristic and conjunction fallacy
- Availability heuristic
- Bounded rationality
- 9.2 Biases
- Fundamental attribution error
- Conformity
- Obedience
- Temporal discounting
- 10.1 Confirmation Bias
- Selective exposure
- Biased assimilation
- Strategies for reducing confirmation bias (AOT, etc.)
- 10.2 Blinding
- Techniques for blind analysis
- Preregistration
- Registered replication
- Adversarial collaboration
- Peer review
- 11.1 Pathological Science
- The spectrum of poor research
- Langmuir's pathological science indicators
- 11.2 When Is Science Suspect
- The validity and reliability of social science metrics
- External validity
- The difficulty of creating and applying social science metrics in cross-cultural contexts
- How researchers' biases and backgrounds shape their research agendas
- 12.1 Wisdom of Crowds and Herd Thinking
- The Wisdom of Crowds effect removing independent biases via error cancellation
- How humans can engage in herd thinking to cluster around biased answers to group questions
- 13.1 Denver Bullet Study
- The Denver Bullet Study process of integrating factual analysis from experts with value ratings from different stakeholders
- 13.2 Deliberative Polling
- The deliberative polling process of having the general public make decisions as though they were experts
- 14.1 Scenario Planning
- The Scenario Planning process of predicting and responding to possible futures
- What makes good drivers to use in Scenario Planning
Grading Rubric
| Category | Points |
|---|---|
Introduction
|
2 |
Course Topics (four total)
|
16 (4 each) |
Conclusion
|
4 |
Visuals
|
2 |
| Total | 24 |