In the English language, are there more words that have 'K' as the first letter OR as the third letter?
In the English language, are there more words that have "K" as the first letter OR as the third letter?
<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
<li>More words with 'K' as the first letter.</li>
<li>More words with "K" as the first letter.</li>
<li>{{Correct|More words with 'K' as the third letter.}}</li>
<li>{{Correct|More words with "K" as the third letter.}}</li>
</ol>
</ol>
{{BoxAnswer|title=Explanation|It's difficult to answer this accurately because it's much easier to think of words with the first letter K than the third letter, just because of how we retrieve words from memory.}}
{{BoxAnswer|title=Explanation|It's difficult to answer this accurately because it's much easier to think of words with the first letter K than the third letter, just because of how we retrieve words from memory.}}
Revision as of 19:21, 15 August 2023
Some of the heuristics biases that make our probability judgments go awry.
The Lesson in Context
Humans make many decisions on a daily basis, often in the absence of complete information or under constraints of limited time and cognitive capacity. We use heuristics as useful shortcuts for quick decision making, which may introduce bias into our conclusions. The purpose of the lesson is not to cast doubt on our use of heuristics, but to recognize the limitations of quick human judgments and their consequences. This parallels 2.1 Senses and Instrumentation and 2.2 Systematic and Statistical Uncertainty, where the limitations of instruments are discussed and quantified, without rejecting the validity and usefulness of instruments altogether.
Senses and instrumentation are inherently imperfect, but imperfect tools can still be useful in obtaining partial knowledge. Similarly, heuristics are flawed, but they can make useful tools when time, knowledge, and mental resources are limited.
The use of heuristics can often introduce bias into our judgments — tendencies to make one decision more often than another, paralleling the idea of systematic uncertainty in instrumental measurements.
This lesson focuses on heuristics that affect our judgments of frequencies — how often things occur or likelihoods of events. The next lesson discusses biases in decision making that stem from a self-centered view of the world—an overemphasis on "me" and "now" and overusing defaults and assumptions.
We single out confirmation bias into its own topic, as it permeates scientific and group decision making, affecting both our sense of the prevalence of events around us as well as the importance of "me" and "now".
Takeaways
After this lesson, students should
Know that we use heuristics as a shortcut in everyday decision making.
Recognize that while heuristics can be useful and necessary, they can also lead us astray by introducing biases into our decision making.
Be able to identify cases of Base Rate Neglect.
Be able to avoid the temptation to engage in Base Rate Neglect.
Be able to identify cases of the Representativeness Heuristic, and not fall for it.
Be able to identify cases of the Conjunction Fallacy, and not fall for it.
Be able to identify cases of the Availability Heuristic, and not fall for it.
Learn the basics of Bayesian reasoning.
Heuristics
Mental shortcuts people make instead of analyzing all relevant information while making decisions.
Base Rates
The base frequency of a given attribute in a whole population.
Base Rate Neglect
The act of overlooking the importance of base rates when calculating the probability of an event based on probabilities that seem more relevant to the specific case.
Cases in which how representative something is of a category or outcome is used as a proxy/heuristic for evaluating how likely the category membership or outcome is (not taking base rates into account).
Conjunction Fallacy
The tendency to neglect that something is less likely to be part of a subset of a set than a set itself. In reality, [math]\displaystyle{ A }[/math] is always more likely to be true than [math]\displaystyle{ A }[/math]and[math]\displaystyle{ B }[/math] because if [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math] is true then [math]\displaystyle{ A }[/math]must be true. This usually happens as a consequence of the representativeness heuristic, when [math]\displaystyle{ B }[/math] appears more representative of the set in question.
Availability Heuristic
Cases in which people use how readily something comes to mind as a proxy for an estimate of its probability. This can be influenced by factors unrelated to probability, such as vividness or frequency of appearance in the media.
Bounded Rationality
The idea that people have limited information and capacity to analyze all factors when making decisions (i.e. "rationality" is "bounded"). Given bounded rationality, it's often reasonable (that is, better than if we decided randomly) to use heuristics to guide our decisions.
Availability Heuristic: Plane Crashes
Ever since reading about plane crashes in the news, Harriet insists on driving even long distances. She is falling into the availability heuristic. She is actually much more likely to die in a traffic accident than a plane accident. Traffic accidents are extremely common but rarely make the news, whereas plane accidents are rare but always make the news, making them more "available" to her memory and thus seem more common than they are.
Representativeness Heuristic: Pit Bulls
Having been assured that the rescue dog was extremely friendly, Milo was shocked to discover it was a pit bull. He knew that pit bulls are often less friendly than other dogs, and did not know that pit bulls are over-represented among rescue dogs (in part because of this reputation).
Conjunction Fallacy: Tabbies
Even though tabbies are the commonest brown cat, the fact that all brown tabbies are brown cats but not all brown cats are tabbies means that it must be more likely that the cat is brown than that it's a brown tabby.
If you look at the numbers, you can see that there are actually more vaccinated people vaccinated getting sick than unvaccinated people. So getting vaxxed actually makes you more likely to get sick!
This is a case of base rate neglect. Because more people are vaccinated than unvaccinated, you can get this pattern even when vaccination is strongly protective. For example, if 85% of people are vaccinated, and being vaccinated makes you 30% as likely to catch the disease, then out of every hundred people, .85 * .3 * 100 ＝ 25.5 vaccinated people on average will get sick, while .15 * 1 * 100 ＝ 15 unvaccinated people will get sick. So even though more vaccinated people are getting sick in terms of absolute quantity, getting the vaccine still makes any given person 1/3 as likely to get sick.
Heuristics cause us to make wrong judgements so they're bad and we should stop using them.
Heuristics can sometimes lead us towards fallacies. But, that does not mean that they are useless! Heuristics still tend to be better than making decisions arbitrarily. And we don't always have the time or means to fully analyze every decision.
This class in particular has a lot of content. It's also very easy to confuse the students if not explained very carefully. Make sure to take extra time reviewing this lesson.
You're sitting in your room cramming for an exam when your roommate decides to throw an impromptu party and invites people from all over the university. Realizing that you're not going to get any work done, you decide to make the most of it and start mingling with the crowd. In the process, you strike up a conversation with someone named Jessie who starts droning on and on about rockets. They go on for so long that you start to lose interest and begin thinking about what sort of person Jessie really is.
Here's some helpful numbers:
One in four students study engineering at your college.
Half of all engineering majors like rockets.
A sixth of all non-engineering majors like rockets.
Which is more likely?
Jessie is an engineering major.
Jessie is a non-engineering major.
It's equally likely that Jessie's an engineering or non-engineering major.
Don't Reveal The Answer Yet!
It's actually 50/50 odds that Jessie is an engineering major. This is explained later Jessie Again.
KOALA-25
We introduce and practice Bayes' rule in this activity. It makes substantial use of the Bayes' rule visualizer.
Pull up the notebook and project it in front of the class.
1 Minute
Explain what the notebook's colors mean using the legend in the demo.
2 Minutes
Using the notebook as a visual aid, remind the students what prior probability is and that it's the same thing as "base rate," "incidence rate," and "positivity rate."
2 Minutes
Remind the students of what true/false positive/negative rates mean. Drag the rates sliders around to demonstrate the following.
Click on the "Positive" button to highlight the true and false positives. Use this to explain the middle line of the Bayes rule formula below. Then explain the last line step by step as well.
[math]\displaystyle{ \begin{align}
&\ \ \quad\text{(probability that a positive is a true positive)} \\
&= \frac{(\#\text{ true positives})}{(\#\text{ true positives})+(\#\text{ false positives})} \\
&= \frac{\text{(base rate)}\times\text{(true positive rate)}}{\text{(base rate)}\times\text{(true positive rate)}+(1-\text{base rate})\times\text{(false positive rate)}}
\end{align} }[/math]
Have the students do the discussion in small groups.
KOALA-25 Warm-up Question
Suppose there is an epidemic of KOALA-25 breaking out among koalas in New South Wales, which about 1% of the koalas have contracted. A test for KOALA-25 was developed, whose false positive rate is 9% and false negative rate is 10%. If a particular koala has tested positive of KOALA-25, what is the actual probability that it really has KOALA-25?
Between 0-10%
Between 10-25%
Between 25-50%
Between 50-75%
Between 75-90%
Between 90-100%
Don't Reveal The Answer Yet!
This will be worked out in detail below. The correct answer is "a" (9.2%).
KOALA-25 Example
Visualization of the calculations for KOALA-25.
It's now time to reveal the answer to the warm-up question. As shown in the figure, there is a large population with a relatively small base rate of infection. So if you were doing a mass surveillance testing of koalas then most of the ones that test as positive actually don't have the disease at all.
KOALA-25 Discussion Question
Have students work in small groups to work out this problem, following the KOALA-25 example above. Give assistance where needed. They may now use the notebook as well.
Based on a daily case count of 73,000 (November 2021) and a 14-day recovery period, it can be Fermi estimated that the prevalence of Covid-19 is 0.3%. For the commonly used PCR test, the true positive rate (sensitivity) is 98%, and the true negative rate (specificity) is 80%. If you do one PCR test and get a positive result, what are the actual odds that you have Covid?
The students are free to work this out using the notebook. But, they can also do it explicitly.
Ask the students if this sounds right to them. The reason it shouldn't is that we don't actually just give COVID tests out to the general public. People normally take PCR tests once they have some symptoms. Therefore we shouldn't use the COVID rates amongst the general population as our base rate. Instead, we should use the fraction of those who suspect they have COVID (due to symptoms or close contact) that actually do have it.
Suppose one third of those suspecting they have COVID do really have it. Using this as our base rate, what is the probability that you have COVID if your PCR test comes back positive?
Now that we have some practice working out Bayesian probabilities with diseases, it's time to try and figure out who Jessie really is. Recall our "disease" formula.
[math]\displaystyle{ \begin{align}
&\ \ \quad\text{(probability that a positive is a true positive)} \\
&= \frac{(\#\text{ true positives})}{(\#\text{ true positives})+(\#\text{ false positives})} \\
&= \frac{\text{(base rate)}\times\text{(true positive rate)}}{\text{(base rate)}\times\text{(true positive rate)}+(1-\text{base rate})\times\text{(false positive rate)}}
\end{align} }[/math]
We're using the term "test" here very broadly. For example, our original conversation with Jessie counts! In that case, we were using Jessie's interest in rocketry as a "test" for whether or not they're an engineering major. If Jessie is interested in rockets and is also an engineering major then we have a true positive. But if Jessie isn't an engineering major then it's a false positive.
We have a "positive" result because we know Jessie is interested in rocketry.
Solution
The first item gives the "Base rate of positivity." The second item is the "True positive rate of the test" and the third item is the "False positive rate of the test." Plugging all these numbers in, we get a 50% chance that Jessie is an engineering major.
Here are the values from the original problem. Given this information, what is the actual probability that Jessie is an engineering major? The students are free to work this out either directly or with the notebook.
One in four students study engineering at your college.
Half of all engineering majors like rockets.
A sixth of all non-engineering majors like rockets.
Which is more likely?
Jessie is an engineering major.
Jessie is a non-engineering major.
It's equally likely that Jessie's an engineering or non-engineering major.
Conjunction Fallacy
Very quickly poll your students with the following question.
Andy is a junior at Berkeley. His favorite book is Howard Zinn's "People's History of the United States." He's passionate about politics, and he regularly attends local protests. He is most likely to be:
A computer science major.
A computer science and also a political science major (double major).
A computer science major who is a member of the Berkeley College Democrats.
A computer science and also a political science major (double major) who is a member of the Berkeley College Democrats.
Andy is most likely a computer science major. This is because all the other categories are also built on the assumption that he's a computer science major. [math]\displaystyle{ A }[/math] is always more likely than [math]\displaystyle{ A }[/math] and [math]\displaystyle{ B }[/math]. We call people's tendency to neglect this fact the "conjunction fallacy."
Conjunction Fallacy Examples
Conjunction fallacy is often made due to our use of the representativeness heuristic—how representative something is of a category or outcome is used as a proxy. We use it to evaluate how likely the category membership or outcome is without taking base rates into account. Present one or more of the following examples.
One group of Dutch local politicians is asked how likely they think their municipality will make the headlines of all major newspapers next year, while another group is asked how likely they think their municipality will make the headlines next year due to a terrorist attack on King's Day. (For context, the terrorist attack on King's Day in 2009, in which a car drove into a crowd at the royal parade, killing 8, is a salient event to the Dutch public.) Do you expect the first group or the second group to rate their event to be more likely? Which is actually more likely?
One group of participants assesses the likelihood of an earthquake hitting California next year and causing a massive flood, while the other group assesses the likelihood of a massive flood somewhere in North America next year. Do you expect the first group or the second group to rate their event to be more likely?
These questions are posed to different people, so individual participants are not confronted with this seemingly obvious logical fallacy. In quizzes and exams, we ask the students to recognise which heuristic is at play in a given scenario, or to state what the expected experimental result will be.
Availability Heuristic
Ask your students the following classic example.
In the English language, are there more words that have "K" as the first letter OR as the third letter?
More words with "K" as the first letter.
More words with "K" as the third letter.
Explanation
It's difficult to answer this accurately because it's much easier to think of words with the first letter K than the third letter, just because of how we retrieve words from memory.
Now present and discuss one or more of the following examples.
Is it more likely that one dies due to a shark attack or due to vending machine attacks?
It's the latter, but shark attacks are more common in the news.
Repeated vivid stories about a type of events in the media inflate people's perception of the rates or likelihood of such events.
Vivid descriptions of crimes committed by immigrants skew public perception about immigration as a threat to public safety.
Mass murders and terrorist attacks provide more salient memories, compared to domestic homicides, but are actually less common. The public, however, is typically more concerned with terrorist attacks than domestic homicides.
Ask students if they can think of any other events which are commonly reported in the news but probably not actually very common, and/or events which are very common but rarely reported and therefore likely to be underestimated.
"Gratitude journals" encourage one to record only the positives of one's daily life, providing an abundance of examples of good days compared to bad days. We can use availability heuristic to our advantage to improve our mood.
Bounded Rationality
Ask your students if they agree with the following quote. Have them discuss in small groups and then bring everyone back and get the answers for the class as a whole.
"Cognitive heuristics are a hindrance to rational reasoning. They lead to poor judgment and harmful cognitive biases in decision making, and we should strive to avoid them."
