13.1 Denver Bullet Study: Difference between revisions

From Sense & Sensibility & Science
(// Edit via Wikitext Extension for VSCode)
(// Edit via Wikitext Extension for VSCode)
Line 5: Line 5:
{{Navbox}}
{{Navbox}}


== Useful Links ==
== The Lesson in Context ==


* [[:File:13.1 Denver Bullet Study - Worksheet.pdf|Glorin-Fuilder Fireworks Study Handout]]
<!-- Always begin section with a description of this lesson in relation to the course as a whole. -->
* [https://docs.google.com/document/d/1vjfMJ0dPmZFi2BcbIyDeUbBSKGCVHmeMV4t0SZsjL3U/edit?usp=sharing Three Column Overview of the Week]
Near the end of the course, we introduce some techniques for group decision making that have seen some real-world success. Denver Bullet Study exemplifies the division of a contentious issue into facts and values in such a way that the community was able to come to a more generally satisfying decision than initially seemed possible.
* [https://docs.google.com/presentation/d/1ceysMd4bCqIbLsDOmZEsAvyxNvrhApdASCAIVpASFlU/edit?usp=drivesdk Lesson Slides (2023 Master)]
* [https://sensesensibilityscience.berkeley.edu/topic/24 Website Page]


=== Readings and Assignments ===
<!-- Expandable section relating this lesson to earlier lessons. -->
{{Expand|Relation to Earlier Lessons|
{{ContextLesson|1.1 Introduction and When Is Science Relevant}}
{{ContextRelation|Facts and values of an issue are distinguished in the Denver Bullet Study method. Facts are evaluated by experts, while values are decided by polling stakeholders.}}
{{ContextLesson|5.2 Scientific Optimism}}
{{ContextRelation|Even as the public seem to have reached an impasse on a contentious issue, there are still techniques that can be used to reach a consensus, or at least find a solution that addresses the priorities of a larger number of stakeholders.}}
}}
<!-- Expandable section relating this lesson to later lessons. -->
{{Expand|Relation to Later Lessons|
{{ContextLesson|13.2 Deliberative Polling}}
{{ContextRelation|Another technique for group decision making in which experts and stakeholders are involved and facts and values are distinguished. Through group deliberation punctuated by Q&A with an expert panel, it aims to predict the change in public opinion if the public were to be more factually informed on a contentious issue, but it does not aim to reach a consensus or directly decide on a policy.}}
}}


* [[:File:Science, Values, and Human Judgment - Hammond, Adelman.pdf|Science, Values, and Human Judgment]]
== Takeaways ==


==== Lecture Video ====
<tabber>


<youtube>https://youtu.be/Qs6kZ0dv_Ec</youtube>
|-|Learning Goals=
 
== Learning Goals ==


After this lesson, students should
After this lesson, students should
<!-- Learning goals are written as a numbered list. -->
# Be optimistic that a community can come together to make a decision, even when people begin with heterogeneous values and beliefs.
# Be optimistic that a community can come together to make a decision, even when people begin with heterogeneous values and beliefs.
# Be able to identify the stakeholders and experts and distinguish between the values and facts of an issue.
# Be able to identify the stakeholders and experts and distinguish between the values and facts of an issue.


=== Definitions ===
|-|Definitions=
 
<!-- Definitions must be written with the Definition and Subdefinition templates. The first Definition should have the "first=yes" flag at the end. -->
{{Definition|Stakeholders|The set of people who have a stake in the outcome of a decision. This can include people who will implement the decision and all the people affected by it.|first=yes}}
{{Definition|Experts|The set of people who have the most knowledge/information/expertise about the facts relevant to the decision, generally by virtue of having spent the most time and effort learning about the topic.}}
{{Definition|Denver Bullet Study|An experiment in group deliberation in which a community came together to share values and knowledge to make a decision about what kind of bullet the Denver Police should use, which had enough stopping power to keep cops safe but was not so harmful as to cause unnecessary damage to citizens (as did classic hollow bullets).}}
<br />
 
|-|Common Misconceptions=


* '''Stakeholders'''
<!-- Misconceptions must be written with the Misconception template. The first Misconception should have the "first=yes" flag at the end. -->
*: The set of people who have a stake in the outcome of a decision. This can include people who will implement the decision and all the people affected by it.
{{Misconception|The town should vote on whether they think these windmills kill seabirds or cause loud noise.|These are factual factors that most stakeholders (townspeople) would not have informed opinions on. Instead, they should be evaluated by relevant experts. Stakeholders can then vote (or put numerical weights) on how much they value the lives of seabirds or quietness.|first=yes}}
* '''Experts'''
{{Misconception|We should just let the experts make all the decisions.|It is important to involve the public (or relevant stakeholders) in making group decisions, as factual evaluation must be combined with a group's values judgment (of what they deem important). Experts can give their evaluation of factual matters, but they may not represent the values of the stakeholders.}}
*: The set of people who have the most knowledge/information/expertise about the facts relevant to the decision, generally by virtue of having spent the most time and effort learning about the topic.
* '''Denver Bullet Study'''
*: An experiment in group deliberation in which a community came together to share values and knowledge to make a decision about what kind of bullet the Denver Police should use, which had enough stopping power to keep cops safe but was not so harmful as to cause unnecessary damage to citizens (as did classic hollow bullets).


=== Common Misconceptions ===
</tabber>


* ''The town should vote on whether they think these windmills kill seabirds or cause loud noise.''
== Useful Resources ==
*: These are factual factors that most stakeholders (townspeople) would not have informed opinions on. Instead, they should be evaluated by relevant experts. Stakeholders can then vote (or put numerical weights) on how much they value the lives of seabirds or quietness.
* ''We should just let the experts make all the decisions.''
*: It is important to involve the public (or relevant stakeholders) in making group decisions, as factual evaluation must be combined with a group's values judgment (of what they deem important). Experts can give their evaluation of factual matters, but they may not represent the values of the stakeholders.


== Context ==
<tabber>


Near the end of the course, we introduce some techniques for group decision making that have seen some real-world success. Denver Bullet Study exemplifies the division of a contentious issue into facts and values in such a way that the community was able to come to a more generally satisfying decision than initially seemed possible.
|-|Lecture Video=
 
<br /><center><youtube>Qs6kZ0dv_Ec</youtube></center><br />
 
|-|Discussion Slides=
 
{{LinkCard
|url=https://docs.google.com/presentation/d/1ceysMd4bCqIbLsDOmZEsAvyxNvrhApdASCAIVpASFlU/
|title=Discussion Slides Template
|description=The discussion slides for this lesson.
}}
<br />
 
|-|Handouts and Activities=


=== Before ===
{{LinkCardInternal
|url=:File:13.1 Denver Bullet Study - Worksheet.pdf
|title=Glorin-Fuilder Fireworks Study Handout
|description=Handout for all parts of the Glorin-Fuilder fireworks study activity.}}
<br />


: '''[[1.1 Introduction and When Is Science Relevant]]'''
|-|Readings and Assignments=
:: Facts and values of an issue are distinguished in the Denver Bullet Study method. Facts are evaluated by experts, while values are decided by polling stakeholders.
: '''[[5.2 Scientific Optimism]]'''
:: Even as the public seem to have reached an impasse on a contentious issue, there are still techniques that can be used to reach a consensus, or at least find a solution that addresses the priorities of a larger number of stakeholders.


=== After ===
{{LinkCardInternal
|url=:File:Science, Values, and Human Judgment - Hammond, Adelman.pdf
|title=Science, Values, and Human Judgment
|description=Original paper outlining the Denver Bullet Study.}}
<br />


: '''[[13.2 Deliberative Polling]]'''
</tabber>
:: Another technique for group decision making in which experts and stakeholders are involved and facts and values are distinguished. Through group deliberation punctuated by Q&A with an expert panel, it aims to predict the change in public opinion if the public were to be more factually informed on a contentious issue, but it does not aim to reach a consensus or directly decide on a policy.


== Recommended Outline ==
== Recommended Outline ==
Line 62: Line 89:
=== Before Class ===
=== Before Class ===


* [Any essential logistical things that need to be done for this class]
Print out one copy of the handout for each student.
* Prepare a seating chart.
* Review PlayPosit and discussion questions and ask faculty, Gabriel, or Emlen any questions you have.
* (Optional) Prepare a presentation.


=== During Class ===
=== During Class ===


* (5 min) Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
{| class="wikitable" style="margin-left: 0px; margin-right: auto;"
* (10 min) Go through the [[#Review Questions|review questions]].
|5 Minutes
* (20 min) Have the students work through [[#Part 1: Diagraming the Study|part 1]] of the Glorin-Fuilder study.
|Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
* (20 min) Have the students work through [[#Part 2: Finishing the Study|part 2]] of the Glorin-Fuilder study.
|-
* (20 min) Lead the students through the [[#Campus Development|campus development]] discussion.
|10 Minutes
* (5 min) Have the students answer the [[#Final Questions|final questions]].
|Go through the [[#Review Questions|review questions]].
 
|-
=== After Class ===
|20 Minutes
 
|Have the students work through [[#Part 1: Diagraming the Study|part 1]] of the Glorin-Fuilder study.
* Collect answers from notetakers for the forum / plenary.
|-
|20 Minutes
|Have the students work through [[#Part 2: Finishing the Study|part 2]] of the Glorin-Fuilder study.
|-
|20 Minutes
|Lead the students through the [[#Campus Development|campus development]] discussion.
|-
|5 Minutes
|Have the students answer the [[#Final Questions|final questions]].
|}


== Lesson Content ==
== Lesson Content ==
Line 167: Line 200:
==== Complete the Table ====
==== Complete the Table ====


{| class="wikitable floatright"
{| class="wikitable"
|+Total goodness table:
|+Total goodness table:
|-
|-
Line 196: Line 229:
|}
|}


# Use the preferences given (see the two tables above) to fill out the "total goodness" table on the right.
<ol start=1><li>Use the preferences given (see the two tables) to fill out the "total goodness" table on the right.</li></ol>
# Which firework would you choose if you are a person from Fuilder? Which would you choose if you are from Glorin?
<ol start=2><li>Which firework would you choose if you are a person from Fuilder? Which would you choose if you are from Glorin?</li></ol>
#: {{Answer|Fuilder like Fire Tornado. Fire Tornado BIG. Glorin like Warm Greetings. Warm Greetings smol.}}
{{BoxAnswer|Fuilder like Fire Tornado. Fire Tornado BIG. Glorin like Warm Greetings. Warm Greetings smol.}}
# Which firework would you choose as an advisor for the prince?
<ol start=3><li>Which firework would you choose as an advisor for the prince?</li></ol>
#: {{Answer|Kumperdinck like Dancing Butterfly. Dancing Butterfly politically optimal for maintaining tenuous grip of power over the worthless peons ruled over.}}
{{BoxAnswer|Kumperdinck like Dancing Butterfly. Dancing Butterfly politically optimal for maintaining tenuous grip of power over the worthless peons ruled over.}}
# Explain how this technique produces/outputs an optimal type of firework.
<ol start=4><li>Explain how this technique produces/outputs an optimal type of firework.</li></ol>
#: {{Answer|By assigning numerical scores to all the values and finding the result that is most preferred. This seems very utilitarian. But, the nice thing is that it allows all the stakeholders to define ''their own'' utility function. In a real case you likely assign the value scores by giving people certain amounts of points to distribute. This helps bound people's values in a practical way.}}
{{BoxAnswer|By assigning numerical scores to all the values and finding the result that is most preferred. This seems very utilitarian. But, the nice thing is that it allows all the stakeholders to define ''their own'' utility function. In a real case you likely assign the value scores by giving people certain amounts of points to distribute. This helps bound people's values in a practical way.}}
 
=== Campus Development ===
=== Campus Development ===
 
{{BoxCaution|This discussion was written specifically for UC Berkeley. You may want to switch it out for something more relevant to your students.}}
{{Caution|In its smallest form this is a small group activity. But, it could potentially be expanded to a large course-wide activity in a future iteration.}}
 
There are talks of developing the Oxford Field Track (a large plot of land owned by UC Berkeley in Northside near downtown). It currently holds farms, greenhouses, and other research facilities for the College of Natural Resources. Suppose you are a campus administrator tasked with using a method like that of the Denver Bullet Study to determine which of the following development proposals to adopt.
There are talks of developing the Oxford Field Track (a large plot of land owned by UC Berkeley in Northside near downtown). It currently holds farms, greenhouses, and other research facilities for the College of Natural Resources. Suppose you are a campus administrator tasked with using a method like that of the Denver Bullet Study to determine which of the following development proposals to adopt.
# Leave the land as is for CNR.
# Leave the land as is for CNR.
Line 222: Line 252:


* What is a potential policy decision that you ''should not'' decide with a Denver Bullet Study method?
* What is a potential policy decision that you ''should not'' decide with a Denver Bullet Study method?
* What's a current or former policy proposal that ''would'' benefit from a Denver Bullet Study style decision?
* What's a current or former policy proposal that ''would'' benefit from a Denver Bullet Study style decision?{{NavCard|prev=12.2 Grill the Guest|next=13.2 Deliberative Polling}}
 
{{NavCard|prev=12.2 Grill the Guest|next=13.2 Deliberative Polling}}
[[Category:Lesson plans]]
[[Category:Lesson plans]]

Revision as of 07:46, 15 August 2023

Topic Cover - 13.1 Denver Bullet Study.png

The Denver Bullet Study offers one approach to integrating facts and values in a controversial real-world problem, drawing facts from a set of experts, gauging the values of different stakeholders, and bringing these together for a final decision.



The Lesson in Context

Near the end of the course, we introduce some techniques for group decision making that have seen some real-world success. Denver Bullet Study exemplifies the division of a contentious issue into facts and values in such a way that the community was able to come to a more generally satisfying decision than initially seemed possible.

1.1 Introduction and When Is Science RelevantTopic Icon - 1.1 Introduction and When Is Science Relevant.png
  • Facts and values of an issue are distinguished in the Denver Bullet Study method. Facts are evaluated by experts, while values are decided by polling stakeholders.
5.2 Scientific OptimismTopic Icon - 5.2 Scientific Optimism.png
  • Even as the public seem to have reached an impasse on a contentious issue, there are still techniques that can be used to reach a consensus, or at least find a solution that addresses the priorities of a larger number of stakeholders.
13.2 Deliberative PollingTopic Icon - 13.2 Deliberative Polling.png
  • Another technique for group decision making in which experts and stakeholders are involved and facts and values are distinguished. Through group deliberation punctuated by Q&A with an expert panel, it aims to predict the change in public opinion if the public were to be more factually informed on a contentious issue, but it does not aim to reach a consensus or directly decide on a policy.


Takeaways

After this lesson, students should

  1. Be optimistic that a community can come together to make a decision, even when people begin with heterogeneous values and beliefs.
  2. Be able to identify the stakeholders and experts and distinguish between the values and facts of an issue.

Stakeholders

The set of people who have a stake in the outcome of a decision. This can include people who will implement the decision and all the people affected by it.

Experts

The set of people who have the most knowledge/information/expertise about the facts relevant to the decision, generally by virtue of having spent the most time and effort learning about the topic.

Denver Bullet Study

An experiment in group deliberation in which a community came together to share values and knowledge to make a decision about what kind of bullet the Denver Police should use, which had enough stopping power to keep cops safe but was not so harmful as to cause unnecessary damage to citizens (as did classic hollow bullets).

The town should vote on whether they think these windmills kill seabirds or cause loud noise.

These are factual factors that most stakeholders (townspeople) would not have informed opinions on. Instead, they should be evaluated by relevant experts. Stakeholders can then vote (or put numerical weights) on how much they value the lives of seabirds or quietness.

We should just let the experts make all the decisions.

It is important to involve the public (or relevant stakeholders) in making group decisions, as factual evaluation must be combined with a group's values judgment (of what they deem important). Experts can give their evaluation of factual matters, but they may not represent the values of the stakeholders.

Useful Resources



Discussion Slides Template

The discussion slides for this lesson.


Recommended Outline

Before Class

Print out one copy of the handout for each student.

During Class

5 Minutes Come up with some fun way to assign the roles of spokesperson and notetaker (e.g. earliest birthday in the year, lives furthest from campus). Remind them of the responsibilities of these roles.
10 Minutes Go through the review questions.
20 Minutes Have the students work through part 1 of the Glorin-Fuilder study.
20 Minutes Have the students work through part 2 of the Glorin-Fuilder study.
20 Minutes Lead the students through the campus development discussion.
5 Minutes Have the students answer the final questions.

Lesson Content

Review Questions

A diagram of the Denver Bullet Study process.
  1. Should we just let experts make all the decisions in society? Why or why not?
  2. What is the role of values in the Denver Bullet Study?
  3. What is the role of facts in the Denver Bullet Study?
  4. How does the Denver Bullet Study synthesize facts and values in arriving at a group decision?
    This is a common quiz question!
  5. Suppose all policy decisions were made in a process like the Denver Bullet Study. Would such a country be a democracy or an epistocracy? (Answer: A bit of both)

Part 1: Diagraming the Study

Background

A potential diagram of the Glorin-Fuilder Study process.

Suppose you are the advisor to Prince Kumperdinck of Glorin, who is trying to develop an agreement with the neighboring nation, Fuilder.

The bakers of Glorin, who live close to the Glorin-Fuilder border, have been repeatedly disrupted by Fuilder's tendency to launch loud, bright fireworks late at night (this makes it difficult for the bakers to get up early to prepare the day's bread). The bakers of Glorin believe that the residents of Fuilder should only be allowed to use small, silent sparklers, such as the “Warm Greetings."

The residents of Fuilder consider this solution inconceivable, because launching true fireworks each night is part of their cultural tradition and helps maintain the happiness of the nation. They have been using the “Fire Tornado” for over 100 years since its invention and they don't want to relinquish such flamboyance that represents their culture.

There is rising tension between the citizens of Glorin and Fuilder, as neither side is willing to back down. Thinking back to your favorite class in advisor school, you recall a decision-making technique that involved integration of facts and values to come to a compromise in a similar scenario (the Denver Bullet Study). Prince Kumperdinck puts you in charge of implementing this technique. Your hope is that you can select a kind of firework that will satisfy both parties as much as possible.

Questions

  1. Suppose you are a citizen of one of the two towns. What are some of the factors of these fireworks that you might be concerned about?
  2. The actual citizens of Glorin and Fuilder have been polled on this question. It turns out that the two dominant factors they care about are loudness and brightness. Create a diagram based on the one used for the Denver Bullet Study.

Stakeholders make a list of factors they care about. These are the boxes in column two. The stakeholders assign weights based on how much they care about each of these factors. Then experts make a list of measurable quantities that contribute to the factors the stakeholders are concerned with. These are column one. The experts are blinded as to how much the stakeholders care about each of the items in column two. Then you determine the best final outcome.

Part 2: Finishing the Study

Students are often surprised by the need to do arithmetic in this lesson. Make sure to remind them of the meaning of the numbers on the handout and be on the lookout for anyone that seems lost.

Discuss Relevant Stakeholders

Discuss who the relevant stakeholders and experts are in this situation and describe their roles in the decision-making process. (Note: you are not inventing a decision-making process. You are implementing the process discussed in class for integrating facts and values, exemplified by the Denver Bullet Study.)

Now that you have the different criteria for the fireworks, you decided to conduct a survey among the citizens to get their preferences of each criteria. You ask them on a scale of -10 to 10, how much they like loud or bright fireworks. 10 is the highest preference while -10 is the lowest preference, and 0 is neutral. A negative number would indicate that they hate loud/bright fireworks. The results of the survey are tabulated below.

Citizen's preference based on a survey you conducted:
Goodness of Loudness Goodness of Brightness
Average Fuilder 1 10
Average Glorin −10* 1
*Negative because people in Fuilder really hate loud fireworks. They would strongly prefer the opposite of loudness.

At the same time, your colleague spent time with the firework experts and got some physical measurements of the available fireworks. The numbers are normalized so that 10 means the most loud/bright and 1 means the least loud/bright. The results are tabulated below.

Available fireworks measured by experts:
Loudness Measured Brightness Measured
Fire Tornado 8 10
Warm Greetings 2 3
Happy Flower 5 6
Dancing Butterfly 4 9

Complete the Table

Total goodness table:
Total Goodness for Fuilder Total Goodness for Glorin Total Goodness for All
Fire Tornado 108 -70 38
Warm Greetings 32 -17 15
Happy Flower 65 -44 21
Dancing Butterfly 94 -31 63
  1. Use the preferences given (see the two tables) to fill out the "total goodness" table on the right.
  1. Which firework would you choose if you are a person from Fuilder? Which would you choose if you are from Glorin?

Fuilder like Fire Tornado. Fire Tornado BIG. Glorin like Warm Greetings. Warm Greetings smol.

  1. Which firework would you choose as an advisor for the prince?

Kumperdinck like Dancing Butterfly. Dancing Butterfly politically optimal for maintaining tenuous grip of power over the worthless peons ruled over.

  1. Explain how this technique produces/outputs an optimal type of firework.

By assigning numerical scores to all the values and finding the result that is most preferred. This seems very utilitarian. But, the nice thing is that it allows all the stakeholders to define their own utility function. In a real case you likely assign the value scores by giving people certain amounts of points to distribute. This helps bound people's values in a practical way.

Campus Development

This discussion was written specifically for UC Berkeley. You may want to switch it out for something more relevant to your students.

There are talks of developing the Oxford Field Track (a large plot of land owned by UC Berkeley in Northside near downtown). It currently holds farms, greenhouses, and other research facilities for the College of Natural Resources. Suppose you are a campus administrator tasked with using a method like that of the Denver Bullet Study to determine which of the following development proposals to adopt.

  1. Leave the land as is for CNR.
  2. Build high density student housing on it.
  3. Turn the land into a public park.
  4. Build academic buildings to hold classrooms and offices.

Following the Denver Bullet Study method, answer the following questions in small groups.

  1. Who are the stakeholders?
  2. Who might the experts be?
  3. List some positive and negative consequences of the above plans that the stakeholders might value.
  4. How might the experts determine the consequences of each proposal?

Final Questions

  • What is a potential policy decision that you should not decide with a Denver Bullet Study method?
  • What's a current or former policy proposal that would benefit from a Denver Bullet Study style decision?
⟨

Previous

12.2 Grill the Guest

⟩

Next

13.2 Deliberative Polling

Retrieved from "https://sensibility.berkeley.edu/index.php?title=13.1_Denver_Bullet_Study&oldid=6080"