|
|
| Line 40: |
Line 40: |
|
| |
|
| </tabber> | | </tabber> |
| <restricted>
| |
|
| |
|
| == Useful Resources ==
| | {{#restricted:{{12.2 Grill the Guest}}}}{{NavCard|prev=12.1 Wisdom of Crowds and Herd Thinking|next=13.1 Denver Bullet Study}} |
| | |
| <tabber>
| |
| | |
| |-|Discussion Slides=
| |
| | |
| {{LinkCard | |
| |url=https://docs.google.com/presentation/d/1_IMaZzRLXov-r_WDBpoJpacYTi5NSOz16ONofx9tDbg/
| |
| |title=Discussion Slides Template
| |
| |description=The discussion slides for this lesson.
| |
| }}
| |
| <br />
| |
| | |
| |-|Handouts and Activities=
| |
| | |
| {{LinkCardInternal
| |
| |url=:File:Group Question Assignment.pdf
| |
| |title=Group Question Assignment
| |
| |description=Question topics assigned to different groups.}}
| |
| {{LinkCard
| |
| |url=https://docs.google.com/forms/d/1H3Lfk0i4g8EO0byEY-tSREPsrBfAXA3cyrW5CopY6aw/
| |
| |title=Question Collection Google Form Template
| |
| |description=Google form for compiling student questions.}}
| |
| {{LinkCard
| |
| |url=https://docs.google.com/document/d/1poTfTn6ilLO3_tMOzy_VSN_b2VYoVSKvm3z8jlWlXwY/
| |
| |title=Question Card Template
| |
| |description=Google doc for putting the students' questions on for them to read during class.}}
| |
| <br />
| |
| | |
| |-|Readings and Assignments=
| |
| | |
| {{LinkCardInternal
| |
| |url=:File:Toddlers Use Speech Disfluencies to Predict Speakers' Referential Intentions - Kidd, White, Aslin.pdf
| |
| |title=Toddlers Use Speech Disfluencies to Predict Speakers' Referential Intentions
| |
| |description=The paper used in UC Berkeley's Spring 2023 iteration of the course.}}
| |
| <br />
| |
| | |
| </tabber>
| |
| | |
| == Recommended Outline ==
| |
| | |
| === Before Class ===
| |
| | |
| * Someone invites a guest speaker at least one month in advance, consulting the whole teaching staff for ideas.
| |
| * Send the article by the guest to students several days in advance and remind them to read it before the discussion section. A briefly annotated version should be made to make it easier for students to skim it and pick out the important information.
| |
| * Update the division of course topics to make sense for your version of the course.
| |
| | |
| === During Class ===
| |
| | |
| {| class="wikitable" style="margin-left: 0px; margin-right: auto;"
| |
| |7 Minutes
| |
| |Short [[#Paper Review|review]] of the paper.
| |
| |-
| |
| |23 Minutes
| |
| |Go through the [[#Discussion Round 1|first round of discussion]].
| |
| |-
| |
| |50 Minutes
| |
| |Go through the [[#Discussion Round 2|second round of discussion]].
| |
| |}
| |
| | |
| === After Class ===
| |
| | |
| * Go through the questions the students submitted and put the best ones on a copy of the question card template. Make sure to print these and cut them out so you can pass them off to the students during plenary.
| |
| * Pose any remaining questions to the guest in writing and send a thank you email.
| |
| {{LinkCard
| |
| |url=https://docs.google.com/document/d/1poTfTn6ilLO3_tMOzy_VSN_b2VYoVSKvm3z8jlWlXwY/
| |
| |title=Question Card Template
| |
| |description=Google doc for putting the students' questions on for them to read during class.}}
| |
| == Lesson Content ==
| |
| | |
| === Paper Review ===
| |
| | |
| Do a '''short''' review of the paper. The students should have read the annotated version of it. But, it's likely that at least some students haven't fully grasped it yet.
| |
| | |
| === Discussion Round 1 ===
| |
| | |
| Have the students discuss and come up with answers to the following questions in small groups:
| |
| <ol start=1><li>What was the motivation for this study?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer| | |
| Prior research with adults suggests that disfluencies help facilitate sentence comprehension by signalling the presence of something new to the discourse. Researchers wanted to know if toddlers did something similar and if it plays any role in their language development.}}
| |
| <ol start=2><li>What are the hypotheses of this study?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| Whether children use indicators of processing difficulties in order to anticipate likely referents for an upcoming noun.}}
| |
| <ol start=3><li>What experiment(s) did the researchers perform to test these hypotheses?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| Two experiments main experiments involving toddlers of different ages.
| |
| * First: 2 years 4 months - 2 years 8 months.
| |
| * Second: 1 year 4 months - 2 years 2 months. (Done in two groups.)
| |
| The children were seated on their parents' lap while having their eyes tracked. Participants were presented objects that were either novel or familiar with accompanying fluent or disfluent audio. The experiment tested how much attention the toddlers paid to the objects given the audio.}}
| |
| <ol start=4><li>What are the conclusions of this study?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| That young children have learned that disfluencies contain information, attend to disfluencies in speech, and can make use of the information contained in disfluencies during comprehension in order to infer speaker intention.}}
| |
| === Discussion Round 2 ===
| |
| | |
| Have the students discuss and come up with answers to the following questions in small groups:
| |
| <ol start=1><li>What instruments were used?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| Primarily a Tobii 1750 eye-tracker. The eye-tracker keeps track of what the toddlers are looking at. The children don't have to wear anything to be eye-tracked. It was calibrated by having the children fixate on a shrinking dot located successively at one of five different screen locations.}}
| |
| <ol start=2><li>What was measured by these instruments? What's the signal? What are some sources of noise?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| The eye-tracker measured where the toddlers were looking on the screen. The software tracked the fraction of the time the child spent looking at the object after it was labelled.
| |
| * '''Signal:''' Where the toddler was looking.
| |
| * '''Noise:''' Anything that makes the tracker misidentify this.}}
| |
| <ol start=3><li>What is this used as a proxy for?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| What the child was actually paying attention to. This, in turn, is used as a proxy for the toddler's language acquisition. Eye tracking can be used a proxy for many different things depending on the specifics of the experimental setup. It's not a perfect proxy. But, it's one of the few ways to get at what's going on in the heads of really young toddlers.}}
| |
| <ol start=4><li>What intervention was performed to demonstrate causation?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| Participants were exposed to audio with and without disfluencies to see if their attention differed. The presence of the disfluency in the audio was the intervention.}}
| |
| <ol start=5><li>What are some confounding factors that the researchers controlled for? How did they control for them?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| How interesting the objects and their labels were.
| |
| : Novel and known objects were paired by color/shape. Labels were paired to be have similar sorts of sounds that were as easy to learn.
| |
| The length of the audio given the disfluency.
| |
| : An additional “look…” was added to the third presentation of the object in both fluent and disfluent audio to ensure the toddlers were prompted at the same amount of time before the window of interest.
| |
| The use of “the” before disfluencies in the real world.
| |
| * Prior work shows that 81% of the time, “thee” is followed by a suspension of speech.
| |
| * Disfluencies were preceded by “thee.”
| |
| Cues by the parents.
| |
| * Parents wore headphones playing music to mask auditory stimuli.
| |
| * They were instructed to keep their gaze downwards during the experiment.}}
| |
| <ol start=6><li>What are some confounding factors that the researchers might not have controlled for?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| Were there unknown systematic factors between the novel and known objects? Whether the toddlers were learning the disfluencies as part of the objects' names.
| |
| : This was accounted for only in the analysis by seeing whether or not the toddlers paid less attention to the objects as the trials went on.
| |
| How toddlers were recruited. Are they from similar backgrounds?
| |
| * Maybe parents that have the time to bring kids into studies also have more time to spend with their kids in general?
| |
| * Lots of other possible systematic biases that could affect how externally valid the study is.
| |
| Sample size.
| |
| * It is hard to recruit toddlers. This is not a lot of them.
| |
| * The sample is small enough that there could be systematic differences by random chance.
| |
| This is only tested with one language. Maybe this is just a feature of English? Or is cultural? Lots of other possible confounds as well…}}
| |
| <ol start=7><li>What was at least one alternative hypothesis for the study's findings?</li></ol>
| |
| {{BoxAnswer|title=2023 Answer|
| |
| Children pay more attention overall to the display during disfluencies. Children interpreted some component of the disfluency as a novel object label.}}
| |
| === Developing Questions ===
| |
| | |
| Assign the groups numbers from 1 to 6. Give the groups the following instructions.
| |
| * Based on your group number, you will be given a subset of course concepts to develop questions about.
| |
| * You and your group must write at least two questions and submit them via the form on the right.
| |
| * Further instructions are provided in the form.
| |
| * Links to the form and concepts are also in today's section of the syllabus.
| |
| The relevant course concepts and group assignments are detailed below.
| |
| {{LinkCardInternal
| |
| |url=:File:Group Question Assignment.pdf
| |
| |title=Group Question Assignment
| |
| |description=Question topics assigned to different groups.}}
| |
| {{LinkCard
| |
| |url=https://docs.google.com/forms/d/1H3Lfk0i4g8EO0byEY-tSREPsrBfAXA3cyrW5CopY6aw/
| |
| |title=Question Collection Google Form Template
| |
| |description=Google form for compiling student questions.}}
| |
| ==== Group/Category 1: Getting at Reality ====
| |
| | |
| * [[1.2 Shared Reality and Modeling]]
| |
| ** Raft vs. pyramid
| |
| ** Evaluation of models
| |
| ** Scientific realism vs. anti-realism
| |
| ** Operationalism, conventionalism, and realism
| |
| * [[2.1 Senses and Instrumentation]]
| |
| ** Validation of instruments through interactive exploration, triangulation with other instruments, and comparison with direct senses
| |
| | |
| ==== Group/Category 2: Error Analysis ====
| |
| | |
| * [[2.2 Systematic and Statistical Uncertainty]]
| |
| ** Measurement proxies as sources of systematic and statistical uncertainty
| |
| * [[3.1 Probabilistic Reasoning]] & [[3.2 Calibration of Credence Levels]]
| |
| ** <math>p</math>-values and statistical significance
| |
| ** Error bars and confidence intervals
| |
| | |
| ==== Group/Category 3: Research Quality and Refinement ====
| |
| | |
| * [[4.1 Signal and Noise]] & [[4.2 Finding Patterns in Random Noise]]
| |
| ** Signal-to-noise ratio
| |
| ** <math>p</math>-hacking
| |
| ** Look elsewhere effect
| |
| ** HARKing (hypothesizing after results are known)
| |
| ** Effect size (as distinct from statistical significance)
| |
| * [[5.1 False Positives and Negatives]]
| |
| ** Thresholds between positive and negative detections
| |
| ** Trade offs between false positives and negatives
| |
| * [[5.2 Scientific Optimism]]
| |
| ** Iterative progress
| |
| | |
| ==== Group/Category 4: Untangling Causation ====
| |
| | |
| * [[6.1 Correlation and Causation]]
| |
| ** Causation as correlation under intervention
| |
| ** Randomized controlled trials
| |
| ** Spurious correlations
| |
| * [[6.2 Hill's Criteria]] (Causation in the Messy Real World)
| |
| ** Natural experiments
| |
| ** Hill's criteria for causation
| |
| ** Causal networks
| |
| | |
| ==== Group/Category 5: Understanding Mechanisms ====
| |
| | |
| * [[7.2 Emergent Phenomena]]
| |
| ** Global effects that arise through the interaction of small pieces (rather than general causation)
| |
| ** Explanation at different scales
| |
| ** Scientific reductionism
| |
| * [[8.1 Orders of Understanding]]
| |
| ** Orders of magnitude
| |
| ** Multiple causes of comparable importance
| |
| ** Orders of importance of causes
| |
| ** Refinement of models using higher order descriptions
| |
| | |
| ==== Group/Category 6: Research Practice ====
| |
| {{BoxCaution|Blind analysis is a common and easy topic to criticize on, however the students should be reminded that it is not yet widespread in many fields of science, and perfectly valid scientific results can still be obtained without it.}}
| |
| * [[10.1 Confirmation Bias]]
| |
| ** Selective exposure
| |
| ** Biased assimilation
| |
| ** Strategies for reducing confirmation bias (AOT, etc.)
| |
| * [[10.2 Blinding]]
| |
| ** Techniques for blind analysis
| |
| ** Preregistration
| |
| ** Registered replication
| |
| ** Adversarial collaboration
| |
| ** Peer review
| |
| * [[11.1 Pathological Science]]
| |
| ** Langmuir's pathological science indicators
| |
| * [[11.2 When Is Science Suspect]]
| |
| ** The validity and reliability of social science metrics
| |
| ** External validity
| |
| ** The difficulty of creating and applying social science metrics in cross-cultural contexts
| |
| ** How researchers' biases and backgrounds shape their research agendas</restricted>{{NavCard|prev=12.1 Wisdom of Crowds and Herd Thinking|next=13.1 Denver Bullet Study}}
| |
| [[Category:Lesson plans]] | | [[Category:Lesson plans]] |
