Sensation and Perception (II) 3270 Speech

1. Sensation and Perception (II) 3270 Speech

2. Keywords on hearing (introduction) auditory canal, ear drum, ossicles, oval window, cochlea, helicotrema, basilar membrane, tectorial membrane, hair cells, kinocilium, stereocilia, amplification (by ossicles & area difference between ear drum and oval window), travelling wave, resonance, tonotopic coding, cochleotopic coding, transduction auditory thresholds, effect of age, different animals, fundamental, harmonics, timbre, pitch/frequency, loudness/amplitude, pure tone, equal loudness, masking place theory, periodicity theory, duplicity theory, missing fundamental, goldfish has no basilar membrane - can distinguish freqs, phase-locking, diplacusis,

3. PHONEMES “sounds that create meaning” 47 in English VOICING PLACE OF MANNER OF ARTICULATION ARTICULATION voiced alveloar ridge stop unvoiced labiodental fricative etc… nasal etc… http://nationalstrategies.standards.dcsf.gov.uk/node/85357

5. Formant 3 Formant 2 Formant 1 time

6. SEGMENTATION PROBLEM Formant transitions Formants

7. Segmentation problem Where one sound is influenced by what sound came before or after because of CO-ARTICULATION

8. VARIABILITY PROBLEM Same sounds, different spectrographs...

9. IS SPEECH SPECIAL? • MOTOR THEORY OF SPEECH • -- mirror neurones? • http://www.teachersdomain.org/asset/hew06_vid_mirrorneurons/

10. IS SPEECH SPECIAL? 2. CATEGORICAL PERCEPTION

11. Long VOT Short VOT Voice Onset Time (VOT)

12. “SAME” “DIFFERENT”

13. IS SPEECH SPECIAL? 3. SPEECH PERCEPTION IS MULTIMODAL

14. SOUND VISION McGurk Effect

15. Video demos 1. McGurk effect video=ga sound = ba combo= da 2. Vision helps degraded sound

16. IS SPEECH SPECIAL? 4. Are there INVARIANTS for phonemes? -- practical importance -- some hints -- varied by neighbouring sound -- different conditions eg. speed of speech -- different speakers -- but TOP DOWN INFLUENCES

17. TOP-DOWN INFLUENCES • Segmentation influenced by meaning • Anna Mary Candy Lights Since Imp Pulp • Lay Things. • I scream, you scream, we all scream for ice cream!!

18. TOP DOWN INFLUENCES • Segmentation influenced by meaning • Semantics (meaning) and syntax (grammar) both influence your ability to shadow (repeat what you have heard) a text. 89% 79% 56% percent correct ungrammatical sentences normal sentences anomalous sentences

19. cough * • TOP DOWN INFLUENCES • Segmentation influenced by meaning • Semantics (meaning) and syntax (grammar) both influence your ability to shadow (repeat what you have heard) a text. • Phonemic restoration. • Time to meet with their respective legi latures s time to ave w the flag s the country Pink noise demo: http://www.youtube.com/watch?v=k74KCfSDCn8 Legislatures: http://www.youtube.com/watch?v=UlJs24j3i8E&feature=related

21. SUMMARY “Is speech special?” • Motor theory of speech • Categorical perception • Multimodal aspects • Invariants • Top down influences • (i) Segmentation influenced by meaning • (ii) Shadowing influenced by meaning • (iii) Phonemic restoration

22. PHYSIOLOGY OF SPEECH PERCEPTION

23. Might there be phoneme detectors in the brain? /da/ 100% 0% Percentage heard as “ba” 50% 50% Percentage heard as “da” /ba/ 100% 0% 0 40 20 60 80 Voice onset time (ms)

24. ADAPT TO /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/ /ba/

25. /da/ /ba/ After adapting to /ba/…… 100% 0% Percentage heard as “ba” 50% 50% Percentage heard as “da” 100% 0% 0 40 20 60 80 Voice onset time (ms)

26. As if that wasn’t cool enough….. What happens after adapting to a McGurk effect?

27. acoustic Sound = /ba/ Vision = /ga/ Perception = /da/ perception

28. /da/ If /ba/ (acoustic) adapts If /da/ (perception) adapts /ba/ After adapting to McGurk 100% 0% Percentage heard as “ba” 50% 50% Percentage heard as “da” ?? 100% 0% 0 40 20 60 80 Voice onset time (ms)

29. /da/ Acoustic adapts!!! /ba/ It is the ACOUSTIC signal that adapts 100% 0% Percentage heard as “ba” 50% 50% Percentage heard as “da” 100% 0% 0 40 20 60 80 Voice onset time (ms)

31. Broca’s aphasia http://www.youtube.com/watch?v=f2IiMEbMnPM Wernicke’s aphasia http://www.youtube.com/watch?v=B-LD5jzXpLE

32. Cells in cortex show: • Phoneme information carried in a population code • Special features in auditory cortex (eg, frequency sweeps) compatible with phonemic features

33. Green = average blue = less red = more Background resting activity -- note more activity in frontal regions

34. Frontal eye fields Visual cortex Looking around

35. WERNICKE’S AREA Auditory cortex Listening to words

36. Supplementary motor area BROCA’S AREA (speech production) WERNICKE’S AREA (speech understanding) Counting out loud

37. ????? Internal speech -- counting in your head

38. Frontal regions NOT Broca or Wernicke’s areas !!!! Internal speech -- counting in your head

41. KEYWORDS -- SMELL I olfactory binding protein, olfactory receptors cells continuously regenerate (about every 60 days), cilia (on olfactory receptor cells), glomerulus (contact zones between receptor cells and mitral cells:plural glomeruli), convergence (1,000:1), mitral cell, olfactory tubercle of cortex (part of paleocortex), medial dorsal nucleus of thalamus , orbitofrontal cortex, olfactory neocortex paleocortex associated with limbic system, limbic system associated with emotions (electrical stimulation causes sham rage), limbic system associated with memories (H.M. had lesions here and lost the ability to memorize things), some hot spots in olfactory tubercle and on olfactory mucosa

42. KEYWORDS -- SMELL II odour quality, no primaries identified in olfactory system, poor tuning of receptors (to chemicals or chemical types) (sharpened by lateral inhibition, inhibitory interneurones, granule cells), Henning smell prism, stereochemical theories based on lock and key partially successful, BUT no receptor sites identified, similar shaped molecules can be associated with different smell perceptions cells broadly tuned (responding to many different chemicals associated with many different smells) coding intensity= firing rate/recruitment, quality = distributed pattern code, problems in identifying many smells at once, binding problem

43. KEYWORDS -- SMELL III odour thresholds, olfactorium; unique technical problems!, humans very sensitive (eg. mercaton can be detected at 1 part per 50,000,000,000), affected by gender; can be affected by menstrual cycle, affected by age adaptation, thresholds raised (by exposure), masking (by other chemicals), some cross effects: eg. adapting to orange affects smell of lemons identification, can identify gender from shirt, prefer own odours, odour memories long lasting; associated with emotions (via limbic system) "designed not to forget”, pheromones, releasers (immediate effect), eg. bitch on heat, territorial markers, humans?, McClintock effect (synchronized menstrual cycles), primers (longer term) eg. mice need males around for proper oestrus cycles

44. KEYWORDS -- SMELL IV PATHWAYS olfactory receptor cells to mitral cells in olfactory bulb to olfactory tubercle in paleocortex THEN 1 to medial dorsal thalamus to olfactory cortex (ORBITOFRONTAL CORTEX) 2 to limbic system 3 brain stem pathways associated with pheromones ALSO inhibitory pathway (via inhibitory interneurone: granule cells) from one olfactory bulb to the other to do with detecting the DIRECTION from which a smell originates