Psychoacoustics and Music Perception

1. Psychoacoustics and Music Perception 509.211 VO, 2st. S06, Mi 17:30-19:15 HS 06.03 Richard Parncutt Email: ((my last name))@uni-graz.at Office hours: Thursday 10 am

2. This file is… • available in the internet and updated regularly • only a PART of the course material. Missing: • verbal explanations in lectures • figures drawn on board and displayed with OHP (transparencies) • contents of folder in reading room of department library • sound examples (including those linked to this document – but many of these are on the CD in the folder) • written in point form – but exam answers must be complete sentences! (see “Schriftliche Prüfungen”) Questions and suggestions? ((familyname))@uni-graz.at

3. Lecture 1, 8.3.06 • Adminstrative details • aims • dates • examination • Introduction: Musical relevance of psychoacoustics • Course outline, literature • Philosophy of perception: the “3 worlds” of Popper & Eccles (1977) Literature: • Parncutt, R. (in press). Psychoacoustics and music perception • Terhardt, E. (1998). Akustische Kommunikation. Berlin: Springer. (1. Kapitel)

4. Musical relevance Consider some everyday musical examples: • J. S. Bach: „O Haupt voll Blut und Wunden“ („Baroque choral“) • Frank Sinatra: „White Christmas“ („pop“) • Miles Davis „So what“ („modal jazz“) • Igor Stravinsky: „Sacré du printemps“ („modern orchestral“) Consider some psychological issues: • What do you hear or experience in this music? • Chain: physics – perception – structure – associations • Direct perception: ecological psychology • Indirect perception: cognitive psychology

5. Relevance for music analysis • Perception of pitch structures • harmony, voice-leading, phrasing, tonality, modulation • Quality of sound • consonance/dissonance, timbre • Cognitive organisation • foreground, background • Emotional character • associations Not considered: • Accents: dynamic, grouping, metrical, melodic, harmonic • Expressive timing and dynamics

6. Some course aims • Overview and understand • musically relevant fundamentals of psychoacoustics • perceptual correlates of music-theoretical concepts (cons./diss., root/tonic) • Understand technical primary literature • extract relevant information from it • Show relevance for music theory and analysis • Contribute to understanding of musical meaning • perceptual/cognitive processes • personal/cultural associations • Raise awareness of applications • music theoretical, analytical, and practical • Prepare for the SE "Cognition of Musical Structure"

7. Tentative semester plan (1)

8. Tentative semester plan (2)

9. Preparation for lectures Read the literature in advance! Making up for lost time Students at the first lecture on 8.3.06 preferred to extend each lecture by 15 minutes (i.e. 17:30-19:15) than to schedule two additional lectures.

10. Central literature sources • Houtsma et al.(1987). Auditory demonstrations on compact disc. • Articles in Semester Plan above Both are in folder „Psychoacoustics“ • Handapparat, reading room, musicology To copy articles: • take folder to secretary‘s office

11. Auxiliary literature sources • Bregman (1994). Auditory scene analysis • De la Motte-haber (2005). Musikpsychologie. • Deutsch (Ed., 1999). Psychology of music (2. ed.) • Hall (1997). Musikalische Akustik • Handel (1993). Listening • Harwood & Dowling (1995). Music cognition • Howard & James (1996). Acoustics and psychoacoustics. • McAdams & Bigand (Eds., 1993). Thinking in sound • Pierce (1985). Klang • Roederer (1993). Physikal. und psychoakust. Grundlagen der Musik • Rosen & Howell (1991). Signals & systems for speech & hearing • Terhardt (1998). Akustische Kommunikation • Zwicker (1982). Psychoakustik

12. The process of sound perceptionWhy do we experience a complex tone as one thing?

13. Philosophy of reality: Karl Popper‘s „three worlds“ (1) World 1 physical matter, energy World 2 experiential sensations, emotions World 3 abstract information, knowledge, culture Example: A visit to an art gallery • physical: walls, floor, canvas, paint, light waves, retina • experiential: colors, shapes, emotions (feeling, mood), sound or silence, smell, taste, touch • abstract: program, thoughts, content of conversation, historical knowledge, digital representations, theory of art Group exercise: repeat this analysis for a visit to a concert

14. Philosophy of reality: Karl Popper‘s „three worlds“ (2) Aim: clarity of terminology and thinking Example: A visit to concert • physical: walls, floor, violins, human bodies, sound waves, frequencies, amplitudes, spectra • experiential: what it sounds like, melodic shape, tension-relaxation, sense of time, speed, emotion (mood, feeling) • abstract: music notation, program, thoughts, historical knowledge, digital representations Especially relevant for this course: • physical: freq. amplitude spectrum duration • experiential: pitch loudness timbre perc. duration • abstract: note dynamic instrument note value

15. Lecture 2, 15.3.06 Intro to psychoacoustics • Sound examples Frequency perception • object perception & survival • freq. analysis, physiol., masking, CBW, loudness Literature: • Houtsma, A. J. M. et al. ((1987) Auditory demonstrations. New York: Acoustical Society of America. • Howard, D. M., & Angus, J. (1996). Acoustics and psychoacoustics. Oxford: Focal. Chapter 2 (pp. 65-91): “Introduction to hearing”. • Rasch, R. A., & Plomp, R. (1999). The perception of musical tones. In D. Deutsch (Ed.), Psychology of music (2nd ed., pp. 89-111). • Terhardt, E. (1988). Psychophysikalische Grundlagen der Beurteilung musikalischer Klänge. In J. Meyer (Hg.), Qualitätsaspekte bei Musikinstrumenten (S.1-15) Celle: Moeck.

16. Psychophysics: Worlds 1 and 2 Each experiential parameter depends on each physical parameter! Sound examples: ASA-CD • Pitch depends on spectrum (missing fundamental) (Track 37) • Timbre depends on temporal envelope: backward piano (Track 56) • Loudness does not double when intensity doubles (Track 9) More examples: • Pitch depends on intensity (Tracks 27-28) • Pitch salience depends on tone duration (Track 29) • Loudness depends on frequency (Tracks 17-18) • Loudness depends on spectrum (Track 7)

17. Frequency perception: Intro …as opposed to pitch perception • object perception and survival • frequency analysis • physiology • masking • critical band • loudness

18. Survival value of frequency perception • Darwin‘s theory of evolution • individual differences (mutation) • environment: danger; limited resources • survival = successful reproduction • Relevance for hearing and music • aim: survival by identifying and describing objects • input to ear: superposition of direct and reflected sound • unaffected: frequency randomized: phase • frequency is reliable phase is unreliable • sensitivity to frequency insensitivity to phase

19. Musical implications • Timbre (identifies sound sources) • strongly dependent on spectrum (esp. frequencies) • not directly dependent on waveform (phase) • Music notation and theory • primary: pitch, time • secondary: loudness, timbre • irrelevant: phase

20. Aural frequency analysis • Aim: identify environmental objects (sound sources) • Approach: monitor frequency-time patterns (contours) • Method: frequency analysis (separate frequencies)

21. Physiology of frequency analysis Basilar membrane changes along length • heavy, floppy end: sensitive to low frequencies • light, tight end: sensitive to high frequencies Each hair cell on basilar membrane: • responds to limited range of frequencies • is an „auditory filter“ • filter bandwidth = critical bandwidth

22. Cut-off frequency of a filter Arbitrary cut-off point: 3 dB down from maximum

23. Bandpass filter bandwidth A (dB) f (Hz) Center frequency

24. 1st harmonic 2nd harmonic signal 3rd harmonic bank of bandpass filters Frequency analysis by a filter bank Harmoniccomplextone

25. Critical bandwidth Auditory filters have no sharp cut-off => exact value of critical bandwidth is arbitrary depending on experimental method… • above about 500 Hz: 2...3 semitones • below about 500 Hz: 60...100 Hz (e.g. 80-160 Hz = 1 oct.!) Implications for tonal music If aim is… Separately audible voices in harmony and counterpoint Then need… Separately audible partials in sonorities Physiology: Excite different hair cells with different partials Result: Closer spacing of higher tones in chords

26. Critical bandwidth: Bark vs ERB Bark: Eberhard Zwicker et al. (München); ERB: Brian Moore et al. (Cambridge)

27. Auditory masking • „drowning out“ • everyday example: piano accompanist • simple example: two pure tones • masked threshold of a pure tone (Mithörschwelle) • number of audible partials of a complex tone

28. Auditory threshold

29. Masked threshold of a complex tone

30. 50 Hz 100 Hz 150 Hz 200 Hz Physical bandwidth Loudness Depends on: • number of excited hair cells (hence bandwidth of sound) • excitation of each cell (energy in each auditory filter) Repeat sound demonstration (ASA Track 7) Critical band SPL (dB) Frequency (Hz) 1000 Hz

31. Loudess of a steady-state complex sound after Stevens and Zwicker • within critical bands: • add energy (physical) • across critical bands: • add loudness (experiential)

32. Revision until Easter • Read the literature • Reread the lecture notes • Ask questions (e.g. email)

33. Lecture 3, 26.4.06 Pitch of complex tones • psychoacoustics (explained in lecture) • neuroscience (read Laden and Zatorre) Literature Parncutt, R. (1989). Harmony: A psychoacoustical approach. Berlin: Springer. (Chapter 2, Psychoacoustics). Laden, B. (1994). A parallel learning model of musical pitch perception. Journal of New Music Research, 23, 133-144. Zatorre, R. J. (1988). Pitch perception of complex tones and human temporal-lobe function. Journal of the Acoustical Society of America, 84, 566-572 Handout: Parncutt, R. (2005). Perception of musical patterns: Ambiguity, emotion, culture. Nova Acta Leopoldina NF 92 (341), 33-47

34. Pitch: Introduction • Abbreviations • PT: pure tone CT: complex tone HCT: harmonic CT • SP: spectral pitch VP: virtual pitch • Pitch perception according to Terhardt • SP: (analytic) pitch of an audible partial • VP: (holistic) pitch of a complex tone • Examples • most consciously noticed pitches are VPs • pitch at missing fundamental of HCT is a VP (e.g. telephone) • pitch of a heard-out harmonic is a SP • strike tone of church bell is VP; as sound dies, hear SPs

35. Harmonic series • To typical western ears, harmonics no. 7 and 11 • sound noticeably out of tune: • 7 is 1/3 semitone flatter than a m7 above 4 • 11 is about midway between P4 and TT above 8 • The harmonics are: • equally spaced on a linear frequency scale (e.g in Hz) • unequally spaced on a log frequency scale (e.g. in semitones)

36. Pitch at the missing fundamentalASA track 37 • Conclusions: • Pitch does not necessarily correspond to a partial • Pitch is multiple/ambiguous • VP at missing fundamental • SP at lowest partial 1 2 5 4 3

37. Sound demo: Masking SP and VP ASA-CD tracks 40 41 42

38. Sound demo: Masking SP and VPConclusion Westminster chimes example demonstrates that pitch at missing fundamental is „virtual“, because: • when PT masked by low-pass noise, • missing fundamentals is audible inside the noise • If it were physical it would be masked! • when HCT masked by high-pass noise, • missing fundamental is inaudible outside the noise • If it were physical it would be audible

39. Sound demo: „Shift of VP“ASA-CD Track 39 Conclusion: VP corresponds to: • best-fit subharmonic (or approx. fundamental) of all partials • NOT to difference in frequencies

40. Sound demo: VP with random harmonicsACA-CD tracks 43 44 45 HCTs of 3 random successive harmonics • harmonic numbers 2 to 6 (3 possibilities: 234, 345, 456) • harmonic numbers 5 to 9 • harmonic numbers 8 to 12 Conclusion: • salience of VP depends on effective harmonic number of SPs above it • lower harmonic numbers  more salient VP

41. Sound demo: Strike note of a chimeASA-CD Track 46 47 1. hearing out partials • pure reference tone, then complex test tone • Can you hear the PT inside the CT? • Procedure encourages analytic listening 2. matching a virtual pitch • reverse order: first complex test tone, then pure reference tone • Do the two tones have the same pitch? • Procedure encourages holistic listening Conclusions • partials are audible (as SPs) • „the pitch“ (VP) is ambiguous

42. Experimental determination of pitch Question: • Pitch is an experience. How can it be measured? Answer: • Compare pitch of two successive sounds • Assume pitch of one sound is known • If two sounds have same pitch, pitch of second sound is known The pitch of a pure tone is assumed: • to be unambiguous • to correspond to its frequency (provided SPL constant) Standard experimental method: • Test sound, pause, reference tone (each about 200-400 ms) • Listener adjusts frequency of reference until same pitch • A pitch „exists“ when intra- and inter-listener agreement

43. Pitch properties of complex tones A CT generally evokes several pitches. • If only one is perceived at a time, the pitch is ambiguous. • If more than one can be perceived at a time, the pitch is multiple. The pitches of a CT vary in salience, i.e. either: • the probability of noticing the pitch, or • the subjective importance of the pitch

44. Perception of complex tones Stage 1: Auditory spectral analysis (Ohm, 1843; Helmholtz, 1863) E.g. A HCT in speech or music typically has 10 + 5 audible harmonics. Stage 2: Holistic perception of CTs (Stumpf, 1883; Terhardt, 1976) A HCT is normally experienced as one thing: a complex tone sensation with pitch (VP), timbre, and loudness. But when partials are heard out, the CT is experienced as many things: pure tone sensations, each with pitch (SP), timbre, and salience.

45. Examples of physical spectra (“YL”) and experiential spectra (“salience”) 1. pure tone (PT on C4)2. harmonic complex tone (HCT on C4)3. octave-complex tone (OCT on C)“Pitch category”: 48 = C4, 60 = C5 etc.(Parncutt, 1989)

46. Terhardt‘s model of pitch perception: Input-output • Input: physical spectrum of a steady-state sound (frequency and amplitude of each partial) • Output: „experiential spectrum“ (pitch and salience of each tone sensation) • Aim: predict experiential spectrum from physical spectrum

47. Terhardt‘s model of pitch perception: Detail 1. masking SPs and their saliences • Nearby partials mask each other more strongly • Inner partials are masked more than outer partials 2. recognition of harmonic pitch patterns VPs and saliences • Salience depends on • fit between harmonic template and spectrum • number and accuracy of matches • salience of matching SPs • more salient SPs  more salient VP • harmonic number of matching SPs • lower harmonic nos.  higher VP-salience 3. combination of SPs and VPs all pitches and saliences • experiential spectrum contains both • relative weighting depends on analyic/holistic perception

48. Hearing out harmonics (1)Terhardt CD track 17 • HCT, 200 Hz, 10 harmonics • harmonic numbers 4,3,4,5,6: + 3 dB Conclusion: SPs exist independently of VP

49. Further sound examples See CD in back of Terhardt (1998)

50. Hearing out harmonics (2) Terhardt CD track 18 • HCT, 200 Hz, 10 harmonics • Pure tone 600 Hz • Harmonic not heard out • Same HCT twice, once with missing harmonic • Attention attracted to „replaced“ harmonic Conclusions • Attention is attracted to changes and differences • Again: SPs exist independently of VP