Composing & Performing Interactive Music

1. Composing & PerformingInteractive Music October 18-23, 2004 Faculty of Music McGill University Bruce Pennycook, DMA Dept. of Composition, School of Music Dept. of Radio Television Film, College of Communication University of Texas at Austin

2. Introduction pennycook/mcgill

3. Schedule & Topics Monday 3-6 Principals and Aesthetics of Interactive Music Tuesday 9-12 Impact of interactive music on performance practice and on compositional methods Wed 9-12 Systems Design (composer/performer perspective) Thur 9-12 New directions and possibilities Friday 9-12 Music and Audio Visualization - interactive, real-time video-music pennycook/mcgill

4. Objectives of the Seminars • Present an overview of the history of interactive composition and performance • Provide in depth resources and materials for graduate level study and research • Propose new areas of creativity and research from emerging aesthetics, styles and technologies including visualization systems pennycook/mcgill

5. Seminar Format • Presentations by B. Pennycook • Examples and demonstrations • Daily contributions by students • Short paper (due October 31, 2004) pennycook/mcgill

6. Resources • Read - Rowe, Winkler • CMJ articles • Recordings • Max, Max/MSP; SuperCollider 2, 3 pennycook/mcgill

7. Seminar #1Principals and Aesthetics of Interactive Music (who will turn the knobs when I die) Monday, October 18, 2004 3:00 pm - 6:00 pm Room LSR#3 pennycook/mcgill

8. Principals and Aesthetics of Interactive Music • What is “interactivity” ? (he says/she says) • Are there pre-computer examples ? • What about “live electronic music” ? • Why are these early pieces different from our current thinking on this topic? That is, what role does the computer play in this? • What separates computational control from machine control? pennycook/mcgill

9. Principals and Aesthetics of Interactive Music • What is the essential attraction of interactive music? Why do composers (and for that matter, performers) want to create/play it? • How has the software community influenced music making and vice-versa? • What connection exists (or should exist) with other forms of interactive art? pennycook/mcgill

10. Principals and Aesthetics of Interactive Music • Tape + instruments/voices • Instrumental karaoke (music minus “n”) • Free the player from the tyranny of the clock (tape, cd) • Like asking an actor to perform to video or film • Eliminates the normal elasticity of gestures especially on the large scale (local elasticity is possible - like Chopin LH/RH); rubato has to be “built in” to the score • No error tolerance - clock keeps ticking no matter what pennycook/mcgill

11. Principals and Aesthetics of Interactive Music • BUT - (Music - “n”) works! • Many very important pieces (Berio, Davidovsky, Risset, Lanza, Morrill, Parmagiani, ) that remain performed • Low-stress rehearsal environment • Easily replicated performance after performance • Players can memorize aural events very precisely • Players can emulate “interaction” convincingly • Sonic domain can be managed easily • Low gear, setup, help environment pennycook/mcgill

12. Principals and Aesthetics of Interactive Music • Principals of interactive music • Establish a dialogue between man and machine • Permit modes and levels of adaptation • Seek new performance results - from slightly interpretive to full improvisation • Seek new or at least dynamic compositional results • Explore relationship between man and machine to some degree • Explore machine autonomy to some extent pennycook/mcgill

13. Principals and Aesthetics of Interactive Music • Temporal Control • What are the key “interpretive issues”? • What degree of flexibility is required? • What are the macro/micro level of temporal controls that would have direct and audible impact on the listener? • Why not just fake it? • How can this be managed with minimal impact on the player or ensemble? pennycook/mcgill

14. Principals and Aesthetics of Interactive Music • Computer assisted composition. • Grab/alter/play metaphor is very seductive given the inherent compositional cohesion that ensues. But despite some clever efforts (Risset,Rowe, Pennycook, Pinkston, Hamel, Chafe, others) - these have not been embraced into the “repertoire”) • Is this a MIDI thing? Is MIDI dead? • If yes, why? What problems exist with this method of work that prohibits it from finding a solid niche in the overall ea or chamber music genre? pennycook/mcgill

15. Principals and Aesthetics of Interactive Music • Guided Improvisation • http://smc04.ircam.fr/ProgWorkshop.html • Workshop on computer improvisation, Oct 20 IRCAM • What is “guided improvisation” • In what sense does a player actually “improvise” • In what sense can a computer “improvise” • One-way, two-way, n-way improvisation • Stylistic norms and boundaries help manage conditional environment pennycook/mcgill

16. Principals and Aesthetics of Interactive Music • Guided Improvisation • Essential criterion for improvisation is the ability to “listen” • Application of analytic/generative processes to the incoming musical information • Segmentation, feature extraction and pattern matching • Modular compositional automaton • Other factors such as parallel process management, multi-computer audio, midi and data interfaces pennycook/mcgill

17. Principals and Aesthetics of Interactive Music • Audio Processing Compositions • Modes of operation • “outboard rack” approach • New audio from previous audio (lag issues) • Stored files • Stored audio altered on the fly as per incoming data • Real-time pitch tracking, segmenting, pattern matching (huge topic to be returned to later in the week) • Computational demands much higher than MIDI hence must be worth it pennycook/mcgill

18. Principals and Aesthetics of Interactive Music • General Properties - Summary • Modes of performance • Active - triggers, footswitch, etc.; initiated by operator and/or performer(s) • Passive - system detects appropriate flags from processes such as beat detection, pitch detection, silence/pause detection, motion capture, time-code (clock) etc. • Granularity • Section, event, note, clock time, • Governed by compositional style and process compexity pennycook/mcgill

19. Principals and Aesthetics of Interactive Music • General Properties - Summary • Input and Sources • MIDI - seems passé now but why? • Audio - transducer properties, analytical, processing and generative algorithms • Motion/Image - use of gesture may be critical to effective interaction; secondary channel? • Output • MIDI? - this is really dead • Audio - channels, mixing, loudspeaker management • Video/Image - supporting pennycook/mcgill

20. Principals and Aesthetics of Interactive Music • General Properties - Summary • Longevity (who will turn the knobs) • Very few pieces are playable without the composer or trained operator present • Players largely disenfranchised due to tech-gap • No obvious solution to hardware/software obsolescence • Many components defy notation or even adequate description • Teachers will never (?) undertake this repertoire thus the cycle of master-apprentice is essentially broken beyond repair • Many acoustic composers consider this little more than gear-tinkering • No systematic reviewing process • Vast arena of techno-pop has totally overshadowed the genre pennycook/mcgill

21. Seminar #1Principals and Aesthetics of Interactive Music END Monday, October 18, 2004 3:00 pm - 6:00 pm pennycook/mcgill

22. Seminar #2Composition and Performance of Interactive Music (the chamber music tradition) Tuesday, October 19, 2004 9:00 am - 12:00 pm Room LSR #1 pennycook/mcgill

23. Interactive Compositions • Early adoptors (MIDI): • Roger Dannenburg, Chris Chafe, Joel Chadabe, Dexter Morrill, Keith Hamel, Russell Pinkston, Jean-Claude Risset, Morton Subotnick…. • Early adopters (Audio/DSP) • Cort Lippe, Zack Settel, Tod Machover, Russell Pinkston, …. • Reference - Joel Chadabe “….” pennycook/mcgill

24. Interactive Compositions • Pennycook “PRAESCIO series” • First public performance - Buffalo, April 1987 - Praescio I pennycook/mcgill

25. Praescio - I • Soprano saxophone • Original version constructed with Dannenburg software “cmidi” (?) on PC-AT/286. • Setup included: • PC/at with MIDI IN, IVL Pitchrider • Sax data processed with delays, harmonizations, etc. • PC/at with midi version of score-11 developed by BP and CS grads at Queen’s University, called M-SCORE • Files were “hand triggered” on a section-by-section basis • Extreme underflow occurred during performance causing bursts and cascades pennycook/mcgill

26. Selected Compositions Praescio-I Rec. 1991, McGill pennycook/mcgill

27. Praescio - I • Versions II, III • Reconstructed using MIDI-LIVE software developed at McGill (Pennycook, Fujinaga, Hillborn, Quesnel) • Current version - Max • (more on this tomorrow) pennycook/mcgill

28. Tornado (McGill EMS) Praescio-II: amnesia pennycook/mcgill

29. Praescio - II: amnesia • Commissioned by Geoffrey Wright for 25th anniversary of Peabody Conservatory EMS • With Morton Subotnick “Jacob’s Room” • Instrumentation, System • Soprano, flute, vln, vcello, dx7, system • First version of MIDI-LIVE software (Low-latency MIDI composition system for real-time performance) • Soprano and flute were close mic’d and provided pitch data to software via 2 IVL Pitchriders • Stored sequences were triggered (by operator) • MIDI Channel management was the crucial component pennycook/mcgill

30. MIDI-LIVE 0.8 • Designed to permit fluent interchange of live data with stored data • Premise was that MIDI files could be played like “pieces of tape” • Transformations included: • Assign out channel(s) • Assign tempo, velocity (volume) on a per-track basis • Specifiy harmonization, transposition • Gather inmcoming note-ons, strip temporal info, resend in various ways • Any number of tracks could be active at a time all under their own local metronome • Scripting language + playback interface for live performance that showed channel activity • Read/process standard midi files produced from sequencer, M-SCORE (score11/midi) OR recorded and stored internally pennycook/mcgill

31. PRAESCIO-III The desert speaks Vivien Spiteri, Harpsichord 1989 pennycook/mcgill

32. Praescio-III: the desert speaks • MIDI-LIVE 1.0 - program, much more stable, more processing capabilities, better user interface for scripting • Praescio-III: harpsichord and midilive • Challenge as the harpsichord - first interface was developed with Eric Johnstone at McGill using “organ retrofit” midi package with a complete set of switches for upper manual • Small control unit attached to harpsichord to permit the player to reset, advance, etc. and manage fswitch and vol pedal (critical for the performance) pennycook/mcgill

33. Praescio-III: the desert speaks • Version 2 of the interface was built for concert in Europe - original interface stolen out of a van (very high $ return for sure!) • New version entirely optical - individual LED/Receptor pairs per note on upper manual • Worked OK but susceptible to sudden lighting changes! • Never truly debugged, hard to regulate (but better than the mechanical one) pennycook/mcgill

34. Praescio-III: the desert speaks • 3 movement format • I - colorize • II - record/strip/process/play • III - triggered sequences, colorize pennycook/mcgill

35. PRAESCIO-IV Jean-Guy Boisvert, Clarinet 1991 pennycook/mcgill

36. Praescio-IV • Commission for the 1991 International Clarinet Conference by Jean-Guy Boisvert • Challenge was to provide clarinetist with maximum freedom of control over temporal components • Non-improvisatory • Cheap MIDI tone generators to facilitate travel (but that may have been a bad idea) pennycook/mcgill

37. Praescio-IV • Unique “harness” for the clarinet designed and built by BP and Eric Johnstone • Provided attachment (DIN) for: • Contact mic on reed to improve IVL tracking • 3 ultra-light keys placed by LH thumb and RH little-finger to permit cross-fingered sustain and trigger • Volume pedal on floor was unavoidable then, perhaps with gesture tracking this too could be eliminated • Images in CMJ • Performed successfully by many different players - learning curve very short using the device and score cues pennycook/mcgill

38. Praescio-V • Praescio-V - a kind of “joint performance piece” for Dexter Morrill and myself • 1990 “MIDI-LIVE” road tour in Europe/Eastern Europe • Trumpet, sax and small rack of midi tone generators + Yamaha DMP-11 midi controlled mixer for processing audio (software controlled) • No longer extant but lots of fun to play. • Note that Dexter Morrill made numerous compositions using MIDI-LIVE and even shipped a system around to performers. A version was made for the Yamaha laptop that supported MIDI (not the CX5). pennycook/mcgill

39. Praescio-VI • Praescio-VI commissioned by Christine Little, Toronto flautist • Several performances by different players - Montreal, Toronto, Ottawa, ICMC-San Jose, Mexico City… • Fairly stable; short learning curve • 4 innovations for this piece • Max version of MIDI LIVE under development • MIDI Time Clip (remote signaling device to be described tomorrow) • Use of audio-on-CD as part of the controlled environment, “more than MIDI” output • Digidesign Sample-Cell hence entirely internal to the Mac • But, some serious level issues, hard to control in real-time pennycook/mcgill

40. Praescio-VII (piano and them some..) alcides lanza, piano pennycook/mcgill

41. Praescio-VII • Praescio-VII (piano and then some…) commissioned by ACREQ for alcides lanza to perform • many performances by different players plus tours in Europe, South America by lanza • MIDI Time Clip crucial for both the conception of the piece and the performance - very difficult to perform without feedback from computer->player pennycook/mcgill

42. Praescio-VII • Most complex of the Praescio series • Midi data generated from Common Music/LISP programs written by BP --> SMF’s • Full max implementation of MIDI-LIVE 2.x (Stammen) • Several specialized Max objects written by Dale Stammen • MIDI in from triggers (no piano data) • Feedback to player with MIDI Time Clip, complete Time Clip software package (Pennycook/Stammen) pennycook/mcgill

43. Praescio-VII • Large array of “piano tone modules” for midi out • Dual CD under complex Max control to permit overlap • 8 audio outs and 8 loudspeakers with real-time placement using MIDI-controlled MIXER (simple unity gain device - no EQ) • Full integration of prerecorded audio tracks and prepared MIDI sequences • Temporal management of triggers only - no improvisation • More than 60 events pennycook/mcgill

44. Other mixed pieces • The Black Page Tropes (1995) • Drums, percussion, midi out + audio • One section of triggered improvisation using loops derived from Pyhrite external in Max • Long complex work primarily for the players - audio/MIDI more supportive and commentary • The Yonge Street Variations (1998) • Cello, MIDI (drum head), audio, DSP • Less notes, more processing and sound • Much greater reliance on stored audio files triggered by player • Based on very early work for viola and percussion (recorded) pennycook/mcgill

45. Summary • Much was learned from the development of all these works • End of the MIDI era (almost) • 8 audio outs and 8 loudspeakers with real-time placement using MIDI-controlled MIXER (simple unity gain device - no EQ) • Full integration of prerecorded audio tracks and prepared MIDI sequences • Temporal management of triggers only - no improvisation • More than 60 events pennycook/mcgill

46. Seminar #3Interactive System Design Wednesday, October 20, 2004 9:00 am - 12:00 pm Room 806 pennycook/mcgill

47. Design issues • What is the definition of Tod Machover’s interactive solo cello piece for Yo-Yo Ma, Begin again and again pennycook/mcgill

48. Design issues • 1 cellist, 6 technicians + 2 18-wheeler’s (1991 view) pennycook/mcgill

49. Risset - Duet for one piano Jaffe/Schloss - Wildlife Wessel - phrase recorder Lippe - Music for Clarinet and ISPW Dannenberg - CMU Midi Toolkit Rowe - Cypher Pennycook - MidiLive/Max, T-MAX, Listener Project (with Hillborn, Stammen, Quesnel…) Brief History pennycook/mcgill

50. Looking backwards • Development of interactive, live-perf systems • Max Software (version 2) 1990/91 • This program was written for 68k Mac. • PlaySMF (Dale Stammen - superb Midilive implementation for MAX) • Led to more ambitious implementations especially T-MAX, a version of Rowe’s Cypher running across a Mac IIfx and 4 Inmos T805 Transputers • Listener project - Stammen/Pennycook (see Rowe) pennycook/mcgill