1 / 17

Agust ín Gravano 1,2 Julia Hirschberg 1

Backchannel-Inviting Cues in Task-Oriented Dialogue. Agust ín Gravano 1,2 Julia Hirschberg 1. Columbia University, New York, USA (2) Universidad de Buenos Aires, Argentina. Introduction. Interactive Voice Response Systems. Quickly spreading. Mostly simple functionality.

laith-lopez
Download Presentation

Agust ín Gravano 1,2 Julia Hirschberg 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Backchannel-Inviting Cuesin Task-Oriented Dialogue Agustín Gravano1,2 Julia Hirschberg1 • Columbia University, New York, USA (2) Universidad de Buenos Aires, Argentina

  2. Introduction Interactive Voice Response Systems • Quickly spreading. Mostly simple functionality. • “Uncomfortable”, “awkward”. • ASR+TTS account for most IVR problems. • As ASR and TTS improve, other problems revealed. • Coordination of system-user exchanges. • Backchannels. Agustín Gravano Interspeech 2009

  3. Introduction Backchannels • Short expressions uttered by listeners to: • Convey that they are paying attention. • Encourage the speaker to continue. • Examples: okay, uh-huh, mm-hm, alright. • Very frequent in task-oriented dialogue. • Thus, modeling human usage of BC should lead to an improved system-user coordination. Agustín Gravano Interspeech 2009

  4. Introduction Goal • Learn when backchannels are likely to occur. • Find “backchannel-inviting” cues. • Cues displayed by the speaker “inviting” the listener to produce a backchannel response. • This could improve the coordination of IVRs: • Speech understanding: Detect points in the user’s turn where a backchannel would be welcome. • Speech generation: Display cues inviting the user to produce a backchannel. Agustín Gravano Interspeech 2009

  5. Talk Outline • Previous work • Material • Method • Results • Conclusions Agustín Gravano Interspeech 2009

  6. Backchannel-Inviting Cues Previous Work • Duncan 1972, 1973, 1974, inter alia. • Hypothesized six turn-yielding cues in face-to-face dialogue. • Several studies continued this line of research, but always excluded backchannels. • Ward & Tsukahara 2000. • Region of low pitch lasting 110ms or more. • Cathcart et al. 2003. • Language model based on pause duration and part-of-speech tags to predict the location of BC. Agustín Gravano Interspeech 2009

  7. Material Columbia Games Corpus • 12 task-oriented spontaneous dialogues. • Standard American English. • 13 subjects: 6 female, 7 male. • Series of collaborative computer games. • No eye contact. No speech restrictions. • 9 hours of dialogue. • Manual orthographic transcription, alignment. • Manual prosodic annotations (ToBI). Agustín Gravano Interspeech 2009

  8. Material Columbia Games Corpus Player 1: Describer Player 2: Follower Agustín Gravano Interspeech 2009

  9. Backchannel-Inviting Cues • Cues displayed by the speaker “inviting” the listener to produce a backchannel response. Agustín Gravano Interspeech 2009

  10. Hold Backchannel IPU4 IPU1 IPU2 Speaker A: IPU3 Speaker B: Backchannel-Inviting Cues Method • IPU (Inter Pausal Unit): Maximal sequence of words from the same speaker surrounded by silence ≥ 50ms. • 3 trained annotators identified Backchannels using a labeling scheme described in [Gravano et al. 2007]. • To find BC-inviting cues, we compare: • IPUs preceding Holds, • IPUs preceding Backchannels. Agustín Gravano Interspeech 2009

  11. Backchannel-Inviting Cues Individual Cues • Final rising intonation: 81% of IPUs before BC end in H-H% or L-H%. • Higher pitch level. • Higher intensity level. • Lower NHR (voice quality). • Longer IPU duration (seconds, #words). • Final POS bigram: 72% of IPUs before BC end in DT NN, JJ NN, or NN NN. } • entire IPU • final 1.0 sec • final 0.5 sec Agustín Gravano Interspeech 2009

  12. Backchannel-Inviting Cues Defining Presence of a Cue • 2 representative features for each cue: • Define presence/absence based on whether the value is closer to the mean before BC or H. Agustín Gravano Interspeech 2009

  13. Top Frequencies of Complex Cues digit == cue present dot == cue absent BC-inviting cues: 1: Final intonation 2: Intensity level 3: Pitch level 4: IPU duration 5: Voice quality 6: Final POS bigram Agustín Gravano Interspeech 2009

  14. Backchannel-Inviting Cues Combined Cues Percentage of IPUs followed by a BC r2=0.993 Number of cues conjointly displayed Agustín Gravano Interspeech 2009

  15. Backchannel-Inviting Cues IVR Systems • After each IPU from the user: if estimated likelihood > thresholdthen produce a backchannel • To elicit a backchannel from the user, if desired: Include as many cues as possible in the system’s final IPU. Agustín Gravano Interspeech 2009

  16. Summary • Study of backchannel-inviting cues. • Objective, automatically computable. • Combined cues. • Improve turn-taking decisions of IVR systems. • Results drawn from task-oriented dialogues. • Not necessarily generalizable. • Suitable for most IVR domains. • SIGdial 2009: Study of turn-yielding cues. Agustín Gravano Interspeech 2009

  17. Special thanks to… • My advisor, Julia Hirschberg • Thesis Committee Members • Maxine Eskenazi, Kathy McKeown, Becky Passonneau, Amanda Stent. • Speech Lab at Columbia University • Stefan Benus, Fadi Biadsy, Sasha Caskey, Bob Coyne, Frank Enos, Martin Jansche, Jackson Liscombe, Sameer Maskey, Andrew Rosenberg. • Collaborators • Gregory Ward and Elisa Sneed German (Northwestern U); Ani Nenkova (UPenn); Héctor Chávez, David Elson, Michel Galley, Enrique Henestroza, Hanae Koiso, Shira Mitchell, Michael Mulley, Kristen Parton, Ilia Vovsha, Lauren Wilcox. Agustín Gravano Interspeech 2009

More Related