1 / 19

Tips for Writing Gooder Grammars

Tips for Writing Gooder Grammars. Judi Halperin Speech Engineer. Overview. Why does the grammar development process matter?. Grammar Development Lifecycle Best practices What not to do Why?. Avoid the pitfalls!!. A successful ASR application requires…. A good design….

damaris
Download Presentation

Tips for Writing Gooder Grammars

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tips for Writing Gooder Grammars Judi Halperin Speech Engineer

  2. Overview • Why does the grammar development process matter? • Grammar Development Lifecycle • Best practices • What not to do • Why? • Avoid the pitfalls!!

  3. A successful ASR application requires… A good design… Application Development …efficient coding… Design Infrastructure Grammars …and effectively constrained grammars!!! …supporting infrastructure…

  4. BOTTOM LINE… If the system does not recognize what callers say, they will not be able to complete their task in the IVR!! • As a result… • Caller/Customer satisfaction will be • negatively impacted • Containment rate heads south and • ROI goes with it!

  5. Grammar Development Lifecycle: Steps for Success….

  6. Design and Plan • Should be started in parallel with VUI design • Determine feasibility • Can the grammar be developed? • Can the production hardware support it in real time? • Start large efforts early • Set expectations early • If caller identification requires alphanumeric recognition, make sure your customer understands the recognition capabilities early on! • Host modification takes time • Dynamic and other complex grammars require more testing • New concepts and/or languages may require external resources • A prescription names grammar will typically require both a large grammar AND corresponding lexicon development. • Allow time for collaboration with VUI designer and developer

  7. Creating Your Menu Grammar • Don’t include everything a caller might say, but rather what a cooperative caller would say • Leave the addition of possible synonyms to tuning • The more in-grammar items, the more chance of false accepts • Consider logical synonyms based on the prompt • Callers have a tendency to parrot back the prompt: • PROMPT: “..or say ‘main menu’ to start over.” • Callers would be likely to say “main menu”, “start over” or “main menu to start over” • Include the actual spelling of the word or phrase in the grammar to take advantage of the pronunciations in the system dictionary • “that’s” not “thats” • “t v” not “teevee” • “don’t” not “dont” • “f a q” not “f a queue” (it’s just silly)

  8. General Grammar Development Tips • Minimize the use of pre-and-post-fillers out of the box • PROMPT: “What’s your account number?” • Callers are likely to say “it’s account number...” but not “I think it might be...” • Leave the addition of these items to tuning once sufficient data proves their relevance • Choose your built-ins wisely • Control your content when appropriate • Don’t use alphanumeric built-in grammar for spelling-only input • Unnecessary overhead can have a latency effect • Overgeneration leads to decrease in recognition accuracy (and possible latency) • Beware: Some built-ins overgenerate

  9. Task: Allow spelling using military alphabet <rule id="root"> <item><ruleref uri="letter"><tag>ASCII=letter.LETTER</tag></item> <item repeat="0-1">as in</item> <item> <ruleref uri="militaryAlpha"><tag>MIL = militaryAlpha.MIL</tag> </item> </rule> <rule id="letter"> <one-of> <item>a <tag>LETTER="a"</tag> </item> <item>b <tag>LETTER="b"</tag> </item> … .. <item>y <tag>LETTER="y"</tag> </item> <item>z <tag>LETTER="z"</tag> </item> </one-of> </rule> <rule id="militaryAlpha"> <one-of> <item>alpha <tag>MIL="a"</tag> </item> <item>bravo <tag>MIL="b"</tag> </item> … . <item>yankee <tag>MIL="y"</tag> </item> <item>zulu <tag>MIL="z"</tag> </item> </one-of> </rule>

  10. Task: Allow spelling using military alphabet <rule id="root"> <item><ruleref uri="letter"><tag>ASCII=letter.LETTER</tag></item> <item repeat="0-1">as in</item> <item> <ruleref uri="militaryAlpha"><tag>MIL = militaryAlpha.MIL</tag> </item> </rule> This grammar generates “a as in alpha”… <rule id="letter"> <one-of> <item>a <tag>LETTER="a"</tag> </item> <item>b <tag>LETTER="b"</tag> </item> … .. <item>y <tag>LETTER="y"</tag> </item> <item>z <tag>LETTER="z"</tag> </item> </one-of> </rule> <rule id="militaryAlpha"> <one-of> <item>alpha <tag>MIL="a"</tag> </item> <item>bravo <tag>MIL="b"</tag> </item> … . <item>yankee <tag>MIL="y"</tag> </item> <item>zulu <tag>MIL="z"</tag> </item> </one-of> </rule> …but also “a as in zulu”!!

  11. Consider all methods of constraint • Avoid overgeneration when possible • Don’t “sort it out” on the application side if you don’t have to • But watch out for introduced latency • Don’t allow account numbers that are not valid • Make sure your pre-and-post-fillers make sense at every invocation: • “I want my account balance” but not “I want my help” • Make sure your in-grammar items make sense at every invocation: • “a as in alpha” but not “a as in zulu” • Constrain birth dates to dates before today’s date and make assumptions based on caller demographics as to the earliest allowable date

  12. Remove the overgeneration… <rule id="letter"> <one-of> <item>a as in alpha <tag>LETTER="a"</tag> </item> <item>b as in bravo <tag>LETTER="b"</tag> </item> . . <item>y as in yankee <tag>LETTER="y"</tag> </item> <item>z as in zulu <tag>LETTER="z"</tag> </item> </one-of> </rule> <rule id="root"> <item> <ruleref uri="letter"> <tag>ASCII=letter.LETTER</tag> </item> </rule> This grammar generates “a as in alpha”… …but NOT “a as in zulu”!!

  13. You have the power! • Some platforms allow scripting within the grammar itself • Grammar-internal scripting is useful in many situations • Further constrain dates, times, currency, etc • Implement a skip list or deweight a previous response that was negatively confirmed • Apply a checksum to disallow invalid digit string input • Calculate additional information based on the caller’s response • Day of the week given the date • Determine the time given colloquial response, ex: “in an hour”

  14. <rule id="root"> <item><ruleref uri="letter"> <tag>tmpL=letter.LETTER</tag> </item> <item repeat="0-1">as in</item> <item><ruleref uri="militaryAlpha"> <tag>tmpM=militaryAlpha.MIL</tag> </item> <tag> <![CDATA[ if (tmpL != tmpM) { SWI_DISALLOW = 1; } else { LETTER = tmpL; } ]]> <tag> </rule> <rule id="letter"> <one-of> <item>a <tag>LETTER="a"</tag></item> <item>b <tag>LETTER=“b"</tag></item> . . <item>y <tag>LETTER=“y"</tag></item> <item>z <tag>LETTER="z"</tag></item> </one-of> </rule> <rule id="militaryAlpha"> <one-of> <item>alpha <tag>MIL="a"</tag></item> <item>bravo <tag>MIL="b"</tag></item> . <item>yankee <tag>MIL="y"</tag></item> <item>zulu <tag>MIL="z"</tag></item> </one-of> </rule> Constrain using javascripting…

  15. “Simplify” Continuous String Recognition • Constrain digits, alphanumeric and spelling grammars as tightly as possible • Load valid strings dynamically (especially for alphanumeric and spelling grammars) • If using a static grammar, apply appropriate patterns • Positively weight patterns that occur more often (if appropriate) • Apply logic to the task • If collecting the 1st two letters of a last name: • Use census data or other appropriate information to help constrain to the top 90+% of last names in the US • Don’t allow letter combinations that are not reasonably expected to occur given the task, or even producible by a native speaker, ex: “QB” or “CC”

  16. Test Before Throwing it Over the Fence • Open the grammar in a web browser or XML editor to validate • Use platform tools to test the grammar • Try both valid and invalid entries • Verify correctness of the slot values • Validate the mappings from raw text to slot value • Validate any calculated values returned in slots • Generate a large corpus, extract unique entries and validate • Load the grammar into a test application and give it a spin • Don’t worry about unit testing OOGs – leave that to tuning

  17. Pilot Test and Tune • Collect a sufficiently large data set over various times, dates and demographics • Not just high traffic or low traffic periods • Not during a one-time promotion • Not only when callers might have a higher tendency towards otherwise infrequently used functionality • Tax season, beginning of the month, etc. • Transcribe what the caller said • Analyze performance and make recommendations for improvements • Tuning should be performed periodically as appropriate

  18. Avoid Grammar Development Pitfalls!! Include actual spellings… Use grammar-internal scripting if appropriate… Validate your grammar…. ‘Simplify’ continuous string recognition… • Start large efforts early… Avoid overgeneration… Minimize out of-the-box pre-and-post-fillers Generate custom pronunciations if needed… Plan it out… Constrain as tightly as possible… Include only what a cooperative caller would say… Test, test, test…. Pay attention to the prompt… Use built-ins as appropriate… Pilot test and tune….

  19. Judi Halperin jhalperin@avaya.com 19 © 2007 Avaya Inc. All rights reserved. Avaya – Proprietary & Confidential. Under NDA

More Related