1 / 74

What utility can do for NLG: the case of vague language

What utility can do for NLG: the case of vague language. Kees van Deemter University of Aberdeen Scotland, United Kingdom. Two big issues. Does NLG have proper foundations ? What is its mathematical core? Why is language vague? When does vague language have benefits over crisp language?.

cicero
Download Presentation

What utility can do for NLG: the case of vague language

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What utility can do for NLG:the case of vague language Kees van Deemter University of Aberdeen Scotland, United Kingdom Kees van Deemter, Dublin, Trinity College, May 2009

  2. Two big issues • Does NLG have proper foundations? • What is its mathematical core? • Why is language vague? • When does vague language have benefits over crisp language? Kees van Deemter, Dublin, Trinity College, May 2009

  3. NLG and utility • Why is language vague? • Why should NLG systems be vague? • Conclusions Kees van Deemter, Dublin, Trinity College, May 2009

  4. Why bother about “foundations” for NLG? Kees van Deemter, Dublin, Trinity College, May 2009

  5. Why bother about “foundations” for NLG? • Computing is based on • theory of formal languages/computability • Hoare logic/program correctness • Linguistics is based on • data & stats • the theory of formal languages • mathematical logic/set theory Kees van Deemter, Dublin, Trinity College, May 2009

  6. Why bother about “foundations” for NLG? • Computing is based on • theory of formal languages/computability • Hoare logic/program correctness • Linguistics is based on • data & stats • the theory of formal languages (syntax) • mathematical logic/set theory (semantics) Kees van Deemter, Dublin, Trinity College, May 2009

  7. Why bother about “foundations” for NLG? • Computing is based on • theory of formal languages/computability • Hoare logic/program correctness • Linguistics is based on • data & stats • the theory of formal languages (syntax) • mathematical logic/set theory (semantics) • how about pragmatics? • Not just what’s true, but what’s appropriate given the circ’s. Kees van Deemter, Dublin, Trinity College, May 2009

  8. An NLG program “translates” input to linguistic output • Essentially the problem of choosing the best Form for a given Content. E.g. • McDonald 1987 • Bateman 1997 on Systemic Grammar • What does this choice depend on? Kees van Deemter, Dublin, Trinity College, May 2009

  9. What could this choice depend on? • Roughly: which utterance is most “useful” • In other words, utility • A perspective seldom pursued. Exceptions: • Kibble 2003 (IWCS-5, referring expressions) • Klabunde 2009 (ENLG-12) • NLG programs become more expressive, so this perspective now becomes tempting Kees van Deemter, Dublin, Trinity College, May 2009

  10. Example: the hazards of road ice Kees van Deemter, Dublin, Trinity College, May 2009

  11. Kees van Deemter, Dublin, Trinity College, May 2009

  12. Example: Roadgritting (Turner et al. 2009, this conference) Compare • “Roads above 500m are icy” • “Roads in the Highlands are icy” Decision-theoretic perspective: 1. 100 false positives, 2 false negatives 2. 10 false positives, 10 false negatives Suppose each false positive has utility of -0.1 each false negative has utility of -2 Kees van Deemter, Dublin, Trinity College, May 2009

  13. Example: Roadgritting (Turner et al. 2009, this conference) Suppose false positive has utility of -0.1 false negative has utility of -2 Then 1: 100 false pos, 2 false neg = -14 2: 10 false pos, 10 false neg = -21 So summary 1 is preferred over summary 2. Kees van Deemter, Dublin, Trinity College, May 2009

  14. Communication in Game Theory (following Crawford & Sobel 1982) • Set of contents C • Set of forms F • Set of actions A • Speaker strategy S : CF • Hearer strategy H : FA • Two utility functionsUS : A[0,1] UH : A[0,1](not necessarily the same!) Kees van Deemter, Dublin, Trinity College, May 2009

  15. A special case: Vagueness • Two related questions: • Can Game Theory help us explain why language is so often vague? • Can Game Theory tell NLG systems when to use vagueness? Kees van Deemter, Dublin, Trinity College, May 2009

  16. Vagueness: An expression is vague iffit has borderline cases. • Example of a vague adjective: “poor” is vague because some people are borderline poor. • Saying this differently: different thesholds for “poor” may be used Kees van Deemter, Dublin, Trinity College, May 2009

  17. Example: Is John poor? £30.000 p/a Norm A: “John is poor” is True Norm B: “John is poor” is False poverty Norm A John £20.000 p/a poverty Norm B £10.000 p/a Kees van Deemter, Dublin, Trinity College, May 2009

  18. Other vague adjectives: ‘large’, ‘small’, ... • Vague nouns: ‘girl’, ‘giant’, ‘island’,... • Vague determiners: ‘many’, ‘few’, ... • Vague adverbs: ‘often’, ‘slowly’, ... Relevant for any NLG system with “continuous” input • weather forecasts (FOG, Sumtime-Mousam) • patient data (e.g. Babytalk) Kees van Deemter, Dublin, Trinity College, May 2009

  19. From Babytalk corpus “BREATHING – Today he managed 1½ hours off CPAP in about 0.3 litres nasal prong oxygen, and was put back onto CPAP after a desaturation with bradycardia. However, over the day his oxygen requirements generally have come down from 30% to 25%. Oxygen saturation is very variable. Usually the desaturations are down to the 60s or 70s; some are accompanied by bradycardia and mostly they resolve spontaneously, though a few times his saturation has dipped to the 50s with bradycardia and gentle stimulation was given. He has needed oral suction 3 or 4 times today, oral secretions are thick.” [BT-Nurse scenario 1] Kees van Deemter, Dublin, Trinity College, May 2009

  20. The question is: “When (if ever) is vague communication more useful than crisp communication?” • The question is not: “Can vague communication be of some use?” • Compare Rohit Parikh (2000) • Ann calls Bob to bring “the blue book”, her only book on topology Kees van Deemter, Dublin, Trinity College, May 2009

  21. Kees van Deemter, Dublin, Trinity College, May 2009

  22. Example by R.Parikh (the book scenario) • Bob only has to search all blue books • Ann’s instruction reduces the number of books that Bob can expect to have to check. • Each calls some books blue that the other does not. But they agree on most books. Kees van Deemter, Dublin, Trinity College, May 2009

  23. Ann’s books Ann’s books =1000 blue-Ann=250 blue-Ann-Bob=225 blue-Bob=300 Kees van Deemter, Dublin, Trinity College, May 2009

  24. What’s the utility of “the blue book”? Compare expected search times • without this instruction • with this instruction • Without instruction: ½*1000 = 500 • With instruction: 9/10*(½*300) + 1/10*(300+(1/2*700)) = 200 Kees van Deemter, Dublin, Trinity College, May 2009

  25. In Parikh’s example, “blue” is crisp.Scenario can be generalised to situations where each allows boundary cases. Kees van Deemter, Dublin, Trinity College, May 2009

  26. Why is language vague? Barton Lipman (in A.Rubinstein, “Economics and Language”, CUP 2000; working paper “Why is Language Vague” (2006)) When/why does vague communication give higher payoff than crisp language? Kees van Deemter, Dublin, Trinity College, May 2009

  27. One type of answer: conflict • S and H may have very different utility functions USand UH • (Crawford & Sobel 1982, Aragones and Neeman 2000): If USand UHare very different, it can be advantageous to hide information • “Our food is healthy!” • “Our burgers are big!” Kees van Deemter, Dublin, Trinity College, May 2009

  28. Kees van Deemter, Dublin, Trinity College, May 2009

  29. One type of answer: conflict • S and H may have very different utility functions USand UH • Crawford & Sobel 1982, Aragones and Neeman 2000: If USand UHare very different, it can be advantageous to hide information • “Our food is healthy!” • “Our burgers are big!” • Henceforth (following Lipman): US = UH Kees van Deemter, Dublin, Trinity College, May 2009

  30. Lipman’s questions When does vague communication give higher payoff than crisp language? Lipman: the airport scenario Kees van Deemter, Dublin, Trinity College, May 2009

  31. Kees van Deemter, Dublin, Trinity College, May 2009

  32. Lipman’s scenario Example:Airport scenario: I describe Mr X to you, so you can pick up X from the airport. All I know is X’s height; heights are distributed across people uniformly on [0,1]. If you identify X right away, you get payoff 1; if you don’t then you get payoff -1 Kees van Deemter, Dublin, Trinity College, May 2009

  33. Lipman: the airport scenario What description would work best? • Optimal communication: state X’s height as precisely as possible. If each of us knows X’s exact height then the probability of confusion is close to 0. Kees van Deemter, Dublin, Trinity College, May 2009

  34. Lipman: the airport scenario What description would work best? • Optimal communication: state X’s height as precisely as possible. If each of us knows X’s exact height then the probability of confusion at the airport is close to 0. Lipman: This is not vague, because there are no boundary cases! Kees van Deemter, Dublin, Trinity College, May 2009

  35. Some possible answers to Lipman Kees van Deemter, Dublin, Trinity College, May 2009

  36. 1. Necessary vagueness? “Input may be vague”. E.g.: • verbatim repetition (“hearsay”) • memory may cause details to fade (e.g., number of casualties in a disaster) • perception may have been inadequate (e.g., the height of a seated person) Kees van Deemter, Dublin, Trinity College, May 2009

  37. 1. Necessary vagueness? “Input may be vague”. E.g.: • verbatim repetition (“hearsay”) This begs the question • memory may cause details to faded (e.g., number of casualties in a disaster) ? • perception may have been inadequate (e.g., the height of a seated person)Lipman: Why can’t we convey exactlywhat our perception/memory is? E.g. “24 degrees +/- 2 degrees” Kees van Deemter, Dublin, Trinity College, May 2009

  38. the hazards of measurement 11m 12m Kees van Deemter, Dublin, Trinity College, May 2009

  39. Example: One house of 11m height and one house of 12m height • “the 12m house needs to be demolished” • “the tall house needs to be demolished” • Comparison is easier and more reliable than measurement  prefer utterance 2 (Van Deemter 2006) Kees van Deemter, Dublin, Trinity College, May 2009

  40. Example: One house of 11m height and one house of 12m height • “the 12m house needs to be demolished” • “the tall house needs to be demolished” • Comparison is easier and more reliable than measurement  prefer utterance 2 • But arguably, this utterance is not vague Its vagueness is merely local Kees van Deemter, Dublin, Trinity College, May 2009

  41. Apparent vagueness is frequent • ‘the tall house’ the tallest house • ‘Physical exercise is good for young and old’ regardless of age • ‘Bad for bacteria, good for gums’gums improve as a result of bacterial death • ‘Fast-flowing rivers are deep’the faster the deeper (positive correlation between variables) Kees van Deemter, Dublin, Trinity College, May 2009

  42. 3. Production/interpretation Effort • GTh can reason about the utility of an utterance • Effort needs to be commensurate with utility. In many cases, more precision adds little benefit (cf. Prashant Parikh 2000, Van Rooij 2003, Jaeger 2008) • E.g., the feasibility of an outing does not depend on whether it’s 20C or 30C. • ‘Mild’ takes fewer syllables than ‘twenty three point seven five’. • Vague words tend to be short (Krifka 2002) Kees van Deemter, Dublin, Trinity College, May 2009

  43. But: Why not round the figure?“The temperature is 24C” Kees van Deemter, Dublin, Trinity College, May 2009

  44. 4.Evaluation payoff • Example: You ask the doctor about your blood pressure. • Utterance 1: “Your blood pressure is 150/90.” • Utterance 2: “Your blood pressure is high.” • U2 offers less detail than U1 • But U2 also offers more: an evaluation of your condition. • A link with actions (cut down on salt, etc.) • Especially useful if metric is “difficult” Kees van Deemter, Dublin, Trinity College, May 2009

  45. But why does English not have a (brief) expression that says “Your blood pressure is 150/90 and too high”? Compare “You are obese” means “Your BMI is above 30 and this is dangerous”. Kees van Deemter, Dublin, Trinity College, May 2009

  46. 5.Lack of a good metric • Maybe areas where there exists a generally accepted measurement are rare • Multidimensional measurements: What’s the size of a house? • Maths: How difficult is a proof? (“As the reader may easily verify”) Kees van Deemter, Dublin, Trinity College, May 2009

  47. 5.Lack of a good metric • Maybe areas where there exists a generally accepted measurement are rare • Multidimensional measurements: What’s the size of a house? • Maths: How difficult is a proof? (“As the reader may easily verify”) • How beautiful is a sunset? Kees van Deemter, Dublin, Trinity College, May 2009

  48. Kees van Deemter, Dublin, Trinity College, May 2009

  49. 6.Future contingencies • Indecent Displays Control Act (1981) forbids public display of indecentmatter Kees van Deemter, Dublin, Trinity College, May 2009

  50. Indecent Displays Control Act (1981) forbids public display of indecentmatter • Indecent at the time  the law has been parameterised (Waismann 1968, Hart 1994, Lipman 2006) Kees van Deemter, Dublin, Trinity College, May 2009

More Related