1 / 46

Lecture 13 Information Extraction

Lecture 13 Information Extraction. CSCE 771 Natural Language Processing. Topics Name Entity Recognition Relation detection Temporal and Event Processing Template Filling Readings: Chapter 22. February 27, 2013. Overview. Last Time Dialogues Human conversations Today

annis
Download Presentation

Lecture 13 Information Extraction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lecture 13Information Extraction CSCE 771 Natural Language Processing • Topics • Name Entity Recognition • Relation detection • Temporal and Event Processing • Template Filling • Readings: Chapter 22 February 27, 2013

  2. Overview • Last Time • Dialogues • Human conversations • Today • Slides from Lecture24 • Dialogue systems • Dialogue Manager Design • Finite State, Frame-based, Initiative: User, System, Mixed • VoiceXML • Information Extraction • Readings • Chapter 24, Chapter 22

  3. Information extraction • Information extraction – turns unstructured information buried in texts into structured data • Extract proper nouns – “named entity recognition” • Reference resolution – \ • named entity mentions • Pronoun references • Relation Detection and classification • Event detection and classification • Temporal analysis • Template filling

  4. Template Filling • Example template for “airfare raise”

  5. Figure 22.1 List of Named Entity Types

  6. Figure 22.2 Examples of Named Entity Types

  7. Figure 22.3 Categorical Ambiguities

  8. Figure 22.4 Categorical Ambiguity

  9. Figure 22.5 Chunk Parser for Named Entities

  10. Figure 22.6 Features used in Training NER • Gazetteers – lists of place names • www.geonames.com • www.census.gov

  11. Figure 22.7 Selected Shape Features

  12. Figure 22.8 Feature encoding for NER

  13. Figure 22.9 NER as sequence labeling

  14. Figure 22.10 Statistical Seq. Labeling

  15. Evaluation of Named Entity Rec. Sys. • Recall terms from Information retreival • Recall = #correctly labeled / total # that should be labeled • Precision = # correctly labeled / total # labeled • F- measure where βweights preferences • β=1 balanced • β>1 favors recall • β<1 favors precision

  16. NER Performance revisited • NER performance revisited • Recall, Precision, F • High performance systems • F ~ .92 for PERSONS and LOCATIONS and ~.84 for ORG • Practical NER • Make several passes on text • Start by using highest precision rules (maybe at expense of recall) make sure what you get is right • Search for substring matches or previously detected names using probabilistic searches string matching metrics(Chap 19) • Name lists focused on domain • Probabilistic sequence labeling techniques using previous tags

  17. Relation Detection and classification • Consider Sample text: • Citing high fuel prices, [ORG United Airlines] said [TIME Friday] it has increased fares by [MONEY $6] per round trip on flights to some cities also served by lower-cost carriers. [ORG American Airlines], a unit of [ORG AMR Corp.], immediately matched the move, spokesman [PERSON Tim Wagner] said. [ORG United Airlines] an unit of [ORG UAL Corp.], said the increase took effect [TIME Thursday] and applies to most routes where it competes against discount carriers, such as [LOC Chicago] to [LOC Dallas] and [LOC Denver] to [LOC San Francisco]. • After identifying named entities what else can we extract? • Relations

  18. Fig 22.11 Example semantic relations

  19. Figure 22.12 Example Extraction

  20. Figure 22.13 Supervised Learning Approaches to Relation Analysis • Algorithm two step process • Identify whether pair of named entities are related • Classifier is trained to label relations

  21. Factors used in Classifying • Features of the named entities • Named entity types of the two arguments • Concatenation of the two entity types • Headwords of the arguments • Bag-of-words from each of the arguments • Words in text • Bag-of-words and Bag-of-digrams • Stemmed versions • Distance between named entities (words / named entities) • Syntactic structure • Parse related structures

  22. Figure 22.14 a-part-of relation

  23. Figure 22.15 Sample features Extracted

  24. Bootstrapping Example “Has a hub at” • Consider the pattern • / * has a hub at * / • Google search • 22.4 Milwaukee-based Midwest has a hub at KCI • 22.5 Delta has a hub at LaGuardia • … • Two ways to fail • False positive: e.g. a star topology has a hub at its center • False negative? Just miss • 22.11 No frill rival easyJet, which has established a hub at Liverpool

  25. Figure 22.16 Bootstrapping Relation Extraction

  26. Using Features to restrict patterns • 22.13 Budget airline Ryanair, which uses Charleroi as a hub, scrapped all weekend flights • / [ORG] , which uses a hub at [LOC] /

  27. Semantic Drift • Note it will be difficult (impossible) to get annotated materials for training • Accuracy of process is heavily dependant on initial sees • Semantic Drift – • Occurs when erroneous patterns(seeds) leads to the introduction of erroneous tuples

  28. Fig 22.17 Temporal and Durational Expressions • Absolute temporal expressions • Relative temporal expressions

  29. Fig 22.18 Temporal lexical triggers

  30. Fig 22.19 MITRE’s tempEx tagger-perl

  31. Fig 22.20 Features used to train IOB

  32. Figure 22.21 TimeML temporal markup

  33. Temporal Normalization • iSO 8601 - standard for encoding temporal values • YYYY-MM-DD

  34. Figure 22.22 Sample ISO Patterns

  35. Event Detection and Analysis • Event Detection and classification

  36. Fig 22.23 Features for Event Detection • Features used in rule-based and statistical techniques

  37. Fig 22.24 Allen’s 13 temporal Relations

  38. Figure 22.24 continued

  39. Figure 22.25 Example from Timebank Corpus

  40. Template Filling

  41. Figure 22.26 Templates produced by Faustus 1997

  42. Figure 22.27 Levels of processing in Faustus

  43. Figure 22.28 Faustus Stage 2

  44. Figure 22.29 The 5 Partial Templates of Faustus

  45. Figure 22.30 Articles in PubMed

  46. Figure 22.31 biomedical classes of named entities

More Related