Linguistic Resources for the 2013 TAC KBP Entity Linking Evaluation

1. Linguistic Resources for the 2013 TAC KBP Entity Linking Evaluation Joe Ellis (presenter), Justin Mott, Xuansong Li, Jeremy Getman, Jonathan Wright, Stephanie Strassel Linguistic Data Consortium University of Pennsylvania, USA

2. 2013 Source Corpus TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

3. Entity Linking Overview Stage 1: Select name strings and ref docs Stage 2: Link namestrings to KB or mark as NIL TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

4. Entity Linking Overview Stage 1: Select name strings and ref docs Stage 2: Link namestrings to KB or mark as NIL TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

5. Entity Linking Overview Stage 1: Select name strings and ref docs Stage 2: Link namestrings to KB or mark as NIL TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

6. Entity Linking Overview Stage 1: Select name strings and ref docs Stage 2: Link namestrings to KB or mark as NIL Stage 3: Co-reference NIL entities Wendy Wendy Gaxiola TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

7. Entity Linking Overview Stage 1: Select name strings and ref docs Stage 2: Link namestrings to KB or mark as NIL Stage 3: Co-reference NIL entities Wendy Wendy Gaxiola TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

8. Entity Linking – Stage 1 • Run named entity taggers over source corpora • Provides guided search through the corpus • Thanks KBP coordinators! • Namestring Selection • Confusable TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

9. Entity Linking – Stage 1 • Run named entity taggers over source corpora • Provides guided search through the corpus • Thanks KBP coordinators! • Namestring Selection • Confusable TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

10. Entity Linking – Stage 1 • Run named entity taggers over source corpora • Provides guided search through the corpus • Thanks KBP coordinators! • Namestring Selection • Confusable TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

11. Entity Linking – Stage 1 • Run named entity taggers over source corpora • Provides guided search through the corpus • Thanks KBP coordinators! • Namestring Selection • Confusable • Ambiguous • Varied ? TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

12. Entity Linking – Stage 1 • Run named entity taggers over source corpora • Provides guided search through the corpus • Thanks KBP coordinators! • Namestring Selection • Confusable • Ambiguous • Varied ? TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

13. Entity Linking – Stage 1: Namestring Selection • Ratios • NIL & non-NIL • Entity types • Genre • Measurable confusability • Multiple-entity namestrings (“Smith”) • Multiple-namestring entities (“Barack Obama”, “Bam-Bam”, “Bammy”) • NIL singletons • Cross-lingual TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

14. Entity Linking – Stages 2 & 3: KB Linking and NIL Coref • KB Linking • Review ref document and search KB for matching node • Multiple namestrings viewed together for quicker linking • NIL Coreference • NIL queries (no KB match) require manual co-reference annotation • Time-limited quality control pass to enhance completeness and accuracy TAC KBP Evaluation Workshop – NIST, November 18-19, 2013

15. Delivered 2013 Resources TAC KBP Evaluation Workshop – NIST, November 18-19, 2013