1 / 17

Dynamic Programming Viterbi

Dynamic Programming Viterbi. Today. Take a look at dynamic programming algorithms Walk through some examples Simple chart parse Viterbi example (from speech) Viterbi example (from POS tagging). Dynamic Programming.

gayora
Download Presentation

Dynamic Programming Viterbi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic ProgrammingViterbi

  2. Today • Take a look at dynamic programming algorithms • Walk through some examples • Simple chart parse • Viterbi example (from speech) • Viterbi example (from POS tagging)

  3. Dynamic Programming • Definition: Algorithmic approach for solving optimization problems by caching sub-problem solutions rather than recomputing them. • Typical device used is a chart to hold the various sub-problem solutions. • Common implementation: Charts used in parsing (based on the Earley algorithm)

  4. BU Chart Parser Walk Through Sample Sentence: The large can can hold the water. Sample Grammar: 1. S  NP VP 2. NP  D ADJ N 3. NP  D N 4. NP  ADJ N 5. VP  AUX VP 6. VP  V NP

  5. Chart Parser • Allows one to preserve the results of the parse so far in a chart • Partial results and all previous results are kept so that work is not repeated

  6. 1. S  NP VP 2. NP  D ADJ N 3. NP  D N 4. NP  ADJ N 5. VP  AUX VP 6. VP  V NP The large can can hold the water. D1 ADJ1 1 2 3 NP  D ° ADJ N NP  D ADJ ° N NP  D ° N NP  ADJ ° N

  7. 1. S  NP VP 2. NP  D ADJ N 3. NP  D N 4. NP  ADJ N 5. VP  AUX VP 6. VP  V NP The large can can hold the water. NP2 NP1 D1 ADJ1 N1 AUX1 - V1 1 2 3 4 NP  D ° ADJ N NP  D ADJ N ° NP  D ADJ ° N NP  D ° N NP  ADJ ° N NP  ADJ N ° S  NP ° VP S  NP ° VP

  8. Viterbi Example See word models, p. 245

  9. Viterbi Example

  10. Viterbi example • Input is: [aa n iy dh ax] • “I need the…” • Actually dialectal/fast speech variant, “Ah nee the” • [a ni ðə]

  11. [aa n iy dh ax]

  12. Viterbi Example

  13. A smaller example very quick very quick 0.1 0.9 0.8 0.2 0.7 end start JJ RB 1 1 0.1 0.3 0.9 • What is the best sequence of states for the input string “very very quick”? • Computing all possible paths and finding the one with maximum prob. is exponential

  14. very quick very quick 0.1 0.9 0.8 0.2 0.7 end start JJ RB 1 1 0.1 0.3 0.9

  15. very quick very quick 0.1 0.9 0.8 0.2 0.7 end start JJ RB 1 1 0.1 0.3 0.9

  16. very quick very quick 0.1 0.9 0.8 0.2 0.7 end start JJ RB 1 1 0.1 0.3 0.9

  17. Implementation // Initialize viterbi[1,PERIOD] = 1.0 at the start, assuming dummy period for i:=1 to n step 1 do // all input words (presumably in a sentence) for all tags tj do // for all possible tags // max probability of being in state j (tag j) at word i+1 (path probability matrix) viterbi[i+1,tj] := max1≤k≤T(viterbi[i,tk] x P(wi+1|tj) x P(tj|tk)) // most likely state (tag) at word i given that we’re in state j at word i+1 // In other words, state[i+1] is keeping a pointer to the state that got us here state[i+1,tj] := argmax1≤k≤T ( viterbi[i,tk] x P(wi+1|tj) x P(tj|tk)) end end //Termination and path-readout bestPathn+1 := argmax1≤j≤T viterbi[n+1,j] for j:=n to 1 step -1 do // for all input words bestPathj := state[i+1, bestPathj+1] end P(bestPath1,…, bestPathn ) := max1≤j≤T viterbi[n+1,j] Emission probability State transition probability

More Related