l6-hmm-v2

.pdf

School

University of Melbourne *

*We aren’t endorsed by this school

Course

90042

Subject

Computer Science

Date

Oct 30, 2023

Type

pdf

Pages

46

Uploaded by DeanFinch3797

Report
COPYRIGHT, THE UNIVERSITY OF MELBOURNE 1 Semester 1 Week 3 Jey Han Lau Sequence Tagging: Hidden Markov Models COMP90042 Natural Language Processing Lecture 6
COMP90042 L6 2 POS Tagging Recap Janet will back the bill Janet/ NNP will/ MB back/ VB the/ DT bill/ NN Local classifier: prone to error propagation What about treating the full sequence as a “class”? Output: “NNP_MB_VB_DT_NN” Problems: Exponentially many combinations: |Tags| M ,for length M How to tag sequences of different lengths?
COMP90042 L6 3 A Better Approach Tagging is a sentence-level task but as humans we decompose it into small word-level tasks. Janet/ NNP will/ MB back/ VB the/ DT bill/ NN Solution: Define a model that decomposes process into individual word level steps But that takes into account the whole sequence when learning and predicting (no error propagation) This is the idea of sequence labelling , and more general, structured prediction.
COMP90042 L6 4 A Probabilistic Model Goal: obtain best tag sequence t from sentence w [Bayes] Let’s decompose: [Prob. of a word depends only on the tag] [Prob. of a tag depends only on the previous tag] This is a Hidden Markov Model (HMM) ^ ࠵? = ࠵?࠵?࠵?࠵?࠵?࠵? ࠵?   ࠵? ( ࠵? ࠵? ) ^ ࠵? = ࠵?࠵?࠵?࠵?࠵?࠵? ࠵?   ࠵? ( ࠵? ࠵? ) ࠵? ( ࠵? ) ࠵? ( ࠵? ) = ࠵?࠵?࠵?࠵?࠵?࠵? ࠵?   ࠵? ( ࠵? ࠵? )   ࠵? ( ࠵? ) ࠵? ( ࠵? ࠵? ) =   ࠵? ࠵? =1 ࠵? ( ࠵? ࠵? | ࠵? ࠵? ) ࠵? ( ࠵? ) =   ࠵? ࠵? =1 ࠵? ( ࠵? ࠵? | ࠵? ࠵? 1 )
COMP90042 L6 5 Two Assumptions of HMM Output independence An observed event (word) depends only on the hidden state (tag) Markov assumption The current state (tag) depends only on previous state ^ ࠵? = ࠵?࠵?࠵?࠵?࠵?࠵? ࠵?   ࠵? ( ࠵? ࠵? )   ࠵? ( ࠵? ) ࠵? ( ࠵? ࠵? ) =   ࠵? ࠵? =1 ࠵? ( ࠵? ࠵? | ࠵? ࠵? ) ࠵? ( ࠵? ) =   ࠵? ࠵? =1 ࠵? ( ࠵? ࠵? | ࠵? ࠵? 1 )
COMP90042 L6 6 HMMs - Training Parameters are the individual probabilities = emission ( O ) probabilities = transition ( A ) probabilities Training uses Maximum Likelihood Estimation (MLE) This is done by simply counting word frequencies according to their tags (just like N -gram LMs!) P ( w i | t i ) P ( t i | t i 1 ) ࠵? ( ࠵?࠵?࠵?࠵? ࠵?࠵? ) =   ࠵?࠵?࠵?࠵?࠵? ( ࠵?࠵? ,    ࠵?࠵?࠵?࠵? ) ࠵?࠵?࠵?࠵?࠵? ( ࠵?࠵? ) ࠵? ( ࠵?࠵? ࠵?࠵? ) =   ࠵?࠵?࠵?࠵?࠵? ( ࠵?࠵? , ࠵?࠵? ) ࠵?࠵?࠵?࠵?࠵? ( ࠵?࠵? )
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
  • Access to all documents
  • Unlimited textbook solutions
  • 24/7 expert homework help

Browse Popular Homework Q&A

Q: Why is it so important to have a plan to achieve your goals?
Q: In Drosophila, females heterozygous for the recessive alleles c, d, and e were crossed to males…
Q: A proton is moving under the combined influence of an electric field (E = 1747 V/m) and a magnetic…
Q: How many software architectural models are there and what are they?
Q: A speeder travelling at a constant velocity of 40 m/s passes by a police car that is initially at…
Q: kg suitcase slides from rest r = 8.5 m down the smooth ramp as shown in (Figure 1). (a) Determine…
Q: Provide an explanation of the three most important cloud computing platforms.
Q: (a) Select all of the correct statements about reaction rates from the choices below. The lower the…
Q: Subtract the polynomials. (3x³-x+5)-(-6x³ + 8x+2)
Q: What power contact lens is needed to correct for farsightedness where the uncorrected near point is…
Q: Cestit
Q: A 20 ft ladder leans agaainst a building so that the angle between the ground and the ladder is 70…
Q: Pm Pc W th H = Assume that the drawing at left represents the cross section of a broad plateau that…
Q: Read the thermometer and report the temperature with the proper number of digits.
Q: do resistance and resistivity depend on: - the wire length? - the wire cross-section? - the wire…
Q: Dr Dr 9 Supples o ASSETS O Prop O Office Exppmant O LABAITS Accounts Payable O or Account Name Cash…
Q: 5) Historically, the percentage of U.S. residents who support stricter gun control laws has been…
Q: (1x30'+ 3xD+1)y:о, усідів, у'(1)-о утобу'+0.09у= y (0) = 2.2, y' (0) = 0.14 -0.3x -o-3x e хе
Q: Find the unknown sides (in m) of the right triangle. (See figure below. Round your answers to four…
Q: Describe the procedure for trach care and suctioning
Q: Illustrate your algorithm by showing on paper similar to Fig. 8.3, page 198 in the textbook (make…
Q: 319 270 59 245 30 der Accurate 131 der Not Accurate 37 17 rder is selected, find the probability of…