hmm viterbi.docx

hmm viterbi.docx

《hmm viterbi.docx》由会员分享，可在线阅读，更多相关《hmm viterbi.docx（17页珍藏版）》请在冰豆网上搜索。

hmmviterbi

Viterbialgorithm

FromWikipedia,thefreeencyclopedia

Jumpto:

navigation,search

TheViterbialgorithmisadynamicprogrammingalgorithmforfindingthemostlikelysequenceofhiddenstates–calledtheViterbipath–thatresultsinasequenceofobservedevents,especiallyinthecontextofMarkovinformationsources,andmoregenerally,hiddenMarkovmodels.Theforwardalgorithmisacloselyrelatedalgorithmforcomputingtheprobabilityofasequenceofobservedevents.Thesealgorithmsbelongtotherealmofinformationtheory.

Thealgorithmmakesanumberofassumptions.

∙First,boththeobservedeventsandhiddeneventsmustbeinasequence.Thissequenceoftencorrespondstotime.

∙Second,thesetwosequencesneedtobealigned,andaninstanceofanobservedeventneedstocorrespondtoexactlyoneinstanceofahiddenevent.

∙Third,computingthemostlikelyhiddensequenceuptoacertainpointtmustdependonlyontheobservedeventatpointt,andthemostlikelysequenceatpointt−1.

Theseassumptionsareallsatisfiedinafirst-orderhiddenMarkovmodel.

Theterms"Viterbipath"and"Viterbialgorithm"arealsoappliedtorelateddynamicprogrammingalgorithmsthatdiscoverthesinglemostlikelyexplanationforanobservation.Forexample,instatisticalparsingadynamicprogrammingalgorithmcanbeusedtodiscoverthesinglemostlikelycontext-freederivation（parse）ofastring,whichissometimescalledthe"Viterbiparse".

TheViterbialgorithmwasconceivedbyAndrewViterbiin1967asadecodingalgorithmforconvolutionalcodesovernoisydigitalcommunicationlinks.FormoredetailsonthehistoryofthedevelopmentofthealgorithmseeDavidForney'sarticle[1].ThealgorithmhasfounduniversalapplicationindecodingtheconvolutionalcodesusedinbothCDMAandGSMdigitalcellular,dial-upmodems,satellite,deep-spacecommunications,and802.11wirelessLANs.Itisnowalsocommonlyusedinspeechrecognition,keywordspotting,computationallinguistics,andbioinformatics.Forexample,inspeech-to-text（speechrecognition）,theacousticsignalistreatedastheobservedsequenceofevents,andastringoftextisconsideredtobethe"hiddencause"oftheacousticsignal.TheViterbialgorithmfindsthemostlikelystringoftextgiventheacousticsignal.

Contents

[hide]

∙1Overview

∙2Algorithm

∙3Example

∙4Javaimplementation

∙5C#Implementation

∙6Extensions

∙7Seealso

∙8Notes

∙9References

∙10Externallinks

[edit]Overview

Theassumptionslistedabovecanbeelaboratedasfollows.TheViterbialgorithmoperatesonastatemachineassumption.Thatis,atanytimethesystembeingmodeledisinsomestate.Thereareafinitenumberofstates.Whilemultiplesequencesofstates（paths）canleadtoagivenstate,atleastoneofthemisamostlikelypathtothatstate,calledthe"survivorpath".Thisisafundamentalassumptionofthealgorithmbecausethealgorithmwillexamineallpossiblepathsleadingtoastateandonlykeeptheonemostlikely.Thiswaythealgorithmdoesnothavetokeeptrackofallpossiblepaths,onlyoneperstate.

Asecondkeyassumptionisthatatransitionfromapreviousstatetoanewstateismarkedbyanincrementalmetric,usuallyanumber.Thistransitioniscomputedfromtheevent.Thethirdkeyassumptionisthattheeventsarecumulativeoverapathinsomesense,usuallyadditive.Sothecruxofthealgorithmistokeepanumberforeachstate.Whenaneventoccurs,thealgorithmexaminesmovingforwardtoanewsetofstatesbycombiningthemetricofapossiblepreviousstatewiththeincrementalmetricofthetransitionduetotheeventandchoosesthebest.Theincrementalmetricassociatedwithaneventdependsonthetransitionpossibilityfromtheoldstatetothenewstate.Forexampleindatacommunications,itmaybepossibletoonlytransmithalfthesymbolsfromanoddnumberedstateandtheotherhalffromanevennumberedstate.Additionally,inmanycasesthestatetransitiongraphisnotfullyconnected.Asimpleexampleisacarthathas3states—forward,stopandreverse—andatransitionfromforwardtoreverseisnotallowed.Itmustfirstenterthestopstate.Aftercomputingthecombinationsofincrementalmetricandstatemetric,onlythebestsurvivesandallotherpathsarediscarded.Therearemodificationstothebasicalgorithmwhichallowforaforwardsearchinadditiontothebackwardsonedescribedhere.

Pathhistorymustbestored.Insomecases,thesearchhistoryiscompletebecausethestatemachineattheencoderstartsinaknownstateandthereissufficientmemorytokeepallthepaths.Inothercases,aprogrammaticsolutionmustbefoundforlimitedresources:

oneexampleisconvolutionalencoding,wherethedecodermusttruncatethehistoryatadepthlargeenoughtokeepperformancetoanacceptablelevel.AlthoughtheViterbialgorithmisveryefficientandtherearemodificationsthatreducethecomputationalload,thememoryrequirementstendtoremainconstant.

[edit]Algorithm

SupposingwearegivenaHiddenMarkovModel（HMM）withstatesY,initialprobabilitiesπiofbeinginstateiandtransitionprobabilitiesai,joftransitioningfromstateitostatej.Sayweobserveoutputs

.Thestatesequence

mostlikelytohaveproducedtheobservationsisgivenbytherecurrencerelations:

[1]

HereVt,kistheprobabilityofthemostprobablestatesequenceresponsibleforthefirstt+1observations（weaddonebecauseindexingstartedat0）thathaskasitsfinalstate.TheViterbipathcanberetrievedbysavingbackpointerswhichrememberwhichstateywasusedinthesecondequation.LetPtr（k,t）bethefunctionthatreturnsthevalueofyusedtocomputeVt,kift>0,orkift=0.Then:

Thecomplexityofthisalgorithmis

.

[edit]Example

Considertwofriends,AliceandBob,wholivefarapartfromeachotherandwhotalktogetherdailyoverthetelephoneaboutwhattheydidthatday.Bobisonlyinterestedinthreeactivities:

walkinginthepark,shopping,andcleaninghisapartment.Thechoiceofwhattodoisdeterminedexclusivelybytheweatheronagivenday.AlicehasnodefiniteinformationabouttheweatherwhereBoblives,butsheknowsgeneraltrends.BasedonwhatBobtellsherhedideachday,Alicetriestoguesswhattheweathermusthavebeenlike.

AlicebelievesthattheweatheroperatesasadiscreteMarkovchain.Therearetwostates,"Rainy"and"Sunny",butshecannotobservethemdirectly,thatis,theyarehiddenfromher.Oneachday,thereisacertainchancethatBobwillperformoneofthefollowingactivities,dependingontheweather:

"walk","shop",or"clean".SinceBobtellsAliceabouthisactivities,thosearetheobservations.TheentiresystemisthatofahiddenMarkovmodel（HMM）.

Aliceknowsthegeneralweathertrendsinthearea,andwhatBoblikestodoonaverage.Inotherwords,theparametersoftheHMMareknown.TheycanbewrittendowninthePythonprogramminglanguage:

states=（'Rainy','Sunny'）

observations=（'walk','shop','clean'）

start_probability={'Rainy':

0.6,'Sunny':

0.4}

transition_probability={

'Rainy':

{'Rainy':

0.7,'Sunny':

0.3},

'Sunny':

{'Rainy':

0.4,'Sunny':

0.6},

}

emission_probability={

'Rainy':

{'walk':

0.1,'shop':

0.4,'clean':

0.5},

'Sunny':

{'walk':

0.6,'shop':

0.3,'clean':

0.1},

}

Inthispieceofcode,start_probabilityrepresentsAlice'sbeliefaboutwhichstatetheHMMisinwhenBobfirstcallsher（allsheknowsisthatittendstoberainyonaverage）.Theparticularprobabilitydistributionusedhereisnottheequilibriumone,whichis（giventhetransitionprobabilities）approximately{'Rainy':

0.57,'Sunny':

0.43}.Thetransition_probabilityrepresentsthechangeoftheweatherintheunderlyingMarkovchain.Inthisexample,thereisonlya30%chancethattomorrowwillbesunnyiftodayisrainy.Theemission_probabilityrepresentshowlikelyBobistoperformacertainactivityoneachday.Ifitisrainy,thereisa50%chancethatheiscleaninghisapartment;ifitissunny,thereisa60%chancethatheisoutsideforawalk.

AlicetalkstoBobthreedaysinarowanddiscoversthatonthefirstdayhewentforawalk,ontheseconddayhewentshopping,andonthethirddayhecleanedhisapartment.Alicehasaquestion:

whatisthemostlikelysequenceofrainy/sunnydaysthatwouldexplaintheseobservations?

ThisisansweredbytheViterbialgorithm.

#HelpsvisualizethestepsofViterbi.

defprint_dptable（V）:

print"",

foriinrange（len（V））:

print"%7s"%（"%d"%i）,

print

foryinV[0].keys（）:

print"%.5s:

"%y,

fortinrange（len（V））:

print"%.7s"%（"%f"%V[t][y]）,

print

defviterbi（obs,states,start_p,trans_p,emit_p）:

V=[{}]

path={}

#Initializebasecases（t==0）

foryinstates:

V[0][y]=start_p[y]*emit_p[y][obs[0]]

path[y]=[y]

#RunViterbifort>0

fortinrange（1,len（obs））:

V.append（{}）

newpath={}

foryinstates:

（prob,state）=max（[（V[t-1][y0]*trans_p[y0][y]*emit_p[y][obs[t]],y0）fory0instates]）

V[t][y]=prob

newpath[y]=path[state]+[y]

#Don'tneedtoremembertheoldpaths

path=newpath

print_dptable（V）

（prob,state）=max（[（V[len（obs）-1][y],y）foryinstates]）

return（prob,path[state]）

Thefunctionviterbitakesthefollowingarguments:

obsisthesequenceofobservations,e.g.['walk','shop','clean'];statesisthesetofhiddenstates;start_pisthestartprobability;trans_parethetransitionprobabilities;andemit_paretheemissionprobabilities.Forsimplicityofcode,weassumethattheobservationsequenceobsisnon-emptyandthattrans_p[i][j]andemit_p[i][j]isdefinedforallstatesi,j.

In