JunctionTree.docx

JunctionTree.docx

《JunctionTree.docx》由会员分享，可在线阅读，更多相关《JunctionTree.docx（31页珍藏版）》请在冰豆网上搜索。

JunctionTree

JunctionTrees:

Motivation

•Standardalgorithms（e.g.,variableelimination）areinefficientiftheundirectedgraphunderlyingtheBayesNetcontainscycles.

•Wecanavoidcyclesifweturnhighlyinterconnectedsubsetsofthenodesinto

“supernodes.”

ARunningExamplefortheStepsinConstructingaJunctionTree

ImaginewestartwithaBayesNethavingthefollowingstructu

StApeadrpdean1ntse:

odfMgtheebasekatwmeeeetcnhhniloedn.-Gadjraacepnth（uMnmaorrrieadl

Step2:

RemoveDirectionality

Step3:

TriangulatetheGraph

RepeatAwedhrditliecaetchsheinorersdeu（xceihsdtagseacybcceyletcw）le.eeonfltewnognthon>-3adwjaitchennotchord:

v

Step3:

TriangulatetheGraph

cRheopredAajetodrwdintihcianielngesevtihndeegrstreuiecce（echsxhaitoshtrcasdyt）acablecreey.tcwnleeoetonafdltewjanocgenthnotn>i-na3tdhwjaeitchceynncov

Step3:

TriangulatetheGraph

cRheopredAajetodrwdintihcianielngesevtihndeegrstreuiecce（echsxhaitoshtrcasdyt）acablecreey.tcwnleeoetonafdltewjanocgenthnotn>i-na3tdhwjaeitchceynncov

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

TriangulationChecking

ThefollowingMaximumCardinalitySearchalgorithmterminateswith

successifandonlyifthegraphistriangulated.Itprocesseseachnode,andthetimetoprocessanodeisquadraticinthenumberofadjacentnodes.

Chooseanynodeinthegraphandlabelit1.

Fori=2ton（totalnumberofnodesinthegraph）:

Choosethenodewiththemostlabeledneighborsandlabeliti.

Ifanytwolabeledneighborsofiarenotadjacenttoeachother,FAIL.SUCCEED.

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

IsitTriangulatedYet?

Itappearstobetriangulated,buthowcanwebesure

ItisNotTriangulated

Noedgebetweennodes5and6,bothofwhichareparentsof

FixingtheFaultyCycle

ContinuingourCheck...

Minutrsotdruecsetadrtaanlgeowricthymcle..OImrdaegriinngemreipgehattcinhganpgroec,eodruwreemwiitghhtthhisavgera.

ContinuingourCheck...

F6owlleowseinegtohuartietsarnlieeirgchhbaonrsge5,a5nids3noawreannoetigahdbjaocreonft6--.aWnohtehnerwfeix.r.

FixingthisProblem

Aofd6dipngroavindeesdgaembisestwinegecnhtohrednaosnb-aedlojawc.entparents3and5

ContinuingourCheck...

Wtthwroeomnuogohwrenhoeaddvgeeet7hseaasngdbraefpilnohawbl.lyeAlsosuwcw.ceeRceeodsnitnatigrntuiwneigt,h,we.e.v.eernydthuinpgaidsdfiinneg

TheFollowingisTriangulated

11

Triangulation:

KeyPoints

•Previousalgorithmisanefficientchecker,butnotnecessarilybestwaytotriangulate.

•Ingeneral,manytriangulationsmayexist.Theonlyefficientalgorithmsareheuristic.

•JensenandJensen（1994）showedthatanyschemeforexactinference（beliefupdatinggivenevidence）mustperformtriangulation（perhapshiddenasinDraper1995）.

Definitions

•Completegraphornodeset:

allnodesareadjacent.

•Clique:

maximalcompletesubgraph.

•Simplicialnode:

nodewhosesetofneighborsisacompletenodeset.

Step4:

BuildCliqueGraph

lbaeeibncleodolwema,dlelthwcs1eliiatqyhunateohrsedeeiijnroiiinntnhteteehdremsineocclrtithqaioeluiznecefldgirqor,uamtrepiahbgn.ergaloIufpwlthaw（tobesyhdc1alagi1qrnreuadeepdsnhgo.inedAteesrcs）li.eqcutebF

TheCliqueGraph

Thelabelofanedgebetweentwocliquesiscalledthesepa

1C,12,32,3C2,23,4,555C,77,9,105533,4,545,55,755,7,99,10

C3,34,5,64,5,6C4,45,6,75,7C5,67,8,99C9,810,11

5,666C,98,61286,85,6,7C5,56,57,,785,7,8

JunctionTrees

•Ajunctiontreeisasubgraphofthecliquegraphthat

（1）isatree,

（2）containsallthenodesofthecliquegraph,and（3）satisfiesthejunctiontreeproperty.

•Junctiontreeproperty:

ForeachpairU,VofcliqueswithintersectionS,allcliquesonthepathbetweenUandVcontainS.

CliqueGraphtoJunctionTree

•Wecanperformexactinferenceefficientlyonajunctiontree（althoughCPTsmaybelarge）.Butcanwealwaysbuildajunctiontree?

Ifso,how?

•Lettheweightofanedgeinthecliquegraphbethecardinalityoftheseparator.

Thananymaximumweightspanningtreeisajunctiontree（Jensen&Jensen1994）.

Step5:

BuildtheJunctionTree

1C,12,32,3C2,23,4,55C,77,9,10

3,4,55,7,99,10

C3,34,5,64,5,6C4,45,6,7C5,67,8,9C9,810,11

5,6,75,7,8

C6,98,126,8C5,56,7,8

Step6:

ChooseaRoot

5,6,7

4C,45,6,75C6C,,586,7,85,7,8

34,,3C45,,345,5,66,98,1255CC,7,677,9,8,51,,907,9,6

2C2,3,23,4,5C9,8109,1,110

C1,12,3

Step7:

PopulateCliqueNodes

•Foreachdistribution（CPT）intheoriginalBayesNet,putthisdistributionintooneofthecliquenodesthatcontainsallthevariablesreferencedbytheCPT.（Atleastonesuchnodemustexistbecauseofthemoralizationstep）.

•Foreachcliquenode,taketheproductofthedistributions（asinvariableelimination）.

BetterTriangulationAlgorithm

SpecificallyforBayesNets,BasedonVariableElimination

•Repeatuntilnonodesremain:

–Ifthegraphhasasimplicialnode,eliminateit（considerit“processed”andremoveittogetherwithallitsedges）.

–Otherwise,findthenodewhoseeliminationwouldgivethesmallestpotentialpossible.Eliminatethatnode,andnotetheneedfora“fill-in”edgebetweenanytwonon-adjacentnodesintheresultingpotential.

•Addthe“fill-in”edgestotheoriginalgraph.

FindCliqueswhileTriangulating

•nWsiemhidgipelh（elaibocenoixradreslciintaunsosteidandtegrgp,ierotheassecsanoipsbrgrlbdeeeuvtcfhiolloaiarquteustn.）eeao.lddg（eoTgwrhiietrthnhamrap:

elmlhfoiot）rsveeatchhatno

•oAnfteerthraetciosradisnugbaselltpoofsasniboltehecrli.ques,throwoutany

••OtTtrrhii（aanenn3rgg）e,uumgllaauattaieenrddai.nnggtreasepedhts.coarreretchteocnlliyquifesgrianpthheis

ChooseRoot,Assigne|CCPTs

|

bbdP..47（BD|..B63d）|BBCDBCCCDDEDEDccEd..45F.5e..65e.e4|CdP.（8eE|C）

P（A,B,C）ABCf|D,E..e91.5.6.2

b

aa..01c0672b..0604c38..0016c83..0072c27f|D,EP（F|D,E）

JunctionTreeInference

•IncorporateEAvidlgenocrei:

tFhomreachevidencevariable,gotoonetablethatincludesthatvariable.Setto0allentriesinthattablethatdisagreewiththeevidence.

•UpwardStep:

Foreachleafinthejunctiontree,sendamessagetoitsparent.Themessageisthemarginalofitstable,...

J.T.Inference（Continued）

•（UpwardStepcontinued）…summingoutanyvariablenotintheseparator.Whenaparentreceivesamessagefromachild,itmultipliesitstablebythemessagetabletoobtainitsnewtable.Whenaparentreceivesmessagesfromallitschildren,itrepeatstheprocess（actsasaleaf）.Thisprocesscontinuesuntiltherootreceivesmessagesfromallitschildren.

J.T.Inference（Continued）

•DownwardStep:

（Roughlyreversestheupwardprocess,startingattheroot.）Foreachchild,therootsendsamessagetothatchild.Morespecifically,therootdividesitscurrenttablebythemessagereceivedfromthatchild,marginalizestheresultingtabletotheseparator,andsendstheresultofthismarginalizationtothechild.Whena...

J.T.Inference（Continued）

•（DownwardStepcontinued）…childreceivesamessagefromitsparent,multiplyingthismessagebythechild’scurrenttablewillyieldthejointdistributionoverthechild’svariables（ifthechilddoesnotalreadyhaveit）.Theprocessrepeats（thechildactsasroot）andcontinuesuntilallleavesreceivemessagesfromtheirparents.

OneCatchforDivision

•Attimeswemayfindourselvesneedingtodivideby0.

•Wecanverifythatwheneverthisoccurs,wearedividing0by0.

•Wesimplyadopttheconventionthatforthisspecialcase,theresultwillbe0ratherthanundefined.

InferenceExample（assumeno

evidence）:

GoingUp

P（PB（,CC,）c6D9）.6cc5c1..31d3204..1422Bd60CDBCCDDEDPE（|DD,|EE|）ddF11|e..00|11..00e

C

bb..0181.099ABC

StatusAfterUpwardPass

bbP..（00dB6587,Cc..0,2D4.）069c.0339d01CDDEDEcc..01e6322d..0169e28..10e6683d..2056e23P（C,D,E）10d1.2d60.

C

BCDBCDEF..19..55..46..82

P（F|D,E）

ABC..010672..060438..001683..007227

P（A,B,C）

GoingBackDown

Weffiellchta-vieg1nn.c0oore1B.0cCDBCCDDEDEDEeeF..d213914..32d1650P（D,E）C

ABC

StatusAfterDownwardPass

P（B,C

b.06d8.1c0,d1D.2）6d0.3c9d1c.06e2.d06e2.06e3.0d6e3P（C,D,E）

b.057.024.0BA6CB9.0CDB3C0aaC..CD016D2b.E06D04cE38..00c16c83.1b..00372c227.198.168.e2159d2.13e0.0e92d.25e2c07.DEFff..0175P.1（3D0,.1E3,F9.）063

AnsweringQueries:

FinalStep

•Havingbuiltthejunctiontree,wenowcanaskaboutanyvariable.Wefindthecliquenodecontainingthatvariableandsumouttheothervariablestoobtainouranswer.

•Ifgivennewevidence,wemustrepeattheUpward-Downwardprocess.

•Ajunctiontreecanbethoughtofasstoringthesubjointscomputedduringelimination.

SignificanceofJunctionTrees

•“…onlywell-understood,efficient,provablycorrectmethodforconcurrentlycomputingmultiplequeries（AIMag’99）.”

•Asaresult,theyarethemostwidely-usedandwell-knownmethodofinferenceinBayesNets,although…

•JunctiontreessoonmaybeovertakenbyapproximateinferenceusingMCMC.

TheLinkBetweenJunctionTreesandVariableElimination

•Toeliminateavariableatanystep,wecombineallremainingdistributions（tables）indexedon（involving）thatvariable.

•Anodeinthejunctiontreecorrespondstothevariablesinoneofthetablescreatedduringvariable