HammersleyClifford定理.docx

1、HammersleyClifford定理Hammersley Clifford 定理Hammersley- Clifford theorem The Hammersley - Clifford theorem is aresult in probability theory,mathematical statistics and statistical mecha ni cs,that gives n ecessary and sufficie nt con diti ons un der which apositive probability distributio n can be rep

2、rese nted as aMarkov n etwork(also known as aMarkov ran dom field)t states that aprobability distribution that has apositive mass or density satisfies one of the Markov properties with respect to an undirected graph Gif and only if it is aGibbs random field,that is,its density can be factorized over

3、 the cliques(or complete subgraphs)of the graph.The relatio nship betwee n Markov and Gibbs ran dom fields was in itiated by Rola nd Dobrush in 1a nd Frank Spitzer2i n the con text of statistical mecha ni cs.The theorem is n amed after Joh n Hammersley and Peter Clifford who proved the equivale nee

4、in an un published paper in 1971.34Simpler proofs using the in clusi on-exclusi on prin ciple were give n in depe nden tly by Geoffrey Grimmett,5Presto n 6a nd Sherma n 7in 1973,with afurther proof by Julian Besag in 1974.8NotesADobrushin,P.L.(1968),The Description of aRandom Field by Means of Condi

5、tional Probabilities and Conditions of Its Regularity,Theory of Probability and its Applications 13(2) : 197 - 224,doi :10.1137/1113026,Spitzer,Frank(1971),Markov Random Fields and Gibbs Ensembles,The American Mathematical Monthly 78(2) : 142 - 154,doi :10.2307/2317621,JSTOR 2317621,Hammersley,J.M.

6、; Clifford,P.(1971),Markov fields on finite graphs and lattices,Clifford,P.(1990),Markov random fields in statistics,in Grimmett,G.R. ； Welsh,D.J.A.,Disorder inPhysical Systems : A Volume in Honour of John M.Hammersley,Oxford University Press,pp.19 - 32,ISBN 0198532156,MR 1064553,retrieved 2009-05-0

7、4AGrimmett,G.R.(1973),A theorem about random fields,Bulletin of the Lon don Mathematical Society 5(1) : 81 - 84,doi :10.1112/blms/5.1.81,MR 0329039APreston,C.J.(1973),Generalized Gibbs states and Markov ran dom fields,Adva nces in Applied Probability 5(2)1426035,Sherma n,S.(1973),Markov ran dom fiel

8、ds and Gibbs ran dom fields,lsrael Journal of Mathematics 14(1) : 92 - 103,doi : 10.1007/BF02761538,MR 0321185ABesag,J.(1974),Spatial in teraction and the statistical analysis of lattice systems,Journal of the RoyalStatistical Society.Series B(Methodological)36(2) : 192 - 236,MR0373208.JSTOR 2984812

9、 Further readi ng Bilmes,Jeff(Spri ng 2006),Ha ndout 2： Hammersley- Clifford,course notes from University of Washington course.Grimmett,Geoffrey,Probability on Gr aphs,Chapter 7,Helge,The Hammersley- Clifford Theorem and its Impact on Moder n Statistics,probability-related article is astub.You can h

10、elp Wikipedia by expa nding it.Retrieved from - Clifford_theoremFrom Wikipedia,the free en cyclopediaThe first after noon of the memorial sessi on for Julia n Besag in Bristol was an intense and at times emotional moment,where friends and colleagues of Julia n shared memories and stories.This collec

11、tio n of tributes showed how much of alarger-tha n-life character he was,from his Iong-termed and wide-ranged impact on statistics to his very high expectati on s,both for himself and for others,leadi ng to atotal and un compromis ing research ethics,to his passi on forextremesports and outdoors.(Th

12、e stories duri ng and after diner were of amore pers onal nature,but at least as much enjoyable ! )The talks on the second dayshowed how much and how deeply Julian had contributed to spatial statistics and agricultural experiments,to pseudo-likelihood,to Markov ran dom fields and image an alysis,a n

13、d to MCMC methodology and practice .I hope Idid not botch too much my prese ntati on on the history of MCMC,while Ifound reading through the 1974,1986 and 1993 Read Papers and their discussi ons an imme nsely rewardi ng experime nt(l wish Ihad done prior to completing our Statistical Science paper,b

14、ut it was bound to be in complete by n ature ! ).Some in terest ing links made by the audie nee werethe prior publicati on of proofs of the Hammersley-Clifford theorem in 1973(by Grimmet,Prest on,and Steward,respectively),as well as the proposal of aGibbs sampler by Bria n Ripley as early as 1977(ev

15、e n though Hasti ngs did use Gibbs steps in one of his examples).Christophe An drieu also poin ted out to me avery early Monte Carlo review by Joh n Halt on in the 1970 SIAM Rewiew,review that Iwill read(a nd commme nt)as soon as possible.Overall,I am quite glad Icould take part in this memorial and

16、 Iam grateful to both Peters for organising it as afitting tribute toJulia n.Markov Cha in Mon te Carlo(MCMC)methods are curre ntly avery active field of research.MCMC methods are sampli ng methods,based on MarkovChai ns which are ergodic with respect to the target probability measure.The principle

17、of adaptive methods is to optimize on the fly some desig n parameters of the algorithm with respect to agive n criteri on reflect ing the samplers performa nce(opti mize the accepta nee rate,optimize an importa nee sampli ng fun ctio n, etc ).A postdoctoralposition is opened to work on the numerical

18、 analysis of adaptive MCMC methods： con verge nce,nu merical efficie ncy,developme nt and an alysis of new algorithms.A particular emphasis will be give n to applicati ons in statistics and molecular dynamics.(Detailed description)Position funded by the French Natio nal Research Age ncy(ANR)through

19、the 2009-2012 project ANR-08-BLAN-0218.The position will benefit from an in terdiscipli nary environment involving numerical analysts,statisticians and probabilists,a nd of strong in teracti ons betwee n the part ners of the project ANR-08-BLAN-021 In the most recent issue of Statistical Science,the

20、 special topic isCelebrating the EM Algorithms Quandun ciace ntenni al .It contains an historical survey by Marti n Tanner and Wing Wong on the emerge nee of MCMC Bayesia n computati on in the 1980 s,This survey is more focused and more in formative tha n our global history(also to appear in Stati s

21、tical Science).In particular,it provides the authorsa nalysis as to why MCMC was delayed by ten years or so(or eve n more whe n con sideri ng that aGibbs sampler as asimulati on tool ap pears in both Hast in gs(1970)a nd Besags(1974)papers).They dismissourc oncerns about computi ng power(I was runni

22、ng Monte Carlo simulations on my Apple lie by 1986 and asingle mean square error curve evaluati on for aJames-Ste in type estimator would the n take close to aweekend! )and Markov innumeracy,rather attributing the reluctanee to alack of con fide nee into the method.This perspective rema ins debatabl

23、e as,apart from Tony OHaga n who was the n fighti ng aga in Monte Carlo methods as being un-Bayesian(1987,JRSS D),l do not remember any negative attitude at the time about simulation and the immediate spread of the MCMC methods from Ala n Gelfa nds and Adria n Smiths prese ntati ons of their 1990 pa

24、per shows on the opposite that the Bayesia n com mun ity was ready for the move.Another interesting point made in this historical survey is that Metropolisa nd other Markov cha in methods were first prese nted outside simulatio n sect ions of books like Hammersley and Handscomb(1964),Rubinstein( 198

25、1)and Ripley(1987),perpetuating the impressi on that such methods were mostly optimisatio n or ni che specific methods.This is also why Besags earlier works(not mentioned in this survey)did not get wider recognition,until later.Something Iwas notsampling(i.e.population Monte Carlo)n the Bayesian lit

26、erature of the 1980 s,with proposals from Herman va n Dijk,Adrian Smith,and others.The即 pe ndix about Smith et al.(1985),the 1987 special issue of JRSS D,a nd the computation contents of Valencia 3(that Isadly missed for being in the Army ! )is also quite in formative about the percepti on of comput

27、ational Bayesian statistics at this time.A missing connection in this survey is Gilles Celeux and Jean Diebolts stochastic EM(or SEM).As early as 1981,with Michel Bron iatowski,they proposed asimulated versi on of EM for mixtures where the late nt variable zwas simulated from its conditional distrib

28、ution rather than replaced with its expectation .So this was the first half of the Gibbs sampler for mixtures we completed with Jea n Diebolt about ten years later.(Also found in Gelma n and Kin g,1990.)These authors did not get much recog niti on from the com mun ity,though,as they focused almost e

29、xclusively on mixtures,used simulati on to produce arandomn ess that would escape the local mode attract ion ,rather tha n targeti ng the posterior distributi on,and did not an alyse the Markovia n n ature of their algorithm until later with the simulated annealing EM algorithm.Share: Share概率图模型分为有向

30、和无向的模型。有向的概率图模型主要包括贝叶斯网络和隐马 尔可夫模型，无向的概率图模型则主要包括马尔可夫随机场模型和条件随机场模 型。2001 年，卡耐基.梅隆大学的 Lafferty 教授(John Lafferty ，An drew McCallum， Ferna ndo Pereira) 等针对序列数据处理提出了 CRF模 型(Co nditio nal Ra ndom Fields Probabilistic Models for Segme nti ng and Labeli ng Seque nee Data) 。这种模型直接对后验概率建模，很好地解决了 MRF莫型利用多特征时需要复

31、杂的似然分布建模以及不能利用观察图像中上下文信息的问题。 Kumar博士在2003年将CRF模型扩展到2-维格型结构，开始将其引入到图像分析领域，吸引了学术界的高 度关注。对给定观察图像，估计对应的标记图像 y观察图像，x未知的标记图像1.如果直接对后验概率建模(即考虑公式中的第一项)，可以得到判别的 (Discriminative) 概率框架。特别地，如果后验概率直接通过 Gibbs分布建模，(x,y)称为一个CRF得到的模型称为判别的CRF模型。2.通过对(x,y)的联合建模 (即考虑公式中的第二项)，可以得到联合的概率框架？。特别地，如果考虑双随机 场(x,y)的马尔可夫性，即公式的第二

32、项为 Gibbs分布，那么(x,y)被称为一个双MRF(Pairwise MRF,PMRF)9。 3.后验概率通过公式所示的 p(x)和p(y|x)建模，其 中p(y|x)为生成观察图像的模型，因此这种框架称为生成的 (Generative)概率框架。特别地，如果先验p(x)服从Gibbs分布，x称为一个MRF12，得到的模型称 为生成的MRF莫型。-【面向图像标记的随机场模型研究】运用 Hammersley- Clifford 定理，标记场的后验概率服从 Gibbs分布其中，z(y, 9 )为归一化函数， c为定义在基团c上的带有参数0的势函数。 CRF模型中一个关键的问题是定义合适的势函数。因此发展不同形式的扩展 CRF模型是当前CRF模型的一个主要研究方向。具体的技 术途径包括：一是扩展势函数。通过引进更复杂的势函数，更多地利用多特征和上 下文信息；二是扩展模型结构。通过引入更复杂的模型结构，可以利用更高层次、 更多形式的上下文信息。扩展势函数(1)对数回归(Logistic Regression,LR) 支持向量机(Support Vector Machine,SVM)核函数(4)Boost(5)Probit扩展模型结构(1)动态CRF模型动态CRF(Dynamic