JointMAPregistrationandhighresolutionimageestimationusingasequenceofundersample.docx

JointMAPregistrationandhighresolutionimageestimationusingasequenceofundersample.docx

《JointMAPregistrationandhighresolutionimageestimationusingasequenceofundersample.docx》由会员分享，可在线阅读，更多相关《JointMAPregistrationandhighresolutionimageestimationusingasequenceofundersample.docx（81页珍藏版）》请在冰豆网上搜索。

JointMAPregistrationandhighresolutionimageestimationusingasequenceofundersample

JointMAPRegistrationandHighResolutionImage

EstimationUsingaSequenceof

UndersampledImages1

RussellC.Hardie†,KennethJ.Barnard‡andErnestE.Armstrong‡

†DepartmentofElectricalandComputerEngineering

UniversityofDayton

300CollegeParkAvenue

Dayton,OH45469

Phone:

（513）229-3178

Fax:

（513）229-2471

Email:

rhardie@engr.udayton.edu

‡SensorsTechnologyBranch

WrightLaboratoryWL/AAJT

Building622

3109PStreet

WPAFBOH45433-7700

Phone:

（513）255-9609

Fax:

（513）255-6489

Abstract

Inmanyimagingsystems,thedetectorarrayisnotsufficientlydensetoadequatelysample

thescenewiththedesiredfieldofview.Thisisparticularlytrueformanyinfraredfocalplane

arrays.Thus,theresultingimagesmaybeseverelyaliased.Thispaperexaminesatechnique

forestimatingahighresolutionimage,withreducedaliasing,fromasequenceofundersampled

frames.Severalapproachestothisproblemhavebeeninvestigatedpreviously.However,in

thispaperamaximumaposteriori（MAP）frameworkforjointlyestimatingimageregistration

parametersandthehighresolutionimageispresented.Severalpreviousapproacheshaverelied

onknowingtheregistrationparametersaprioriorhaveutilizedregistrationtechniquesnot

specificallydesignedtotreatseverelyaliasedimages.Intheproposedmethod,theregistration

parametersareiterativelyupdatedalongwiththehighresolutionimageinacycliccoordinate-

descentoptimizationprocedure.Experimentalresultsareprovidedtoillustratetheperformance

oftheproposedMAPalgorithmusingbothvisibleandinfraredimages.Quantitativeerror

analysisisprovidedandseveralimagesareshownforsubjectiveevaluation.

EDICSNumber:

IP1.12（ImageSequenceProcessing）

Permissiontopublishthisabstractseparatelyisgranted.

SubmittedtotheIEEETransactionsonImageProcessing,Feb.1996.Revisedmanuscript

submittedDec.1996.

1

ThisworkhasbeensupportedinpartunderAirForcecontractF33601-95-DJ010.

ListofFigures

1

Discretedetectormodelshowingthosehighresolutionpixels（ontheleft）thatcon-

tributetoalowresolutionpixel（ontheright）.Theimagezrepresentsthetrue

underlinghighresolutionimagewewishtoestimateandykisthek’thobservedlow

resolutionframe.Notethedifferentgridsizesforzandyk...............

5

2

Thehighresolutionimagepriorneighborhoodmodelshowingthecardinalneighbors

ofapixelzi.Inthiscase,di,jwouldbenon-zeroonlyforjsuchthatzjisanimmediate

spatialneighborofzi（thoseshadedpixels）........................

8

3

Convolutionkernelusedtoobtaintheimagepriorgradient.

.............

12

4

5

6

Simulatedcamerashiftsintermsofhighresolutionpixelsfor16frames........

（a）Originalimage“Aerial”（b）simulatedlowresolutionframe1（L1=L2=4,

ση2=100）（c）MAPestimatewithλ=150（d）MAPestimatewithλ=∞（e）

bilinearinterpolationofframe1（f）bicubicinterpolationofframe1..........

LearningcurveshowingtheMAPcostfunction,L（z,s）,versusiterationnumberfor

14

15

thesimulateddata.Here16framesareusedwithL1=L2=4,ση2=100,andλ=150.18

7

Meanabsoluteerrorfortheestimatorswith（a）varyinglevelsofnoiseusing16frames

and（b）differentnumberofframeswithnoisevarianceση2=100............

19

8

TheoreticalimagingsystemPSFonthehighresolutiongridwhereL1=L2=5.The

PSFisbasedontheassumptionofdiffraction-limitedopticsandincludestheeffects

ofthefinitedetectorsize..................................

21

9

（a）Lowresolutionframe1showingvehiclesimagedfromatower（b）MAPestimate

using16frameswithL1=L2=5andλ=200（c）MAPestimatewithλ=∞（d）

bilinearinterpolationofframe1（e）bicubicinterpolationofframe1..........

22

10

LearningcurveshowingtheMAPcostfunctionfortheinfrareddataasafunctionof

iterationnumberwhereL1=L2=5andλ=200....................

25

11

Estimatedinfraredimagershiftsintermsofhighresolutionpixelsfor16frames...

26

ListofTables

1

ProposedIterativeMAPEstimationAlgorithm......................

i

13

1

Introduction

Ifadetectorarrayusedforimageacquisitionisnotsufficientlydense,soastomeettheNyquist

criterion,theresultingimageswillbedegradedbyaliasingeffects.Sincetheopticsoftheimaging

systemwillservetoeffectivelybandlimittheimageonthedetectorarray,itispossibletoacquire

animagewhichisfreeofaliasing.However,thisrequirestheappropriatecombinationofopticsand

detectorarray.Generallyabroadinstantaneousfieldofviewisdesiredwhichrequiresopticswith

ashortfocallength.Topreventaliasinginthiscaserequiresadensedetectorarraywhichmaybe

verycostlyorsimplyunavailable.Thus,manyimagingsystemsaredesignedtoallowsomelevel

ofaliasingduringimageacquisition.Thisisparticularlytrueforstaringinfraredimagersbecause

offabricationcomplexities.Somevisualchargecoupleddevice（CCD）camerasalsosufferfrom

undersampling.Thegoalofthisworkistodevelopatechniqueforestimatinganunaliasedhigh

resolutionimagefromthealiasedimagesacquiredfromsuchanimagingsystem.Wealsowishto

combatadditivenoiseandblurringduetothefinitedetectorsizeandoptics.

Heretheproblemisapproachedfromtheframeworkofimagesequenceprocessing[1].Thus,the

highresolutionimagewillbeestimatedfromasequenceoflow-resolutionaliasedimages.Thisis

possibleifthereexistssub-pixelmotionbetweentheacquiredframes.Thus,aunique“look”atthe

sceneisprovidedbyeachframe.Inparticular,weconsiderthescenariowhereanimagerismounted

onamovingorvibratingplatform,suchasanaircraft,andisimagingobjectsinthefarfield.Thus,

theline-of-sightjitterwillgenerallyprovidethenecessarymotionbetweenthefocalplanearray

andthesceneateachacquisitiontimewithminimalocclusioneffects.Thisprocessisreferredtoas

uncontrolledmicroscanning[2,3].Thekeytoexploitingthesemultipleframesisaccurateknowledge

ofthesub-pixelregistrationparametersforeachframe.Iftheimagesareseverelyundersampled,we

havefoundthattraditionalmotionestimationtechniques,suchasblockmatching,maynotprovide

thedesiredsub-pixelaccuracy.Thishasmotivatedthedevelopmentoftheapproachpresentedhere.

Thisbasicproblemofhighresolutionimagerecoveryusingmultipleframeswasfirstaddressed

intheliteraturebyTsaiandHuang[4].Theirobservationmodelisbasedontheshiftproperty

oftheFouriertransform.Eachaliasedobservationprovidesasetofequationsinthefrequency

domain.Providedthatenoughframesareavailable,theunaliaseddiscretespectrum,andhence

unaliasedimage,canbesolvedfor.However,onemustknowtheshiftsinordertosolveforthehigh

resolutionimageinthisfashion.Amethodforestimatingtheshiftsusingthemultiplealiasedframes

isproposedin[4]forthecasewheretheimagesarebandlimited.Whilethisisaninsightfulsolution,

itmaybeimpracticalinmanyapplicationsbecauseofprohibitivelyhighcomputationalcomplexity.

Furthermore,itrequiresasetminimumnumberofframestooperate,whichmaynotbeavailable.

Theimagerecoveryalgorithmin[4]isextendedin[5]forthecasewherenoiseisconsideredbyusing

1

arecursiveleastsquaressolutionforthesetoffrequencydomainlinearequations.Thisisextended

againforthecasewhereblurringinconsideredin[6].However,theestimationoftheglobalframe

shiftsisnotaddressedin[5]or[6].Atechniquewhichusestheleastsquaressolution,similarto

thatin[5],alongwithafastsuboptimalschemeforestimatingtheglobalframeshiftsisdescribed

in[7].

Anotherapproachtothehighresolutionimagereconstructionproblemusesaprojectiononto

convexsets（POCS）algorithm[8].ThePOCSapproachhasbeenextendedtotreatmotionblur

andnoisein[1,9,10].Blockmatchingorphasecorrelationissuggestedin[9,10]asameansof

estimatingtherequiredmotionparameters.Arelatedmultiframetechniquewhichconsidersglobal

translationalshiftandrotationispresentedin[11].In[12],thistechniqueisextendedtotreatthe

moregeneralcaseofaperspectiveprojectionofaplane.Allofthesemethodsrelyonatwostage

estimationprocedurewherebytheregistrationisdoneindependentlyofthehighresolutionimage

reconstruction.

Theproblemhasalsobeenapproachedfromastatisticalestimationframework.Specifically,a

maximumaposteriori（MAP）estimatorisdevelopedin[13,14]whichisanextensionofasingle

frameimageexpansionalgorithmproposedin[15].TheMAPestimatorin[13,14]usesanedge

preservingHuber-Markovrandomfieldfortheimageprior.Themotionisestimatedbyablock

matchingalgorithmappliedtotheindividualframes.Theseindividualframesarefirstexpanded

usingthesingleframealgorithmin[15]toallowforsub-pixelestimates.Thisprovidesauseful

solutionanditcantreatthecasewheretheimagemotionisnotglobal,whichiscriticalifthe

scenecontainsmovingobjects.Theblockmatchingtechnique,however,doesnotexploitapriori

informationaboutthemotionifanyisavailable.Italsodoesnotexploitalltheobservedframes

whenestimatingthemotionparametersforeachimage.AnotherrelatedmultiframeMAPtechnique,

whichusesanapproachsimilartothatdescribedhe