Pattern Recognition.docx

Pattern Recognition.docx

《Pattern Recognition.docx》由会员分享，可在线阅读，更多相关《Pattern Recognition.docx（35页珍藏版）》请在冰豆网上搜索。

PatternRecognition

PatternRecognition49（2016）102–114

YiminZhoua,n,1,GuolaiJianga,b,1,YaorongLinb

a

ShenzhenInstitutesofAdvancedTechnology,ChineseAcademyofSciences,China

bSchoolofElectronicandInformationEngineering,SouthChinaUniversityofTechnology,China

Articlehistory:

Received17March2014

Receivedinrevisedform

8August2014

Accepted29July2015

Availableonline8August2015

Thispaperpresentsahigh-levelhandfeatureextractionmethodforreal-timegesturerecognition.Firstly,thefingersaremodelledascylindricalobjectsduetotheirparalleledgefeature.Thenanovelalgorithmisproposedtodirectlyextractfingersfromsalienthandedges.Consideringthehandgeometricalcharacteristics,thehandpostureissegmentedanddescribedbasedonthefingerpositions,palmcenterlocationandwristposition.Aweightedradialprojectionalgorithmwiththeoriginatthe

article

info

abstract

Keywords:

Computervision

Fingermodelling

Salienthandedge

Convolutionoperator

Real-timehandgesturerecognition

wristpositionisappliedtolocalizeeachfinger.Thedevelopedsystemcannotonlyextractextensionalfingersbutalsoflexionalfingerswithhighaccuracy.Furthermore,handrotationandfingeranglevariationhavenoeffectonthealgorithmperformance.Theorientationofthegesturecanbecalculatedwithouttheaidofarmdirectionanditwouldnotbedisturbedbythebarearmarea.Experimentshavebeenperformedtodemonstratethattheproposedmethodcandirectlyextracthigh-levelhandfeatureandestimatehandposesinreal-time.

&2015ElsevierLtd.Allrightsreserved.

1.Introduction

Handgesturerecognitionbasedoncomputervisiontechnologyhasbeenreceivedgreatinterestsrecently,duetoitsnaturalhuman-computerinteractioncharacteristics.Handgesturesaregenerallycomposedofdifferenthandposturesandtheirmotions.However,humanhandisanarticulatedobjectwithover20degreesoffreedom（DOF）[12],andmanyself-occlusionswouldoccurinitsprojectionresults.Moreover,handmotionisoftentoofastandcomplicatedcomparedwithcurrentcomputerimageprocessingspeed.Therefore,real-timehandpostureestimationisstillachallengingresearchtopicwithmulti-disciplinaryworkincludingpatternrecognition,imageprocessing,computervision,artificialintelligenceandmachinelearning.

Inhuman–machineinteractionhistory,keyboardinput&charactertextoutputandmouseinput&graphicwindowdisplayaremaintraditionalinteractionforms.Withthedevelopmentofcomputertechniques,thehuman–machineinteractionviahandpostureplaysanimportantroleunderthreedimensionalvirtualenvironment.Manymethodshavebeendevelopedforhandposerecognition[3,4,10,18,24,29].

AgeneralframeworkforvisualbasedhandgesturerecognitionisillustratedinFig.1.Firstly,thehandislocatedandsegmentedfromtheinputimage,whichcanbeachievedviaskin-colorbasedsegmentationmethods[27,31]ordirectobjectrecognitionalgorithms.Thesecondstepistoextractusefulfeatureforstatichandpostureandmotionidentification.Thenthegesturecanbeidentifiedviafeaturematching.Finally,differenthumanmachineinteractioncanbeappliedbasedonthesuccessfulhandgesturerecognition.

Therearealotofconstraintsanddifficultiesinaccuratehandgesturerecognitionfromimagessincehumanhandisanobjectwithcomplexandversatileshapes[25].Firstly,differentfromlessremarkablemetamorphosisobjectssuchashumanface,humanhandpossessesover20freedegreeplusvariationsinhandgesturelocationandrotationwhichmakehandpostureestimationextremelydifficult.Evidenceshowsthatatleast6-dimensioninformationisrequiredforbasichandgestureestimation.Theocclusionalsocouldincreasethedifficultyinposerecognition.Sincetheinvolvedhandgestureimagesareusuallytwodimensionedimages,itwouldresultinocclusionofsomekeypartsofthehandontheplaneprojectduetovariousheightsofthehandshapes.

Besides,theimpactofthecomplexenvironmenttothebroadlyappliedvisual-basedhandgesturerecognitiontechniqueshasto

nCorrespondingauthor.

E-mailaddresses:

ym.zhou@（Y.Zhou）,gl.jiang@（G.Jiang）.

1firstauthorandsecondauthorcontributeequallyinthepaper.

The

beconsidered.Thelightnessvariationandcomplexbackgroundsuchfactorsmakeitmoredifficultforthehandgesturesegmentation.Uptonow,thereisnouniteddefinitionfordynamichand

http:

//dx.doi.org/10.1016/j.patcog.2015.07.0140031-3203/&2015ElsevierLtd.Allrightsreserved.

Fig.2.Handgesturemodelswithdifferentcomplexities（a）3Dstripmodel;（b）3Dsurfacemodel;（c）papermodel[36];（d）gesturesilhouette;and（e）gesturecontour.

gesturerecognition,whichisalsoanunsolvedproblemtoaccommodatehumanhabitsandfacilitatecomputerrecognition.Itshouldbenotedthathumanhandhasdeformableshapeinfrontofacameraduetoitsowncharacteristics.Theextractionofahandimagehastobeexecutedinreal-timeindependentoftheusersanddevice.Humanmotionpossessesafastspeedupto5m/sfortranslationand3001C/sforrotation.Thesamplingfrequencyofadigitalcameraisabout30–60Hz,whichcouldresultinfuzzificationonthecollectedimageswithnegativeimpactonfurtheridentification.Ontheotherhand,withthehandgesturemoduleaddedinthesystem,thedealtframenumberpersecondforthecomputerwillbeevenless,whichwillbringmoreseriouspressureontherelativelylowersamplingspeed.Moreover,alargeamountofdatahavetobedealtincomputervisualsystem,especiallyforhighcomplexversatileobjects.Undercurrentcomputerhardwareconditions,alotofhigh-precisionrecognitionalgorithmsaredifficulttobeoperatedinreal-time.

Ourdevelopedalgorithmfocusesonsinglecamerabasedrealtimehandgesturerecognition.Someassumptionsaremadewithoutlossofgenerality:

（a）thebackgroundisnottoocomplexwithoutlargeareaskincolordisturbance;（b）lightnessshouldavoidtoolowortoolightsuchworseconditions;（c）thepalmisrightfacedtothecamerawithdistanceintheranger0:

5m.Thesethreelimitationsarenotdifficulttoberealizedintheactualapplicationscenarios.

Firstly,anewfingerdetectionalgorithmisproposed.Comparedtopreviousfingerdetectionalgorithms,thedevelopedalgorithmisindependentofthefingertipfeaturebutcanextractfingersdirectlyfromthemainedgeofthewholefingers.Consideringthateachfingerhastwomain“parallel”edges,afingerisdeterminedfromconvolutionresultofsalienthandedgeimagewithaspecificoperatorG.Thealgorithmcannotonlyextractextensionalfingersbutalsoflexionalfingerswithhighaccuracy,whichisthebasisforcompletehandposehigh-levelfeatureextraction.Afterthefingercentralareahasbeenobtained,thecenter,orientationofthehandgesturecanbecalculated.Duringtheprocedure,anovelhigh-levelgesturefeatureextractionalgorithmisdeveloped.Throughweightedradiusprojectionalgorithm,thegesturefeaturesequencecanbeextractedandthefingerscanalsobelocalizedfromthelocalmaximaofangularprojection,thusthegesturecanbeestimateddirectlyinreal-time.

Theremainderofthepaperisorganizedasfollows.Section2describeshandgesturerecognitionprocedureandgenerallyusedmethods.FingerextractionalgorithmbasedonparalleledgecharacteristicsisintroducedinSection3.Salienthandimagecanalsobeachieved.ThespecificoperatorGandthresholdisexplainedindetailinSection4.High-levelhandfeatureextractionthroughconvolutionisdemonstratedinSection5.ExperimentsindifferentscenariosareperformedtoprovetheeffectivenessoftheproposedalgorithminSection6.ConclusionsandfutureworksaregiveninSection7.

2.Methodsofhandgesturerecognitionbasedoncomputer

vision

2.1.Handmodelling

Handposturemodellingplaysakeyroleinthewholehandgesturerecognitionsystem.Theselectionofthehandmodelisdependentontheactualapplicationenvironments.Thehandmodelcanbecategorizedasgestureappearancemodellingand3Dmodelling.GenerallyusedhandgesturemodelsaredemonstratedinFig.2.

3Dhandgesturemodelconsidersthegeometricalstructurewithhistogramorhyperquadricsurfacetoapproximatefingerjointsandpalm.Themodelparameterscanbeestimatedfromsingleimageorseveralimages.However,the3Dmodelbasedgesturemodellinghasquiteahighcalculationcomplexity,andtoomanylinearizationandapproximationwouldcauseunreliableparameterestimation.Asforappearancebasedgesturemodels,theyarebuiltthroughappearancecharacteristics,whichhavetheadvantagesoflesscomputationloadandfastprocessingspeed.Theadoptionofthesilhouette,contourmodelandpapermodelcanonlyreflectpartialhandgesturecharacteristics.Inthispaper,basedonthesimplifiedpapergesturemodel[36],anewgesturemodelisproposedwhereeachfingerisrepresentedbyextensionandflexionstatesconsideringgesturecompletenessandreal-timerecognitionrequirements.

Manyhandposerecognitionmethodsuseskincolor-baseddetectionandtakegeometricalfeaturesforhandmodelling.Handposeestimationfrom2Dto3Dusingmulti-viewpointsilhouetteimagesisdescribedin[35].Inrecentyears,3Dsensors,suchasbinocularcameras,Kinectandleapmotion,havebeenappliedforhandgesturerecognitionwithgoodperformance[5].However,handgesturerecognitionhasquitealimitation,since3Dsensorsarenotalwaysavailableinmanysystems,i.e.,GoogleGlasses.

2.2.Descriptionofhandgesturefeature

Thefeatureextraction