Pattern Recognition.docx

1、Pattern RecognitionPattern Recognition 49 (2016) 102114Yimin Zhou a,n,1, Guolai Jiang a,b,1, Yaorong Lin baShenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Chinab School of Electronic and Information Engineering, South China University of Technology, ChinaArticle history:Rece

2、ived 17 March 2014Received in revised form8 August 2014Accepted 29 July 2015Available online 8 August 2015This paper presents a high-level hand feature extraction method for real-time gesture recognition. Firstly, the fingers are modelled as cylindrical objects due to their parallel edge feature. Th

3、en a novel algorithm is proposed to directly extract fingers from salient hand edges. Considering the hand geometrical characteristics, the hand posture is segmented and described based on the finger positions, palm center location and wrist position. A weighted radial projection algorithm with the

4、origin at thea r t i c l ei n f oa b s t r a c t Keywords:Computer visionFinger modellingSalient hand edgeConvolution operatorReal-time hand gesture recognitionwrist position is applied to localize each finger. The developed system can not only extract extensional fingers but also flexional fingers

5、with high accuracy. Furthermore, hand rotation and finger angle variation have no effect on the algorithm performance. The orientation of the gesture can be calculated without the aid of arm direction and it would not be disturbed by the bare arm area. Experiments have been performed to demonstrate

6、that the proposed method can directly extract high-level hand feature and estimate hand poses in real-time.& 2015 Elsevier Ltd. All rights reserved.1. IntroductionHand gesture recognition based on computer vision technology has been received great interests recently, due to its natural human-compute

7、r interaction characteristics. Hand gestures are generally composed of different hand postures and their motions. However, human hand is an articulated object with over 20 degrees of freedom (DOF) 12, and many self-occlusions would occur in its projection results. Moreover, hand motion is often too

8、fast and complicated compared with current computer image processing speed. Therefore, real-time hand posture estimation is still a challenging research topic with multi-disciplinary work including pattern recognition, image processing, computer vision, artificial intelligence and machine learning.I

9、n humanmachine interaction history, keyboard input & character text output and mouse input & graphic window display are main traditional interaction forms. With the development of computer techniques, the humanmachine interaction via hand posture plays an important role under three dimensional virtu

10、al environment. Many methods have been developed for hand pose recognition 3,4,10,18,24,29.A general framework for visual based hand gesture recognition is illustrated in Fig. 1. Firstly, the hand is located and segmented from the input image, which can be achieved via skin-color based segmentation

11、methods 27,31 or direct object recognition algorithms. The second step is to extract useful feature for static hand posture and motion identification. Then the gesture can be identified via feature matching. Finally, different human machine interaction can be applied based on the successful hand ges

12、ture recognition.There are a lot of constraints and difficulties in accurate hand gesture recognition from images since human hand is an object with complex and versatile shapes 25. Firstly, different from less remarkable metamorphosis objects such as human face, human hand possesses over 20 free de

13、gree plus variations in hand gesture location and rotation which make hand posture estimation extremely difficult. Evidence shows that at least 6-dimension information is required for basic hand gesture estimation. The occlusion also could increase the difficulty in pose recognition. Since the invol

14、ved hand gesture images are usually two dimensioned images, it would result in occlusion of some key parts of the hand on the plane project due to various heights of the hand shapes.Besides, the impact of the complex environment to the broadly applied visual-based hand gesture recognition techniques

15、 has ton Corresponding author.E-mail addresses: ym.zhou (Y. Zhou), gl.jiang (G. Jiang).1 first author and second author contribute equally in the paper.Thebe considered. The lightness variation and complex background such factors make it more difficult for the hand gesture segmentation. Up to now, t

16、here is no united definition for dynamic handhttp:/dx.doi.org/10.1016/j.patcog.2015.07.014 0031-3203/& 2015 Elsevier Ltd. All rights reserved.Fig. 2. Hand gesture models with different complexities (a) 3D strip model; (b) 3 D surface model; (c) paper model 36; (d) gesture silhouette; and (e) gesture

17、 contour.gesture recognition, which is also an unsolved problem to accommodate human habits and facilitate computer recognition. It should be noted that human hand has deformable shape in front of a camera due to its own characteristics. The extraction of a hand image has to be executed in real-time

18、 independent of the users and device. Human motion possesses a fast speed up to 5 m/s for translation and 300 1C/s for rotation. The sampling frequency of a digital camera is about 3060 Hz, which could result in fuzzification on the collected images with negative impact on further identification. On

19、 the other hand, with the hand gesture module added in the system, the dealt frame number per second for the computer will be even less, which will bring more serious pressure on the relatively lower sampling speed. Moreover, a large amount of data have to be dealt in computer visual system, especia

20、lly for high complex versatile objects. Under current computer hardware conditions, a lot of high-precision recognition algorithms are difficult to be operated in real-time.Our developed algorithm focuses on single camera based realtime hand gesture recognition. Some assumptions are made without los

21、s of generality: (a) the background is not too complex without large area skin color disturbance; (b) lightness should avoid too low or too light such worse conditions; (c) the palm is right faced to the camera with distance in the range r0:5 m. These three limitations are not difficult to be realiz

22、ed in the actual application scenarios.Firstly, a new finger detection algorithm is proposed. Compared to previous finger detection algorithms, the developed algorithm is independent of the finger tip feature but can extract fingers directly from the main edge of the whole fingers. Considering that

23、each finger has two main “parallel” edges, a finger is determined from convolution result of salient hand edge image with a specific operator G. The algorithm can not only extract extensional fingers but also flexional fingers with high accuracy, which is the basis for complete hand pose high-level

24、feature extraction. After the finger central area has been obtained, the center, orientation of the hand gesture can be calculated. During the procedure, a novel high-level gesture feature extraction algorithm is developed. Through weighted radius projection algorithm, the gesture feature sequence c

25、an be extracted and the fingers can also be localized from the local maxima of angular projection, thus the gesture can be estimated directly in real-time.The remainder of the paper is organized as follows. Section 2 describes hand gesture recognition procedure and generally used methods. Finger ext

26、raction algorithm based on parallel edge characteristics is introduced in Section 3. Salient hand image can also be achieved. The specific operator G and threshold is explained in detail in Section 4. High-level hand feature extraction through convolution is demonstrated in Section 5. Experiments in

27、 different scenarios are performed to prove the effectiveness of the proposed algorithm in Section 6. Conclusions and future works are given in Section 7.2. Methods of hand gesture recognition based on computervision2.1. Hand modellingHand posture modelling plays a key role in the whole hand gesture

28、 recognition system. The selection of the hand model is dependent on the actual application environments. The hand model can be categorized as gesture appearance modelling and 3D modelling. Generally used hand gesture models are demonstrated in Fig. 2.3D hand gesture model considers the geometrical

29、structure with histogram or hyperquadric surface to approximate finger joints and palm. The model parameters can be estimated from single image or several images. However, the 3D model based gesture modelling has quite a high calculation complexity, and too many linearization and approximation would

30、 cause unreliable parameter estimation. As for appearance based gesture models, they are built through appearance characteristics, which have the advantages of less computation load and fast processing speed. The adoption of the silhouette, contour model and paper model can only reflect partial hand

31、 gesture characteristics. In this paper, based on the simplified paper gesture model 36, a new gesture model is proposed where each finger is represented by extension and flexion states considering gesture completeness and real-time recognition requirements.Many hand pose recognition methods use ski

32、n color-based detection and take geometrical features for hand modelling. Hand pose estimation from 2D to 3D using multi-viewpoint silhouette images is described in 35. In recent years, 3D sensors, such as binocular cameras, Kinect and leap motion, have been applied for hand gesture recognition with good performance 5. However, hand gesture recognition has quite a limitation, since 3D sensors are not always available in many systems, i.e., Google Glasses.2.2. Description of hand gesture featureThe feature extraction