Hand Gesture Spotting and Recognition Based on Generative and Discriminative Models in Stereo Color Image Sequences
Fatma Shabaan Abd Elbary Ebrahim;
Abstract
The main objective of this thesis is to study the Human Computer Interaction (HCI). Mainly, we shall study the system of gesture spotting and recognition by using hand. This faced by several problems: Extracting (spotting) key gesture from the continuous hand motions sequence is the first problem which arises in the recognition of hand gesture. The second problem is that there is quite a bit of variability in trajectory, shape, and duration of the same gesture even for the same person.
Throughout literature, to discover the start points of gestures they firstly detect the end points of gestures and then track back through their optimal paths, this technique is called the backward spotting. The recognizer receives the point’s trajectory between the detection of start and end points for classification. Thus, there is time delay among the spotting process of key (meaningful) gestures and the recognition process. It is being noted that the time delay is not acceptable to online implementation and applications.
In this work, we are handled meaningful gesture spotting and recognition according to two different classification techniques: generative model such as Hidden Markov Models (HMMs) with coupling of Support Vector Machine (SVM) and neural network (NNW) separately as well as discriminative models like Conditional Random Fields (CRFs)in conjunction with SVM and NNW to specify the superior in terms of recognition outcomes.
For designing the model of non-gesture a stochastic method is proposed to spot meaningful gestures accurately with HMMs in conjunction with a SVM and NNW versus CRFs in conjunction with a SVM and NNW with no training data. For HMM, the non-gesture model is used to select the starting and ending points of key gestures as an adaptive threshold which gives a trust measure which is embedded in input videos. Firstly, the hand's segmentation is done via YCbCr color space and 3D depth map. We captured depth map by kind of stereo camera (bumblebee) where complex background sense is neutralized. The depth map which based on passive stereo measuring has been obtained using calibration data of the camera in addition to mean absolute difference. To track a hand motion a set of hand postures is extracted by mean-shift technique coupling of depth map to correctly achieve accurate and robust hand trajectory. Secondly, in addition to the extracted features of three angles we derived also the features of Zernike moments and elliptic Fourier of dynamic affine-invariants from 3Dspatio-temporal hand volume. Finally, the generative model of Hidden Markov Model and discriminative models of Conditional Random Fields perform the processes of spotting as well as recognition using the set of angles features. In addition, we employed the technique of forward spotting in coupling with circular buffer mechanism which carries
Throughout literature, to discover the start points of gestures they firstly detect the end points of gestures and then track back through their optimal paths, this technique is called the backward spotting. The recognizer receives the point’s trajectory between the detection of start and end points for classification. Thus, there is time delay among the spotting process of key (meaningful) gestures and the recognition process. It is being noted that the time delay is not acceptable to online implementation and applications.
In this work, we are handled meaningful gesture spotting and recognition according to two different classification techniques: generative model such as Hidden Markov Models (HMMs) with coupling of Support Vector Machine (SVM) and neural network (NNW) separately as well as discriminative models like Conditional Random Fields (CRFs)in conjunction with SVM and NNW to specify the superior in terms of recognition outcomes.
For designing the model of non-gesture a stochastic method is proposed to spot meaningful gestures accurately with HMMs in conjunction with a SVM and NNW versus CRFs in conjunction with a SVM and NNW with no training data. For HMM, the non-gesture model is used to select the starting and ending points of key gestures as an adaptive threshold which gives a trust measure which is embedded in input videos. Firstly, the hand's segmentation is done via YCbCr color space and 3D depth map. We captured depth map by kind of stereo camera (bumblebee) where complex background sense is neutralized. The depth map which based on passive stereo measuring has been obtained using calibration data of the camera in addition to mean absolute difference. To track a hand motion a set of hand postures is extracted by mean-shift technique coupling of depth map to correctly achieve accurate and robust hand trajectory. Secondly, in addition to the extracted features of three angles we derived also the features of Zernike moments and elliptic Fourier of dynamic affine-invariants from 3Dspatio-temporal hand volume. Finally, the generative model of Hidden Markov Model and discriminative models of Conditional Random Fields perform the processes of spotting as well as recognition using the set of angles features. In addition, we employed the technique of forward spotting in coupling with circular buffer mechanism which carries
Other data
| Title | Hand Gesture Spotting and Recognition Based on Generative and Discriminative Models in Stereo Color Image Sequences | Other Titles | اكتشاف حركه اليد والتعرف عليها باستخدام النماذج التوليديه والتميزيه في سلسله من الصور الملونة | Authors | Fatma Shabaan Abd Elbary Ebrahim | Issue Date | 2017 |
Recommend this item
Similar Items from Core Recommender Database
Items in Ain Shams Scholar are protected by copyright, with all rights reserved, unless otherwise indicated.