Developing a Multimodal Biometric System Based on Single Camera

Abstract

Multimodality is often seen as a way to solve some of the problems raised by the use of single biometric. Indeed, combining two or more modalities can significantly improve the overall accuracy, address the non-universality problem, resist to spoof attacks and help searching a large database in a computationally efficient manner. However, multimodal systems may require using many sensors and additional time for user enrollment especially when using contact-based sensors (e.g. fingerprint) or intrusive sensors (e.g. iris). This makes them more expensive and less convenient to the user. This thesis aims to fuse two hand modalities, palmprint and inner-knuckle-prints (IKPs) of fingers. These modalities can be extracted from the hand using a traditional camera in contactless manner. Using a single camera can facilitate the enrollment process and make it more convenient for users. In addition, due to the low cost, compact size and the widespread availability of such cameras in most laptops and mobiles, it allows the system to be used by large number of users in wide-range of applications (e.g. personal accounts over the web, e-voting, electronic identity in the e-government applications). Moreover, being contactless help addressing the hygiene issue, avoid the possibility of copying the latent hand prints for illegitimate use and avoid the problem of contaminating the device surface in harsh environments. However, many challenges arise from using contactless camera in unconstrained and guideless environment. First, geometric transformations, existence of hand wrist, connected fingers, finger rings and other hand accessories can highly affect the extraction of palm and knuckles region-of-interests (ROIs). Second, pose and illumination variations can affect the layout and visibility of palm and knuckle lines. In addition, different ways of holding out the hand can lead to extra wrinkles and false line-like features. These can affect the performance of some contact-based methods and raise the need to develop special algorithms to cope with contactless conditions. This thesis proposes a complete multibiometric system to cope with the above challenges. The main contributions can be summarized as follows; first, propose a simple method for checking the hand existence and extracting the palm and knuckles ROIs from unconstrained contactless environment. The method is based on blob analysis, morphological and geometrical operations without a need to pre-train or parameter adjustment. The method copes with hand scale, rotation, connected fingers and the existence of finger rings, hand wrist and other false objects. It's tested on four publicity available inner hand databases (DBs) that cover these challenges; namely, Sfax-Miracl, IITD, PolyU 3D/2D and HGC. Based on the proposed evaluation methodology, the method correctly extracts the palm ROI in more than 99% and the knuckles ROIs in more than 97.8% of each DB. For the hand existence checking, the method correctly identifies the absence of the hand in 99% of non-hand images from four skin-based DBs. Second contribution; propose a new SIFT-based method for palmprint and inner-knuckle-print (IKP) recognition with three main modifications from the traditional SIFT. First, the regions with no significant lines/wrinkles are masked out to reduce the false features. A region with multi lines are then described by multi descriptors rather than a single one. Second, instead of matching all query keypoints with all target ones, only those with small rotation difference are matched together. This speedup the matching process and enhance the accuracy by reducing the wrong matches. Third, an align-based refinement is applied to filter out the incorrect matches. The method is tested on three contactless hand DBs; namely, IITD, PolyU 2D/3D and Sfax-Miracl. These DBs cover variations in image resolution, sensor type and capture