Multibiometric Systems
Pushpa Dhamala
Master of Telematics - Communication Networks and Networked Supervisor: Danilo Gligoroski, ITEM
Co-supervisor: Yanling Chen, ITEM
Department of Telematics Submission date: June 2012
Norwegian University of Science and Technology
i
Thesis Description
Reliable and efficient personal recognition is a critical concern in today’s widely interconnected society. As a newly emerging technique, biometric recognition systems are being increasingly used by government, business and forensic applications. In this thesis, multibiometric systems are of interest due to their advantages in improving the matching accuracy, increasing population coverage, detering spoofing attacks and imparting fault tolerance to biometric applications.
Multibiometric systems are biometric systems that consolidate multiple sources of biometric evidences. The integration of evidences is known as fusion. In biometrics, various levels of fusion can be categorized into two broad categories: preclassification (fusion before matching) and postclassification (fusion after matching). In this thesis, a survey of different levels of fusion will be conducted. In particular, fusion at match score level will be examined in detail, since it is the dominant level of fusion in biometric systems.
In a multibiometric system, multiple sources of biometric information are used. Various sources that can be fused will be studied in this thesis. Besides, depending on the nature of these sources, multibiometric systems can be classified into different categories, for instance, multi-sensor systems, multi-algorithm systems, multi-instance systems, multi-sample systems, multimodal systems and hybrid systems. In this thesis, an overview of these systems will be provided. Special attention will be devoted to multimodal systems since multimodal systems consolidate the evidence presented by different body traits and the use of multiple body traits improves the identification accuracy significantly.
Assignment Given: February 2012 Professor: Danilo Gligoroski Supervisor: Yanling Chen
ii
Abstract
Reliable user authentication has become very important with rapid advancements in networking and mobility coupled with increased concerns about security. Biometric systems perform recognition based on specific physiological or behavioral characteristics(s) possessed by a user. Biometrics establishes identity based on who you are rather than what you possess (e.g, tokens) or what you remember (e.g, passwords). Biometric systems have now been deployed in various commercial, civilian, and forensic applications for reliable individual recognition. Unibiometric systems rely on the evidence of a single source of information whereas multibiometric systems consolidate multiple sources of biometric evidences.
Multibiometric systems, if designed properly, are able to enhance the matching performance.
Moreover, they are expected to increase population coverage, deter spoofing attacks and provide fault tolerance to biometric systems. In this thesis, we perform a survey of various categories multibiometric systems based on the levels of fusion and sources of evidences being consolidated. Based on the type of information being consolidated, we discuss the fusion at sensor level, feature level, score level, rank level and decision level with examples from literature. Based on the sources of evidences being consolidated, we discuss the multi-sensor, multi-algorithm, multi-instance, multi-sample, multimodal and hybrid systems with examples from literature.
iii
Acknowledgement
I would like to express my cordial thanks to my professor Danilo Gligoroski for his guidance and valuable advices. I wish to express my sincere gratitude to my supervisor Yanling Chen. Her interests and continuous feedbacks on my work throughout the year proved to be very fruitful in reaching my goals. I am very grateful to her for motivating me throughout the period and always finding sufficient time for me from her schedule.
I would like to thank the Department of Telematics for providing all facilities and good environment. I am also very thankful to the Norwegian Government for providing me the Quota Scheme Scholarship for my master studies.
Finally, I would like to express my heartfelt thanks to my beloved parents and sister for their blessings, wishes and support.
iv
Table of Contents
1 Introduction
………..………..……..11.1 Thesis Motivation………...1
1.2 Related Work……….1
1.3 Thesis Outline……….…..2
2 Background
………..……….42.1 Biometric Systems………..4
2.2 Biometric System Functional Processes……….4
2.3 Biometric System Operations……….5
2.4 Desirable Properties of Biometric Characteristics………..7
2.5 Biometric Characteristics………..7
2.6 Biometric System Errors………....10
2.7 Social Acceptance and Privacy Issues……….…13
2.8 Challenges in Biometric Systems………..…13
2.9 Advantages of Multibiometric Systems over Unibiometric Systems………15
2.10 Levels of Fusion in Multibiometric Systems………16
2.11 Sources of Evidences in Multibiometric Systems………...16
2.12 Application of Biometric Systems……….….17
3 Levels of Fusion in Biometrics
……….………183.1 Sensor Level Fusion………18
3.2 Feature Level Fusion………..21
3.3 Score Level Fusion………..26
3.3.1 Classifier Combination Rules……….……….27
3.3.2 Score Fusion Techniques………..……30
3.3.2.1 Density-based Score Fusion……….30
3.3.2.2 Classifier-based Score Fusion……….32
3.3.2.3 Transformation-based Score Fusion……….…37
3.4 Rank Level Fusion……….46
v
List of Tables
3.1 Verification results for single modalities………..………33
3.2 Summary table of verification results………..………..35
3.3 Confusion matrices indicating performance of C5.0 decision tree……….35
3. 4 Performance of linear discriminant classifier on three different trials………...36
3.5 Summary of normalization techniques……….…....43
3.6 GAR (%) of different normalization and fusion techniques at 0.1% FAR……….44
4.1 Comparison between the different multibiometric systems (categorized on the basis of sources of evidence) [58]………...51
4.2 Errors of single and multi-sensor fingerprint verification systems………..53
4.3 Average recognition rates using AR and palmprint databases………67
3.5 Decision Level Fusion………48
4 Sources of Evidence
……….514.1 Multi-sensor Systems………...51
4.2 Multi-algorithm Systems……….53
4.3 Multi-instance Systems………56
4.4 Multi-sample Systems………..59
4.5 Multimodal Systems……….….63
4.6 Hybrid systems………..………..70
5 Conclusion
………..……..……..726 References
………..……….74vi
List of Figures
2.1 Conceptual structure of a biometric system………..…….6
2.2 Biometric system error rates……….……11
2.3 Receiver operating characteristic curve……….…12
3.1 Images (a) rolled fingerprint (b) dab……….……19
3.2 Result with different composing schemes………20
3.3 Bimodal biometric system using iris and face……….22
3.4 Procedure adopted in [50] to perform feature level fusion……….……24
3.5 Information flow when data from the feature level and match score level are combined…25 3.6 Match score level fusion………26
3.7 Classifier combination schemes and their relationships……….…………29
3.8 ROC curves when the scores are combined using the sum rule : (a) combining face and fingerprint scores (b) combining face and hand geometry scores……….…36
3.9 ROC curves when the scores are combined using the sum rule: (a) combining fingerprint and hand geometry scores and (b) combining face, fingerprint and hand geometry scores…….37
3.10 Conditional distribution of genuine and imposter scores: (a) face (distance score); (b) fingerprint (similarity score); and (c) hand-geometry (distance score)……….….38
3.11 Distribution of genuine and imposter scores after min-max normalization: (a) face; (b) fingerprint ; and (c) hand-geometry………...39
3.12 Distribution of genuine and imposter scores after z-score normalization: (a) face; (b) fingerprint ; and (c) hand-geometry………40
3.13 Distribution of genuine and imposter scores after median-MAD normalization: (a) face; (b) fingerprint ; and (c) hand-geometry………..….40
3.14 Double sigmoid normalization (𝑡= 200,𝑟1 =20,𝑎𝑛𝑑 𝑟2= 30)……….……41
3.15 Distribution of genuine and imposter scores after double sigmoid normalization: (a) face; (b) fingerprint ; and (c) hand-geometry………...42
3.16 Distribution of genuine and imposter scores after tanh normalization: (a) face; (b) fingerprint ; and c) hand-geometry……….……43
vii
List of Acronyms
COTS Commercial Off-the-Shelf CMC Cumulative match characteristic DWT Discrete Wavelet Transform EER Equal Error Rate
FTA Failure to Acquire
FTC Failure to Capture
FTE Failure to enroll
FAR False Acceptance Rate
3.17 ROC curves for unimodal systems……….……..44
3.18 ROC curves for sum of score fusion method………45
3.19 Example of rank level fusion………..…….47
3.20 Advanced decision level fusion……….….50
4.1 Architecture of the proposed multi-sensor fingerprint verification system………..…..52
4.2 Gait recognition by combining context-based classifiers………...55
4.3 Gait recognition by combining context-based classifiers. The context investigated in the system is walking surface type………...56
4.4 Multi-instance fusion block diagram……….…..58
4.5 Limited overlap between the two impressions of the same finger………....59
4.6 Receiver operating curves using Neyman-Pearson rule………..65
4.7 The single sample biometrics recognition procedure………66
4.8 BioID’s main functional units……….68
4.9 Sketch of a vector quantifier……….….69
4.10 Sensor fusion options……….…..70
4.11 Multi-sample and multimodal (hybrid) biometric model………...71
viii FMR False Match Rate
FNMR False Non-match Rate FRR False Rejection Rate
FBI Federal Bureau of Investigation
GA Genetic Algorithm
GAR Genuine Accept Rate
IAFIS Integrated Automated Fingerprint Identification System JFV Joint Feature Vector
k-NN k-nearest-neighbor
k-NN+VQ k-NN classifier with vector quantization LDA Linear Discriminant Analysis
MAD Median Absolute Deviation
NIST-BSSR1 National Institute of Standards and Technology Biometric Score Set- Release 1
PDA Personal Digital Assistant PIN Personal Identity Number PCA Principle Component Analysis ROC Receiver Operating Characteristics RJFV Reduced Joint Feature Vector TAD Threshold Absolute Distance TER Total Error Rate
1
Chapter 1 Introduction
1.1 Thesis Motivation
Reliable identity establishment/conformance is becoming critical in a variety of applications.
Some examples of such applications are sharing networked computer resources, performing remote financial transactions, border security control, and forensic applications. Traditional methods of establishing identity are either knowledge based (e.g., passwords) or possession based (e.g., ID cards). Individuals have certain distinct physiological and behavioral traits that are used by biometric systems for reliable authentication. Biometric systems provide better security and greater convenience than the traditional systems.
Our motivation for working on this project comes from the fact that in near future biometric systems will be supplementing or replacing the traditional systems in many applications.
Most of the biometric systems presently being used are unibiometric systems typically making use of a single biometric trait for recognition purpose. There are several limitations of unibiometric systems and some of these can be addressed by designing multibiometric systems that consolidate multiple sources of biometric information. Multibiometric systems can improve the matching accuracy of a biometric system [54]. They also address challenges such as non-universality, noise, susceptibility to spoof attacks and large intra-class variations.
1.2 Related Work
As multibiometric systems can be one of the important solutions for various applications in near future, there has been a long list of articles addressing this topic. Ross et al. [54]
provide a very good survey of multibiometric systems. The authors focus on the survey of various levels of fusion and go into the details of score level fusion. The ISO/IEC Technical Report [25] contains descriptions and analysis on current practices on various multibiometric fusion. It also discusses requirements and possible routes of standardization to support multibiometric systems.
There are numerous research papers on the various levels of fusion in multibiometric systems. Ratha et al. [49] propose a mosaicking scheme which constructs a composite fingerprint image fingerprint by integrating multiple partial fingerprints as the user rolls finger on the sensor surface. Singh et al. [56] propose a face recognition system combining visible and thermal Infrared (IR) images at sensor level. Kong et al. [18] also discuss a face recognition system performing fusion of visual and thermal infrared images with eyeglass removal at sensor level. Son et al. [60] perform the feature level fusion of face and iris. Ross
2 et al. [50] perform feature level fusion of hand and face biometrics and perform experiments in three different scenarios. Kittler et al. [34] develop a common theoretical framework for combining classifiers and discuss the various classifier combination strategies. Verlinde et al.
[68] compare the performance of score level fusion using three different classifiers based on the k-nearest-neighbor (k-NN) classifier, decision trees and logistic regression. Jain et al. [53]
use the classifiers decision trees and linear discriminant function for fusion of match scores.
Jain et al. [26] study the performance of different normalization techniques and fusion methods in a fusion scenario involving face, fingerprint and hand geometry modalities. Ho et al. [19] describe the three methods: the highest rank method, the borda count method, and the logistic regression method, to combine the ranks assigned by different matchers.
Decision level fusion process is categorized into simple decision level fusion and advanced decision level fusion and discussed in [25].
Many works discuss on the various categories of multibiometric systems based on the sources of information being consolidated. Marcialis et al. [39] discuss a multi-sensor fingerprint system employing optical and capacitive sensors. Jain et al. [31] propose a multi- algorithm system which integrates the evidence obtained from three different minutiae based fingerprint matchers. Han and Bhanu [17] propose a multi-algorithm gait recognition system which probabilistically combines different gait classifiers based on different environmental contexts. Wang et al. [69] discuss a multi-instance iris recognition system where the left and right irises of an individual are combined. Jain et al. [29] describe a multi- sample system which constructs a composite fingerprint template from multiple impressions of the same finger using mosaicking scheme. Bowyer et al. [3] evaluate the performances of face recognition system using both the multi-sensor and multi-sample approaches. Jain et al.
[27] investigate a multimodal biometric identification system combining face, fingerprint and voice modalities. Yao et al. [70] propose a multimodal biometric system combining face and palmprint features. Thian et al. [43] propose a hybrid multibiometric system where fusion of multiple samples obtained from multiple modalities is performed at score level.
1.3 Thesis Outline
Chapter 1. Introduction outlines the motivations for working on this thesis, the related works and the thesis outline.
Chapter 2. Background presents general information about biometric system operations and functional processes, biometric system errors, biometric characteristics and their desirable properties, challenges in biometric systems and advantages of multibiometric systems over unibiometric systems, etc.
Chapter 3. Levels of Fusion in Biometrics provides an overview on various levels of fusion. It describes sensor level fusion, feature level fusion, score level fusion, rank level fusion and
3 decision level fusion with corresponding examples from the literature. The score level fusion is dealt in more details.
Chapter 4 Sources of Evidences provides an overview of six categories of biometric systems depending on the nature of the sources of information being fused. Multi-sensor, multi- algorithm, multi-instance, multi-sample, multimodal and hybrid systems are described with examples from the literature. More examples are studied for multimodal systems.
Chapter 5 Conclusion summarizes the main findings and concludes the thesis.
4
Chapter 2 Background
2.1 Biometric Systems
Biometric systems perform recognition of individuals on the basis of their physical and/or behavioral traits. Some commonly used traits are fingerprint, face, iris, retina, palmprint, voice pattern, signature, gait, etc. Most biometric systems will serve one of the two purposes: identification or verification/authentication. Biometric systems provide several advantages over the traditional methods. Unlike passwords and tokens, biometric traits cannot be lost, forgotten or manipulated. Biometric traits cannot be easily copied, shared, distributed or forged. Biometric systems also add to user convenience by alleviating the need to design and remember passwords. Moreover, use of biometrics can provide negative recognition and non-repudiation which is not possible through traditional methods. Negative recognition is a process by which an individual is found to be enrolled in a system despite his unwillingness to be identified. Non-repudiation is a way to guarantee an individual accessing a certain facility cannot later deny having used it. Multibiometric systems consolidate multiple sources of biometric evidences. The integration of evidences is known as fusion.
Multibiometric systems combine the information from multiple sensors, samples or traits of an individual, matching algorithms operating on the same biometric.
2.2 Biometric System Functional Processes
A biometric system involves the following three main functional processes:
Enrollment Process:
In enrollment process, a subject presents his/her biometric characteristics to the sensor along with his/her non-biometric information. Non-biometric information related to subjects could be name, social security number, driver license’s number, etc. Biometric features extracted from the captured sample and the non-biometric information are enrolled in the database.
Verification Process:
In a verification process, the question being answered is “Is this person who he claims to be?”. Subject who desires to be recognized claims his identity which could be a Personal Identity Number (PIN), a username or a smartcard and presents his biometric characteristic(s). The system then compares the extracted template (from the captured
5 sample) with the enrolled template linked to the claimed identity and determines whether the claim is true or false. Identity verification is used in positive recognition applications where a subject is willing to be recognized.
Identification Process:
In an identification process, the question being answered is “Who is this person?”. In this process, an individual is recognized by searching the templates of all users in an enrollment database against the captured and extracted biometric features for a match. Identification is a critical component in negative recognition applications where the user tries to avoid being found out who he is [45]. Some examples of negative recognition applications are background checks, forensic criminal identification or preventing terrorists from entering certain areas. Though traditional recognition methods such as passwords, PIN, tokens work for positive recognition; the only viable approach for negative recognition is biometric identification [45].
2.3 Biometric System Operations
The overall conceptual structure of a biometric system as given in [24] is shown in Figure 2.1.
The biometric system usually consists of five subsystems enumerated below [24].
Biometric data capture subsystem
This subsystem comprises of suitable capture devices or sensors. A sensor is required to collect signals from a biometric trait and convert the captured signals into a biometric sample such as a fingerprint image, iris image or voice recording.
Signal processing subsystem
This subsystem is responsible for extracting a set of salient discriminatory features from a biometric sample. The extracted feature set represents the underlying trait. The biometric features are suitable for comparing with those extracted from other biometric samples. The biometric feature set extracted in the enrollment process is stored in the data storage subsystem which serves as a biometric reference during recognition process.
Data storage subsystem
During enrollment phase, the feature sets extracted are stored in a data storage subsystem.
The feature sets are possibly stored along with other non-biometric information related to subject such as name, PIN, social security number, etc. In practice, biometric templates and non-biometric information are often stored in different databases which are logically or physically separated for security and privacy concerns.
6 Figure 2.1: Conceptual structure of a biometric system [24].
Comparison or Matching subsystem
In a comparison subsystem, the similarity (or difference) between the extracted features (from input sample) and the enrolled biometric templates is determined. In case of verification process, the captured biometric template is compared with the corresponding enrolled biometric template to produce a comparison score. In the identification process, an extracted feature set of a subject is compared against a set of enrolled biometric templates of more than one subject to return a set of comparison scores.
Enrolment Verification Identification Identity Reference
Request
Identity Reference
Biometric features
Captured Biometric Sample Presentation
Sensor
Data Capture Subsystem
Biometric characteristics
Biometric Reference
Reference Creation
Quality Control Feature Extraction
Signal Processing Subsystem
Biometric features
Decision Subsystem
Candidate?
Match?
Verified? Identified
? Threshold
(Candidate List)
Decision Policy Match/ Non-
match
Verification
Outcome Identification Outcome Biometric
Reference
Comparison Subsystem
Comparison
Comparison Score(s)
Individual
DBIR
DBBR
IR Claim BR Claim IR & BR
Association
Data Storage Subsystem
Identity Registration Identity Claim
7 Decision subsystem
Based on the comparison score(s) and decision policy, a decision subsystem determines if the captured biometric sample and enrolled template are derived from the same subject. In case of verification process, decision made based on a comparison score is either the acceptance or rejection of the subject. In case of identification, a ranking of enrolled identities that meet the decision policy is presented in order to identify an individual.
2.4 Desirable Properties of Biometric Characteristics
Some desirable properties of biometric characteristics for good subject discrimination and reliable recognition performance are described below [54]:
Universality: Every individual should possess the characteristic.
Uniqueness: The characteristics should be sufficiently distinguishable across individuals comprising the population.
Permanence: The biometric characteristics should be sufficiently invariant over a period of time.
Measurability: It should be possible to acquire the characteristics without causing undue inconvenience. The acquired raw data should be suitable for further processing.
From an application point of view, following properties should also be taken into account.
Performance: The required recognition accuracy in an application should be achievable using the characteristics.
Acceptability: Acceptability refers to the willingness by the subject to present his biometric characteristics.
Spoof Resistance: This refers to how difficult it is to use artifacts (for example, fake fingers) in case of physiological characteristics and mimicry in case of behavioral characteristics.
2.5 Biometric Characteristics
There are various physiological and behavioral biometric characteristics that can be used during recognition. The choice of a biometric characteristic to be used in a specific application is made depending upon the nature and requirements of applications and the properties of the biometric characteristics [32]. Physiological biometric traits include but are not limited to fingerprint, face, iris, hand geometry, hand/finger vein, retina, DNA and palm print. Behavioral characteristics include but are not limited to signature and gait. We introduce some of the commonly used biometric characteristics discussed in [32].
8 Face:
Face recognition is a non-intrusive method and also requires minimum cooperation from the subject. The dimensions, proportions and physical attributes of a person’s face are unique. In some application scenario like crowd surveillance, face recognition probably is the only feasible modality to be used. Face recognition can be in a static controlled environment or a dynamic uncontrolled environment. One popular approach to face recognition is based on the location, dimensions and proportions of facial attributes such as eyes, eyebrows, nose, lips, and chin and their spatial relationships. Another approach being widely used is based on the overall analysis of the face image that represents face as a weighted combination of a number of canonical faces.
Face recognition involves two major tasks: i) face location and ii) face recognition. Face location is determining the location of face in the input image. For recognizing the located face, the eigenface approach is one of the very popular methods. The eigenface-based recognition method consists of two stages: i) training stage and ii)operational stage. In the training stage, training set of face images are acquired. The acquired face images are projected into lower dimensional subspace using Principle Component Analaysis (PCA) [63].
A set of images that best describe the distribution of training images in a lower dimensional facespace (the eigenspace) is computed. Then the training facial images are projected into this eigenspace to generate representation of the training images in the eigenspace. In the operational stage, the input face image is projected into the same eigenspace that the training samples were projected into. Then, recognition can be performed by a classifier operating in the eigenspace.
Fingerprints:
Fingerprints are unique and consistent over time and hence being used since a long time. A fingerprint is a pattern of ridges and valleys on the surface of a fingertip. Ridges are the upper skin layer segments of the finger and valleys are the lower segments. The various kinds of discontinuities in ridges (minutiae) have sufficient discriminatory information to recognize fingerprints. Ridge bifurcation (where the ridge splits) and ridge ending (where the ridge ends) are the important minutiae points. A minutiae-based fingerprint recognition usually represents fingerprint by these two ridge characteristics called as minutiae.
The uniqueness of a fingerprint can be determined by the pattern of ridges and furrows as well as the minutiae points. Availability of multiple fingerprints of a person makes fingerprint recognition suitable for use in large-scale identification involving millions of identities.
However, the problem with the large scale fingerprint recognition system is the requirement of huge amount of computational resources, especially in the identification mode.
9 Hand geometry:
Hand geometry recognition systems are based on the different measurements such as shape of the hand, size of palm, lengths and widths of the fingers. Hand features are not very distinctive. They are suitable for verification but not for identification [52]. In certain situations such as immigration and border control, biometrics such as fingerprints may not be suitable because they infringe on privacy. In such situations hand geometry can be used for verification as hand geometry is not very distinctive. Hand geometry features may not be invariant during the growth period of children. The size of such recognition systems is large and hence it is difficult to embed the systems in other devices such as laptops.
Palmprint:
Human palms also contain pattern of ridges and valleys like human fingerprints. Palmprint based recognition is based on the principle lines, wrinkles and ridges on the surface of the palm. Palmprint is distinct for each person. Palmprint scanners are bulkier and more expensive than fingerprint sensors. Features such as principal lines and wrinkles can be captured even with a low resolution scanner. When a high-resolution scanner is used, all features such as geometry, ridges and valley features, principle lines and wrinkles can be combined to achieve higher accuracy. Kumar et al. [36] use both palmprint and hand geometry features for personal verification. Both features are simultaneously acquired from a single hand image.
Iris:
Iris is the annular region of the eye regulating the size of the pupil. It is bounded by pupil and sclera (white of the eye) on either side. Iris develops during prenatal period and stabilizes during the first two years of life. The complex iris texture carries very distinctive information useful for personal recognition. Irises of twins are different as well. Iris based recognition systems provide promising speed and accuracy and support large scale identification operations as well. Contact lenses printed with fake iris [11] can be detected. The hippus movement of the eye can also be used for liveness detection.
Signature:
Signature is a behavioral biometrics. Electronic signature, for example taken at a POS terminal is compared to the signature on our driving license (or another type of ID) for verification. This is not signature recognition but can be called as ‘simple signature comparison’ [9]. Signature recognition involves a process known as ‘dynamic signature recognition’ where the focus is not only on the ‘look’ of the signature, but on the behavioral patterns inherent to the process of signing. This includes changes in timing, pressure, and speed. It is easy for an imposter to duplicate the visual appearance of signature. However, it is difficult to mimic the behavioral characteristics. Signature recognition is particularly suitable for high-value transactions. Signature recognition is also non-invasive. However, the
10 system is prone to high error rates when the behavioral characteristics of signatures are not consistent.
Voice:
Voice recognition is a combination of both physical characteristics and behavioral biometric characteristics. Voice recognition uses the acoustic features of speech that vary among individuals to discriminate among users. The variations in these acoustics properties arise because of the anatomical differences naturally occurring in individuals and the differences in learned speaking habits. The physical characteristics remain constant whereas the behavioral characteristics of voice could change over time because of age, medical conditions, emotional state, etc. Voice is not distinctive enough to be used for large scale identification. A voice recognition system could be either text-dependent or text- independent. In a text based system any subject needs to utter a specific phrase whereas a text-independent system recognizes subject independent of what he speaks. Text- independent systems are more difficult to design and also more robust against frauds.
Gait:
Image-based recognition methods employing fingerprint, face or iris modalities require co- operation from subject, physical contact or close proximity with capture devices. Gait recognition is based on recognizing individuals on the basis of the way they walk. This technique can be appropriate in many practical cases where the environmental condition is changing; subject is not cooperating and is at a distance from the capture device. Gait recognition has several challenges. Gait can be affected by clothing, injuries or other environmental context. There can be large variation in gait characteristics of an individual (both intentionally and unintentionally) making it less unique compared to iris or fingerprint.
However, it is still useful in many visual surveillance applications.
2.6 Biometric System Errors
Two samples of a single user’s biometric trait are rarely read exactly the same. This occurs due to various reasons such as imperfect sensing condition, alterations in user’s biometric characteristics, changes in ambient conditions and user’s interaction with the sensor.
Therefore, the output of a biometric matching system is a similarity score(s) that quantifies similarities between the enrolled and input templates. The system decision depends on the set threshold 𝑡. Pairs of biometric samples generating a similarity score 𝑠 greater than 𝑡 are inferred as mate pairs belonging to the same person. Pairs of samples with similarity score less than 𝑡 as inferred as non-mate pairs belonging to different persons. The distribution of match scores generated from pairs of samples from different persons is called an imposter distribution and the distribution of match scores generated from pairs of samples of the same person is called a genuine distribution (see Figure 2.2).
11 False Acceptance Rate (FAR) (or, the False Match Rate (FMR)) of a biometric system is the rate at which the non-authorized persons are falsely recognized during matching process.
False Rejection Rate (FRR) (or, the False Non-match Rate (FNMR)) of a biometric system is the rate at which authorized people are falsely not recognized during matching process.
Total Error Rate (TER) is obtained by combining these two errors. TER = (Number of False Accepts + Number of False Rejects) / (Total Number of Accesses).
Regulating the threshold 𝑡 changes both FAR and FRR. If the threshold 𝑡 is increased in order to attain higher system security, FRR increases. If the threshold is decreased in order make the system more tolerant to input variations and noise, and reduce annoyance, FAR increases. Therefore, a biometric system needs to make a tradeoff between FAR and FRR.
The system performance at all operating points (thresholds 𝑡) can be depicted by Receiver Operating Characteristics (ROC) Curve. ROC curve represents the FAR as a function of FRR (see Figure 2.3). In many cases ROC curve plots the (1-FRR) (instead of FRR) against the FAR.
The Equal Error Rate (EER), which is the FAR and FRR when they are equal, is often used as a performance measure. However, EER is not a robust measure for system performance [4].
This is because most practical biometric systems do not have threshold adjusted for FAR=FRR. ROC curves of various systems could be very different and thus two systems with the same EER could differ by decades for other ROC points.
Figure 2.2: Biometric system error rates: The curves show FAR and FRR for a given threshold 𝑡 over the genuine and impostor score distributions. FAR is the percentage of the non-mate pairs whose matching scores are greater than or equal to 𝑡, and FRR is the percentage of the mate pairs whose matching scores are less than 𝑡 [45].
12 Figure 2.3: Receiver operating characteristic curve: Different biometric application types make different trade-offs between the FAR and FRR. [45].
Besides the two error rates FAR and FRR discussed above, some other error rates are also used to characterize biometric system’s accuracy. The Failure to Acquire (FTA) (also known as Failure to Capture (FTC)) rate shows the rate at which biometric device fails to automatically capture a sample when presented with a biometric characteristic. This usually occurs because of low quality of inputs (for example, extremely faint fingerprint) and also sensor wear and tear. Failure to enroll (FTE) rate is the proportion of users who cannot be successfully enrolled in a biometric system. FTE rate usually occurs when the system rejects poor quality templates during enrollment.
We now discuss Cumulative Match Characteristic (CMC) curves. These are the most common graphs for evaluating closed-set identification performances of a system. In closed-set identification, every probe sample has a corresponding match in the database. However, such systems usually exist in laboratories and there are very few real-world applications operating under closed-set identification task. CMC curve is relevant to a recognition scenario, in which a probe sample is matched against each of a set of gallery samples. The gallery sample exhibiting best match (for example best similarity score) with the probe sample represents the identity of the probe. If the trial is repeated for all probe samples, it is possible to know how often the top match selected by the system represents a correct identity. The rank one identification rate is the percentage of the probes for which the closest match (top similarity score) in the gallery represents the correct identity. The percentage of probes for which either the closest or second-closest match (top or second ranked score) in the gallery represents the correct identity is the rank two identification rate.
13 Thus rank forty identification rate gives the probability that the correct identity lies somewhere in the top forty similarity scores. A CMC curve shows the probability of identification at numerous ranks. CMC curves are largely dependent on gallery size. The probability of correct identification at a certain rank is higher for smaller databases. For instance, for identification at rank 10, the probability of correct identification is much higher when the database size is 100 than when the database size is 10,000. Therefore, it is important to state the size of database for any CMC curve.
2.7 Social Acceptance and Privacy Issues
The ease and comfort in interaction with the biometric systems largely contribute to acceptability. Acceptability may also be influenced by religious, cultural and ethnic factors.
For example, use of contactless biometric features such as face, iris, or voice may be considered as more user-friendly and hygienic [33]. Likewise, systems requiring lesser co- operation from user may be considered as more convenient to users. However, biometric characteristics which can be captured without user participation might be captured without the knowledge of user which may be perceived as threat to individual privacy.
“Privacy is the ability to lead life free of intrusions, to remain autonomous and to control access to one’s personal information”[45]. The use of biometric data raises several privacy concerns which need to be addressed. We mention some threats to privacy discussed in [24]. Biometric data could be misused for applications other than originally intended.
Biometric data might be misused to retrieve or analyze some other information that is not required for recognition purpose. For example, the subject’s health status or ethnic background could be determined from biometric traits. Biometric data could also be used in linking information of a subject across different databases or systems. Public need to be ensured that their biometric data is used only for the intended purpose and the biometric information remains private by the companies and agencies operating biometric systems.
Appropriate legislation is necessary to ensure that the biometric information is not abused and the misuse is punished. Biometric applications with highly decentralized recognition capabilities are the most acceptable [45]. This can be done by storing the biometric information in decentralized encrypted databases over which a subject has his control. For example, a system can issue user a smart card with his fingerprint template stored on it in an encrypted format.
2.8 Challenges in Biometric Systems
Most biometric systems presently in use employ a single biometric trait for recognizing individuals. Even though these unibiometric systems have offered a reliable solution for identification and verification applications, it is important to consider the vulnerabilities and
14 limitations of these systems. Some of the challenges encountered by the unibiometric systems are described below [51]:
Noise in sensed data:
The biometric data is contaminated by noise mainly due to slight variations in the biometric trait itself or imperfect acquisition conditions. For example, a fingerprint image with a scar or a voice sample altered by cold is noisy data. Inappropriate ambient conditions like poor illumination of user’s face in face recognition or imperfectly maintained sensors like a fingerprint sensor with dirt on its surface lead to noisy data. Noisy data can result in rejection of a genuine user.
Non-universality:
Biometric system may not be able to acquire meaningful biometric data from a subset of individuals. This results in a failure-to-enroll (FTE) error. For example, an iris recognition system may not be able to obtain the iris information of users with long eyelashes, drooping eyelids or certain pathological conditions of eyes.
Spoof attacks:
Spoofing attack is more relevant in cases where behavioral traits such as signature and voice are used. In such cases, an imposter tries to mimic the traits corresponding to the enrolled user. However, physical traits such as fingerprints and iris are also vulnerable to spoof attacks by creating biometric artifacts. Matsumoto et al. [41] report that the gummy fingers, made using cheap and easily obtainable tools and materials, were accepted with high rates by the 11 different fingerprint systems they used. Those fingerprint systems employed optical or capacitive sensors. Gelatin, a readily available and cheap soft plastic material was used to make gummies. Not only gummy fingers made using impression taken from live fingers but also the gummy fingers made from residual fingerprints were readily accepted by their systems. Targeted spoof attacks can seriously undermine the security of biometric systems. Different ways have been suggested to protect the system from spoofing attacks.
For example, in the case of fingerprint and iris, liveness detection can be used. There are two complementary approaches to liveness detection [65]. One is detection of the known artifacts (e.g. silicon and gelatin fingerprints, photograph of a face etc.). The other approach is to look for evidences of liveness in the presented biometric (for example temperature, pulse, humidity etc). In the case of behavioral traits such as voice, a challenge response mechanism could be used (for example system prompts “Please say 5-3-4-8”).
Intra-class variations:
Changes in biometric characteristics of a person with the passage of time (for example, change in hand geometry) or user interactions with the sensor in a wrong manner (for example, incorrect facial pose) are the main factors resulting in intra-class variations
15 between the enrolled and input template of an individual. Some ways to address the intra- class variations could be storing multiple templates for every user during enrollment and updating these templates at certain intervals of time [65]. Intra-class variations are more serious concerns in biometric systems using behavioral traits since the variations in psychological makeup of an individual might result in very different behavioral traits at different times. For example, the voice of a person can vary depending on stress levels, health conditions. Similarly, gait can be affected by clothing, injuries, inebriation and other environmental context.
Inter-class similarities:
Inter-class similarity refers to overlapping of feature spaces corresponding to multiple classes or individuals. Inter-class similarity is prominent in an identification system comprising a large population of enrolled users resulting in an increased false match rate.
Therefore, there is an upper bound on the number of individuals that can be discriminated effectively which determines the capacity of an identification system.
2.9 Advantages of Multibiometric Systems over Unibiometric Systems
We discuss some of the advantages multibiometric systems offer over unibiometric systems in the following paragraphs [51].
Multibiometric systems address the issue of non-universality i.e., limited population coverage. For example, if a person’s poor quality of fingerprints prevents him from enrolling in the system; then the use of other biometric traits such as iris, face, voice etc. will help the system acquire meaningful biometric data and enroll the user.
It is extremely difficult to spoof multiple biometric traits of a legitimate user. If each subsystem determines the probability of the particular trait being a spoof, it is possible to find out the probability of the user being an imposter by using an appropriate fusion technology. Moreover, a challenge response mechanism can be included that asks user to present the random subset of traits (in a particular order) at the point of acquisition. This would ensure that the system is interacting with a live user.
Multibiometric systems effectively address the problem arising because of noisy data. When the information acquired from one biometric trait is corrupted by noise, it is possible to use information acquired from the other biometric trait. Some systems also take into considerations the quality of acquired input biometric signals during the fusion process.
Estimating the quality of acquired biometric data is in itself a challenging problem. However, if done appropriately, multibiometric systems gain significant benefits.
16 A multibiometric system acts as a fault tolerant system by continuing to operate even when information from certain biometric sources becomes unreliable because of sensor or software malfunctions or intentional user manipulation. Fault tolerance is usually desirable in authentication systems involving large number of subjects (for example, in border control applications).
Consolidation of evidences from multiple sources can offer substantial improvement in the accuracy of biometric systems. Use of proper sources of information and the right fusion methodology determines the improvement in matching accuracy. The availability of multiple sources also increases the feature space thereby increasing the number of individuals that can be discriminated reliably. Therefore, the capacity (i.e., the number of users that can be enrolled) of an identification system can be increased.
2.10 Levels of Fusion in Multibiometric Systems
Fusion in multibiometric systems can be performed utilizing information available in any of the modules (data capture module to decision module). Fusion can take place at these levels: i) sensor level ii) feature level iii) score level iv) rank level and v) decision level. In sensor level fusion raw data captured by the sensor(s) are combined. In feature level fusion features originating from each individual biometric process are combined to form a single feature set or vector. In score level fusion, match scores provided by different matchers indicating degree of similarity (differences) between the input and enrolled templates, are consolidated to reach the final decision. In rank level fusion each biometric sub-system assigns a rank to each enrolled identity and the ranks from the subsystems are combined to obtain a new rank for each identity. In decision level fusion the final Boolean result from every biometric subsystem are combined to obtain final recognition decision. We provide a more detailed description of fusion at various levels in Chapter 3.
2.11 Sources of Evidences in Multibiometric Systems
Various sources of biometric information can be used in a multibiometric system. Based on these sources, multibiometric systems can be classified into six different categories [54]:
multi-sensor, multi-algorithm, multi-instance, multi-sample, multimodal and hybrid. Multi- sensor systems employ multiple sensors to capture a single biometric trait in order to extract diverse information. In multi-algorithm systems, multiple algorithms are applied to the same biometric data. Multi-instance systems use multiple instances of the same body trait (for example, left and right irises or left and right index fingers). In multi-sample system, multiple samples of the same biometric trait are acquired using the same sensor in order to obtain a more complete representation of the underlying trait. Multimodal systems combine evidences obtained from different (two or more) biometric traits. In [54] hybrid is used to refer to those systems integrating two or more of the scenarios mentioned earlier. We
17 conduct a detailed survey of multibiometric systems based on the sources of information in Chapter 4.
2.12 Application of Biometric Systems
Biometric applications can be categorized into three main groups [45]:
1) Commercial applications such as computer network login, e-commerce, Internet access, ATMs or credit cards, physical access control, mobile phones, Personal Digital Assistant (PDA)s, medical records management, distance learning, etc.
2) Government applications such as national ID card, driver’s license, social security, border control, passport control, welfare-disbursement, etc.
3) Forensic applications such as corpse identification, criminal investigation, terrorist identification, parenthood determination, etc.
18
Chapter 3
Levels of Fusion in Biometrics
It is important to determine the type of information that should be consolidated during fusion process. The amount of information available decreases after each level of processing in different modules of a biometric system. The raw data represents the richest source of information whereas the final decision just contains an abstract level of information. The various levels of fusion are categorized as (i) preclassification or fusion before matching and (ii) postclassification or fusion after matching [55]. This categorization is based on the fact that the amount of information available for fusion is drastically reduced once the matcher is invoked. Fusion before matching can take place either at the sensor level or at the feature level. Fusion at score level, rank level and decision level occur after matching module is invoked (postclassification). In this chapter we discuss the various levels of fusion in multibiometric systems.
3.1 Sensor Level Fusion
The raw biometric data represents the richest source of information. However, it is highly probable that raw data is contaminated by noise (for example, non-uniform illumination, background clutter, etc.). Sensor level fusion refers to the consolidation of raw data obtained using multiple compatible sensors or multiple snapshots of a biometric using a single sensor [51].
Example 3.1 Mosaicking multiple fingerprint impressions to construct rolled fingerprint Image mosaicking refers to aligning of two or more images into a new aggregate image without distortion in the overlapping areas. Mosaicking multiple fingerprint impressions to construct a composite image is an example of sensor level fusion. Ratha et al. [49] describe a mosaicking scheme which constructs a rolled fingerprint by integrating multiple partial fingerprints as the user rolls finger on the sensor surface. A rolled fingerprint is preferable over plain touch impression known as dab during enrollment of a person in database. A sample rolled fingerprint and dab are shown in Figure 3.1. This rolled fingerprint covers larger area of the finger, thereby including larger number of feature points. Therefore, the overlap is higher when the partial fingerprint impression (query impression) is matched to rolled fingerprint template than when it is matched to another partial fingerprint.
19
(a) (b) Figure 3.1: Images (a) rolled fingerprint (b) dab [49].
The first step to fingerprint mosaicking algorithm is to segment each frame into foreground, the fingerprint area and background, the non-fingerprint area. The second step is to construct a rolled fingerprint mosaic from the set of frames of partial fingerprint impression.
For this purpose, the frames stacked are visualized as image planes. If it is assumed that there was no slipping when user rolled his finger on sensor, the resultant fingerprint should be the aggregate of the individual image components. To determine the aggregate, a pixel in all the frames is considered and the resultant pixel is computed as a mathematical function of the pixels. Authors describe five schemes for constructing rolled fingerprint image. The results with different composing schemes are shown in Figure 3.2.
The simplest approach is naïve averaging over the whole image. The second approach ignores the foreground masks and only takes the minimum of the intensity value at each pixel. The third approach does averaging only in the region where fingerprint is detected.
The fourth approach is similar but it uses a mask that tapers from zero at the edges of foreground to one at the central region. The last approach shrinks the foreground mask and only uses the central portion of each fingerprint image. The final step is to compute the confidence level at each pixel in order to evaluate the reconstructed image.
a)Naive b) Minimum
20
c) Foreground d) Smoothed
e) Center
Figure 3.2: Result with different composing schemes [49].
Example 3.2 Fusion of infrared (IR) and visible face Images for face recognition
Fusion of visible and thermal face images at sensor level is discussed in several literatures.
Singh et al. [56] describe a face recognition system by fusion of visible and thermal infrared images at sensor level. Face recognition is not sufficiently accurate in uncontrolled environments even when efficient approach to face recognition like the eigenface approach is implemented. Using IR images can be a good alternative to using visible images for face recognition applications under changing illuminations as the IR images are relatively insensitive to illumination changes. However, IR image has other limitations. It is opaque to glass and is sensitive to surrounding temperature changes and variations in the heat patterns of the face. On the other hand, visible image is more robust to the mentioned factors but very sensitive to illumination changes. In [56], authors concentrate on the sensitivity of IR images to eyeglasses. Eye glasses act as temperature screen and hide the parts located behind them degrading the recognition performance significantly. The experiments in [56] show that face recognition performance in IR spectrum is significantly degraded when eyeglasses are present in the probe image but not in the gallery image and vice versa. In order to address the serious problem arising from the sensitivity of IR image to facial occlusion due to eyeglasses, [56] proposes fusion of information from both IR and visible spectra. Genetic Algorithm (GA) (see [56] for more on GA) is employed for feature selection and fusion where group of wavelet features (see [56] for review on wavelets) from
21 visible and thermal face images are selected and fused to form a fused image. Experiments are performed using the Equinox face dataset [21]. The eigenface approach to face recognition is used. The experimental results show substantial improvements in recognition performance suggesting the potentials of fusing IR with visible images.
Example 3.3 Fusion of visible and infrared images with eyeglass removal for face recognition
Another example of fusion of visual and thermal infrared images at sensor level is by Kong et al. [18]. By integrating visual and thermal face images, a new face image is obtained that is invariant to illumination conditions and also robust under low lighting conditions. In the fusion process, eyeglasses which block thermal energy are detected from thermal images with an ellipse fitting method (see [18] for more). The detected eyeglass regions are replaced with template eye pattern in order to retain information for face recognition. A commercial face recognition software FaceIt® is used as an individual recognition module.
From the experiments performed under conditions of varying illumination and facial expressions, it is observed that sensor-fusion based face recognition outperforms individual visual and infrared face recognitions.
3.2 Feature Level Fusion
In feature level fusion, feature sets originating from multiple information sources are integrated into a new feature set. For homogeneous feature sets (for example, multiple measurements of a person’s hand geometry), fusion can be achieved by calculating the weighted average of the individual feature vectors [54]. For non-homogeneous feature sets (for example, features of different modalities like face and hand geometry), a single feature set can be obtained by concatenation. However, for incompatible feature sets (for example, fingerprint minutiae and eigenface coefficients) concatenation is not possible.
Dimensionality reduction scheme like feature selection/transformation is applied to obtain a minimal feature set. The key benefit of this fusion scheme is that it enables detection/removal of correlated feature values improving recognition accuracy. Fusion at match score level and decision level are extensively studied in literatures. Fusion at feature level is relatively less studied.
Feature level fusion is challenging for the following reasons [58]:
1) The feature vectors of multiple modalities might be incompatible. For example, the minutiae set of fingerprints and eigen-coefficients of face.
2) The relationship between the feature spaces of different biometric systems may not be known.
3) Concatenation of two feature vectors might result in a feature vector with very large dimensionality leading to the curse-of-dimensionality problem. In such cases, when
22 sufficiently large numbers of training samples are not available, increasing number of features might degrade system performance.
4) Most commercial biometric system vendors do not provide access to the feature sets.
5) More complex matchers might be required to operate on concatenated feature vectors.
Example 3.4 Feature level fusion of face and iris
Son et al. [60] perform feature level fusion of face and iris (see Figure 3.3). They apply multilevel two-dimensional Discrete Wavelet Transform (DWT) to extract feature vectors from the iris and face images. For fusion, concatenation is done between the iris and face feature vectors to form a Joint Feature Vector (JFV). The feature dimensionality is further reduced by applying Direct Linear Discriminant Analysis (DLDA) in order to extract Reduced Joint Feature Vector (RJFV). RJFV has a lower dimensionality and a higher discriminating power than the JFV. Their experiments show that the multimodal authentication system using RJFV exhibits considerably better performance than unimodal system.
Figure 3.3: Bimodal biometric system using iris and face [59].
23 Example 3.5 Feature level fusion of hand and face biometrics
In this section we summarize the feature level fusion suggested by Ross et al. [50].
Let 𝑋 = {𝑥1,𝑥2, … ,𝑥𝑚} and 𝑌 = {𝑦1,𝑦2, … ,𝑦𝑛} denote the feature vectors (𝑋 ∈ 𝑅𝑚 and 𝑌 ∈ 𝑅𝑛) representing information extracted from two different sources. In order to yield the new feature vector 𝑍, vectors 𝑋 and 𝑌 are augmented and then feature selection is performed on the resultant vector in order to reduce its dimensionality. The different stages adopted in [50] are:
Feature Normalization: The individual feature values of the vectors 𝑋 and 𝑌 may be significantly different in terms of their range and distribution. For example, the values of 𝑥𝑖’s may be in the range [0,100] while 𝑦𝑖’s values may be in the range [0,1]. Therefore, feature normalization is performed to modify the mean and variance of the feature values in order to ensure the contribution of each feature vector is comparable [30]. Ross et al. test two normalization techniques: the simple min-max and median normalization (see [23] for details on these techniques). In their experiments they use the median normalization scheme because of its robustness to presence of outliers in the training data.An outlier is an observation that is numerically distant from the rest of the data. After normalization the modified feature vectors are represented as 𝑋′= {𝑥′1,𝑥′2, … ,𝑥′𝑚} and 𝑌’ = {𝑦′1,𝑦′2, … ,𝑦′𝑛}.
Feature Selection: When two feature vectors 𝑋’ and 𝑌’ are augumented, a new feature vector 𝑍′= {𝑥′1,𝑥′2, … ,𝑥′𝑚,𝑦′1,𝑦′2, … ,𝑦′𝑛} (𝑍′ ∈ 𝑅𝑚+𝑛} is obtained. The curse of dimensionality dictates that the augmented vector might not result in an improved performance [62]. Feature selection process is a dimensionality reduction scheme. Some feature values maybe noisy compared to others. In the feature selection process, a minimal feature set of size 𝑘 < (𝑚+𝑛) is chosen such that classification performance on a training set of feature vectors is improved. The feature selection algorithm employed here is sequential forward floating selection technique (see [47] for more on this technique). A new feature vector 𝑍= {𝑧1 ,𝑧2, … ,𝑧𝑘} is obtained when the feature selection algorithm is applied.
Match Score Generation: Let (𝑋𝑖,𝑌𝑖) and (𝑋𝑗,𝑌𝑗) be the feature vectors obtained at the two different time instances 𝑖 and 𝑗 where 𝑋 and 𝑌 represent the feature vectors derived from two different information sources. Let (𝑍𝑖,𝑍𝑗) denote the corresponding fused feature vectors.
24 Figure 3.4: Procedure adopted in [50] to perform feature level fusion.
Let (𝑠𝑥,𝑠𝑦) denote the normalized match score generated by comparing 𝑋𝑖 with 𝑋𝑗 and 𝑌𝑖
with 𝑌𝑗 respectively. Let 𝑠𝑚𝑎𝑡𝑐ℎ = (𝑠𝑥+𝑠𝑦) 2⁄ represent the fused match score obtained using simple sum rule.
To compare the fused vectors 𝑍𝑖 and 𝑍𝑗, two different distance measures are used. They are:
Euclidean distance (𝑠𝑒𝑢𝑐) = ∑𝑘𝑟=1(𝑧𝑖,𝑟− 𝑧𝑗,𝑟)2
Threshold Absolute Distance or TAD (𝑠𝑡𝑎𝑑) = ∑𝑘𝑟=1𝐼(�𝑧𝑖,𝑟− 𝑧𝑗,𝑟�,𝑡)
Here, 𝐼(𝑦,𝑡) = 1, if 𝑦 >𝑡 (and 0, otherwise) and 𝑡 is a pre-specified threshold. Thus, we see that TAD measure determines the number of normalized feature values that differ by a magnitude greater than the set threshold 𝑡. One feature level score 𝑠𝑓𝑒𝑎𝑡 is obtained from 𝑠𝑒𝑢𝑐 and 𝑠𝑡𝑎𝑑 using simple sum rule (Figure 3.4). Finally, the information at match score level 𝑠𝑚𝑎𝑡𝑐ℎ and the feature level 𝑠𝑓𝑒𝑎𝑡 are combined using simple sum rule to obtain the final score 𝑠𝑡𝑜𝑡 (Figure 3.5).
25 Figure 3.5: Information flow when data from the feature level and match score level are combined [50].
Ross et al. [50] carry out experiments in three different scenarios:
a) Fusion of Principle Component Analysis (PCA) and Linear Discriminant Analysis (LDA) coefficients of face: Two different face recognition algorithms based on PCA and LDA are combined at feature level (see [1] for details on these methods). It is observed that performance of LDA-based matcher is higher than the performance of PCA-based matcher.
In this situation applying match score level fusion is found to degrade matching performance. The proposed fusion involving the combination of feature level and match score level fusion neither degrades nor improves matching performance. Authors mention that using fusion rules other than simple sum rule could have however improved performance.
b) Fusion of R, G, B channels: Three different feature sets are generated for a face image by subjecting each color channel to LDA separately. These feature sets are then combined at both feature and match score levels. It is observed that the scheme combining feature level and match score level information performs significantly better than match score level fusion.
c) Fusion of Hand and Face Biometrics: Face and hand feature sets are combined performing multimodal fusion. The matching performance of the scheme combining feature level and match score level fusion is slightly inferior compared to match score level fusion. However, when the same experiment is conducted with different dataset, the performance of the proposed scheme is found to be superior compared to match score level fusion.
26
3.3 Score Level Fusion
In score level fusion, different biometric matchers provide match scores indicating the degree of similarity between the input and template vectors. These match scores are consolidated to reach the final recognition decision. After the sensor level and feature level information, match scores contain the richest information about the input biometric sample.
Fusion at score level provides the best tradeoff between the available information content and convenience of fusion. Therefore, this scheme is extensively studied in literature. This is also known as fusion at measurement level or confidence level.
From theoretical point of view the performance obtained by combining match scores from any number of matchers is guaranteed (on average) to be no worse than the best of the individual biometric matcher [25]. The key is to identify the appropriate method which combines the matching scores reliably and maximize the matching performance. Two guidelines for good combination of scores are mentioned in [25]. Firstly, each biometric matcher must provide a match score to the combiner. Secondly, in advance of operational use, each biometric matcher must make available to the combiner, its technical performance (such as score distributions).
Match scores generated by individual matchers might not be homogenous. For example, one matcher may produce a similarity score where a high value indicates better match whereas the other matcher may produce a dissimilarity score where a smaller value indicates better match. The match scores generated from different matchers may not be in the same range and may have different probability distributions. Because of these reasons, scores are normally normalized prior to fusion. However, some fusion methods use probability density functions (PDFs) directly and do not require normalization methods. The general flow of information in a match score level fusion taking normalization into account is shown in Figure 3.6.
Fusion methods at score level can be broadly classified into three categories [54]: density- based schemes, transformation-based schemes, and classifier-based schemes.
Figure 3.6: Match score level fusion [25].
