What the eye doesn’t see : using infrared to improve face recognition of individuals with highly pigmented skin
Date
2022-12
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Stellenbosch : Stellenbosch University
Abstract
ENGLISH ABSTRACT: Face recognition technology has become commonplace in security and access
control applications. However, their performance leaves a lot to be desired
when working with highly pigmented skin tones. One reason for this is the
training bias introduced by under-representation in existing datasets. The
other is inherent to pigmentation – darker skins absorb more light and therefore
could reflect l ess d iscernible d etail i n t he v isible s pectrum. We s how how
this can be enhanced by incorporating the infrared spectrum, which electronic
sensors can perceive. We collect a database with images of highly pigmented
individuals, captured using the visible, infrared and full spectra We fine-tune
state-of-the-art face recognition systems and compare the performance of these
three spectra. We also assess the impact of narrow and wide cropping, different
facial orientations, and sunlit and shaded conditions. We find a marked
improvement in the accuracy and in the AUC values of the ROC curves when
including the infrared spectrum, with performance increasing from 97.5% to
99.1% for highly pigmented faces. Including different facial orientations and
narrow cropping also improves the performance, and can therefore be deemed
as recommended best practices. Analysis of the activation maps of the CNNs
finds t hat fi ne-tuning mo dels ac tivate mo re ge nerally ov er al l re gions of the
face while models with pre-trained weights, focus on fewer features with higher
activation intensity values over those regions. In both cases, the nose region
appears as the most important feature for face recognition for highly pigmented
faces.
Description
Thesis (MEng) -- Stellenbosch University, 2022.
Keywords
Infrared spectroscopy, Human face recognition (Computer science), Computers -- Access control, Human skin color, UCTD