(Left) Frontal and profile views of the same subject as observed by thermal, visible, and polarimetric imagers, illustrating signature differences. (Right) Illustration of 3-D face models reconstructed from a single set of polarimetric thermal images. [Polarimetric imagery collected using sensor from Polaris Sensor Technologies Inc.]
The absence of natural illumination during night-time surveillance operations prevents covert identification of persons of interest when using traditional, visible-spectrum imagers. Passive thermal imaging that acquires heat naturally radiating from facial skin tissue can be used for day- and night-time face recognition; however, thermal face images must be matched to biometric databases and watch lists, which mostly contain only visible-spectrum imagery. Therefore, identification requires specialized cross-spectrum thermal-to-visible recognition algorithms designed to bridge the large modality gap between visible and thermal facial signatures.
In 2015, we developed a methodology for thermal-to-visible face recognition based on partial least squares (PLS) regression.1 The PLS-based method attempts to find a set of components (latent vectors) from a matrix of descriptor variables that best predict the corresponding univariate response variables. The latent vectors are found with maximal covariance by computing a normalized weight vector via an optimization algorithm. Thermal cross-examples are introduced to the PLS model building to implicitly learn the mapping between visible and thermal domains.
Building on this work, we introduced and developed a novel framework for acquiring and exploiting the polarization state of long-wave infrared emissions to increase the natural degree of correlation between visible and thermal face signatures.2,3 A polarimetric imager acquires thermal and polarimetric information, from which geometric and textural details of the face can be computed and correlated with their reflected-light counterparts. These details are not available in conventional thermal imagery. We improved thermal-to-visible face recognition accuracy by more than 20 percent when fusing complementary features derived from polarization-state information.
Additionally, we developed a method to extract a 3-D topological facial surface from the polarimetric IR images.4 We manipulate the polarization state, re-creating local surface slopes using the Fresnel transmission equations, and then perform numerical optimizations of the surface relief using integration techniques. A 3-D surface model can provide pose invariance for facial recognition by correcting off-angle subject imagery and performing 3-D-to-2-D matching.
This work represents a significant departure from previous methods of obtaining information from thermal face images, in that we exploit the natural properties of the polarization of emitted thermal radiation to enhance cross-spectrum face recognition performance.
Researchers
Nathaniel Short, Booz Allen Hamilton, USA
Shuowen Hu, Kristan P. Gurton, Alex J. Yuffa, Prudhvi K. Gurram and Gorden Videen, U.S. Army Research Laboratory, USA
References
1. S. Hu et al. J. Opt. Soc. Am. A 32, 431 (2015).
2. K. Gurton et al. Opt. Lett. 39, 3857 (2014).
3. N. Short et al. Opt. Lett. 40, 882 (2015).
4. A. Yuffa et al. Appl. Opt. 53, 8514 (2014).