Biometrics used for video surveillance identification is generally based on soft biometric (SB) features. SB features provide categorical information about an individual. While usually not sufficient to identify an individual reliably and uniquely, they can help with filtering large surveillance databases by limiting the number of entries for each query. SB features can be classified as:
- Physical (skin color, eye color, hair color, presence of beard, presence of moustache, body shape and dimensions such as height, weight)
- Behavioral (gait, movement patterns)
- Adhered (tattoos, upper and lower-body clothing, accessories)
Extracting robust SB features from surveillance video is a challenging task since video/image quality is generally unreliable due to surveillance camera characteristics (resolution, framerate, lenses, view angle) and environmental factors (changes in illumination, occlusion, shadows, etc.). For robust biometric feature extraction, the surveillance system must guarantee high quality images. To help improve SB feature extraction, a system can use EMCCD (electron-multiplying charge-coupled device, for high dynamic range), high quality lenses, rolling shutter distortion compensation, debluring, superresolution and noise reduction.
SB feature extraction should start once a human is detected and tracking has started. When an SB feature is extracted, it can be used to narrow down the search area of the database.
The human position can be detected to analyze the face location and apply face recognition. There are a number of robust methods that may be used for face recognition such as: Gabor wavelets, histogram of oriented gradients (HOG), modified scale invariant feature transform (SIFT) or 3D face recognition.