An Eye toward Multispectral Imaging

The mantis shrimp is a marvel of nature that represents a truly multispectral scanning system.



Unlike anything that humans have engineered, the mantis shrimp can independently direct two optical systems in two different directions, detect targets, then scan the targets to gather spectral information. It can also simultaneously measure four linear and two circular polarization components as well as monocular range to the target.

Multispectral imaging systems

Multispectral imaging systems use a limited number of spectral bands defined by an optical filter assembly. The filters are arranged like the fingers on your hand and are tuned to specific wavelengths. The choice of optical filters depends on the objects that are under investigation. Other filters are used for atmospheric correction, to indicate cloud cover, cloud height and atmospheric water vapor concentration; others have been used for atmospheric light scattering correction.

A space-based remote sensing system images the Earth or ocean onto a two-dimensional detector array. As the instrument is translated, each element in the scene passes across the optical filter assembly, in proximity to the detector array, gathering spectral data. The detector arrays that are often used are silicon or high-temperature mercury cadmium telluride (HgCdTe), to cover the visible and near-infrared range. In red and near-IR (NIR) regions, indium gallium arsenide might be is used; in the mid-wave IR (MWIR) and long-wave IR (LWIR), HgCdTe detectors are a good choice.


imageOptical configuration of a pushbroom multispectral scanner.

The uniqueness of the NASA LANDSAT radiometric multispectral filter assembly comes about from many factors. These include the precision cutting and bonding of the individual filters, coplanarity of the filters, thickness tolerances of each filter, tolerances on the filter's central wavelengths, spectral bandwidths, sharpness of the cut-on and cut-off slopes, out-of-band blocking, low scattering and elimination of spectral crosstalk. In a push-broom optical configuration, a two-dimensional detector array is orientated with its long axis in the cross-track direction—in other words, orthogonal to the direction of travel of the instrument. Each detector pixel in this cross-track defines a spatial element in the scene.

While many LANDSAT "radiometric" multispectral filters have sharp cut-on and cut-offs, with little or no spectral overlap, there may be other applications where it is necessary to distinguish subtle color differences. For example, low color contrast environments or camouflage surroundings may be better served with more spectral bands with filters that have substantial spectral overlap.

Nature's multispectral imager

The mantis shrimp (Odontodactylus scyllarus) has a scanning imaging multispectral optical system that is unique and that inspires biomimicking. Two apposition compound eyes are mounted on mobile stalks that are independent of one another. The eye movements involve rapid sequencing or redirection for targeting prey (i.e., saccadic movement) after which they stabilize the image and slowly track the object. Of special interest is its biological image processing, which is often overlooked. In the mantis shrimp, the upper and lower hemispheres are devoted to motion and shape detection. The scanned, central band is for color detection, ranging and revealing linear and circular polarization. Each eye can target a different part of its visual field, and all of this is done in real time.


imageLANDSAT multispectral optical filters.


imageMantis shrimp (O. scyllarus)รข€….

The mantis shrimp eye has a band of six ommatidia (components of compound eyes that contain photoreceptors) that separate the upper and lower hemispheres of the eye. Rows one through four in the midband are for color vision. Each of the four photoreceptor bands has three different visual photo pigments. Combined with carotenoid compound color layers to tune and shape the spectral bands, eight visible bands (400 to 750 nm), along with four ultraviolet (UV) ones, create 12, almost equally spaced 40-nm wide bands. To capture color, the midband must be scanned at roughly 40° per second to allow for the effective transfer of information. Covering the spectrum from 310 to 710 nm, O. scyllarus is a true multispectral scanning system that is able to distinguish subtle color differences.

Rows 5 and 6 of the midband are for detecting left-hand and right-hand circularly polarized light. Sensitivity to this type of light depends on having a good quarter-wave plate of the correct thickness. Manmade quarter-wave plates have refractive indices that depend on the wavelength of incident light. Thus, they are good for only a very limited spectral band. The mantis shrimp is, however, sensitive to circularly polarized light over the entire visible spectrum, thereby completely outperforming manmade systems components. Its ability to detect and analyze the scene for the polarization components enhances the contrast in a liquid environment, as any microscopist knows, in attempting to image living cells in a liquid.

The upper and lower hemispheres of the eye have only two visual pigments. These eye regions are likely responsible for the panchromatic sense of these marine crustaceans. In some species, they may also mediate linear polarization sense. Overlapping by as much as 20° to 30°, the two hemispheres may help with motion detection and the monocular judgment of distance. In order for the upper and lower hemispheres to function effectively during detection, the eye must be stabilized. To gather spectral and polarization information, the center band must be scanned. The paradoxical situation of stabilizing the upper and lower hemispheres and scanning the midband is solved by time sharing these functions of the eye.


imageSpectral sensitivity of O. scyllarus R1-R7 cells.


imageColor bands of WorldView-2.

The WorldView-2 multispectral imager built by DigitalGlobe and Ball Aerospace is a spectral imager that comes close to mimicking the mantis shrimp spectrally. In this instance, the bands are not spectrally isolated, as are the radiometric bands of other multispectral instruments; rather, they have some overlap. This should result in improved hue discrimination. Launched on 9 October 2009, this seven-color band instrument also has a panchromatic band. The two graphs show the normalized spectral transmissions for the mantis shrimp and six WorldView "color" bands. The spectral axes of the two curves have been aligned at 350 and 650 nm for comparison. Both O. scyllarus and WorldView-2 have several color bands that come close to corresponding; both have some spectral overlap of the bands, and both have a panchromatic band (not shown for the mantis shrimp).

In summary, the linear scanning capability of the mantis shrimp eye is analogous to the multispectral scanners designed by humans, but the mantis shrimp can do much more. It is capable of monocular range finding, and it has the ability to detect linear polarization and both-handedness of circular polarization—both of which are advantages in low-contrast conditions. When it comes to the sophisticated engineering of multispectral systems, nature appears to have the upper hand over us.

H.D. Wolpert is director of engineering at Bio-Optics in Los Angeles, Calif., U.S.A.

References and Resourcess

>> W.T. Cronin et al. "The unique visual system of the mantis shrimp," American Scientist, 82.
>> T.H. Chiou et al. "Circularly polarized vision in a stomatopod crustacean," Current Biology, 18, 429-34.
>> P.A. Ensminger. Life Under the Sun, R.R. Donelley & Sons (2001).
>> D.S. Falk et al. Seeing the Light, Optics in Nature, Photography, Color, Vision and Holography, John Wiley & Sons, 1986.
>> J. Marshall et al. "Stomatopod Eye Structure and Function: A Review," Arthropod Structure & Development," 36, 420-48.
>> N.W. Roberts et al. "A biological quarter-wave retarder with excellent achromaticity in the visible wavelength region," Nature Photon. 3 (2009).
>> WorldView-2 White Paper, "The Benefits of the Eight Spectral Bands of WorldView-2," August 2009.

Publish Date:

An Eye toward Multispectral Imaging

The mantis shrimp is a marvel of nature that represents a truly multispectral scanning system.

Become a member or log in to view the full text of this article.

OSA Members get the full text of Optics & Photonics News, plus a variety of other member benefits.

Publish Date:

Add a Comment