Shown is spectral imaging of a scene containing objects with different colors, as well as the letters NCSU, which contain different polarization states. [Image: Ali Altaqui]
An organic photodetector, inspired by mantis shrimp, steps beyond nature’s innovation to combine hyperspectral and polarization imaging in a single pixel, a team led by researchers at North Carolina State University (NCSU), USA, reports (Sci. Adv. doi: 10.1126/sciadv.abe3196).
This compact, powerful sensing modality could become an important new tool with broad potential for diverse areas of science and technology, including AI-driven classification tasks, chemical sensing and deep-space astronomical observing, say co-authors Ali Altaqui, Brendan T. O’Connor, Michael Kudenov, and colleagues in the NCSU Departments of Electrical and Computer Engineering and Mechanical and Aerospace Engineering.
The imaging system they developed takes its cue from nature, setting aside inorganic materials in favor of stacked polarization-sensitive organic photovoltaics (P-OPVs) and folded polymer retarders (FRs). The sensor reportedly can pick up 15 spectral channels over a 350-nm bandwidth, while the detector is reported to register simultaneously four spectral and three polarization channels.
Mantis shrimp inspiration
While the eyes of invertebrate animals have evolved a variety of different visual capabilities to serve in their ecological niches, Altaqui says, “mantis shrimp vision puts everything to shame.” The marine crustaceans can detect up to 12 colors, have full polarization vision and can rotate their eyes in three degrees of freedom—yaw, pitch and torsion. Plus, their compound eyes are equipped with super achromatic quarter-wave plate elements, enabling them to modulate and detect light efficiently.
The NCSU team thus “took inspiration from the structural and functional features of the mantis shrimp eye,” according to Altaqui. The compound eye of these stomatopods, he explains, contains spectrally selective elements (pigmented retinular cells) and polarization-sensitive elements (tiered microvilli) vertically stacked along a single optical axis.
“As light propagates deeper into the stack, the mantis shrimp extracts more spectral and polarization information,” Altaqui says. “Analogously, our organic sensor comprises spectrally selective elements, the folded retarders, and polarization-sensitive elements, the organic photovoltaics, vertically stacked along a single optical axis.” The spectral and polarization information are detected in a similar manner as the mantis-shrimp eye, he says, enabling simultaneous spectral and polarimetric detection.
In honor of the colorful diminutive sea creature, the researchers have named their device the stomatopod-inspired multispectral and polarization-sensitive (SIMPOL) sensor.
Packing in the pixels
While inspired by stomatopod image sensors, Altaqui believes that the new device actually goes beyond them, by combining multispectral and polarization imaging in a single pixel. The research team—which also includes researchers from the University of North Carolina, Chapel Hill, USA, and KAIST, Republic of Korea—is convinced that its compact organic photodetector would radically outperform existing sensors, including smartphone cameras, and would be capable of acquiring more colors with a higher spatial resolution—that is, higher number of pixels.
Kudenov explains, “if you want nine colors and have a 9-megapixel silicon camera, you could combine nine of your pixels into one so-called ‘superpixel.’ But at the end of the day, you're still left with a 1-megapixel image.”
With the organic materials in the team’s SIMPOL sensor, Kudenov says, “we could have nine colors with nine megapixels each, 81 megapixels total, in which all of the color pixels are arranged directly on top of each other on the camera. We do not have to sacrifice the total number of image pixels that we want to get color, as is done in many of today's camera sensors. We can also get polarization information as a bonus,” he says.
Inside the SIMPOL sensor
The researchers believe that their system is potentially far more compact and functional than the current state of the art for combined imaging modalities, spectral polarization imaging (SPI). While SPI plays a crucial role in biomedical imaging (for example, in imaging of cancer), the hardware is bulky, and suffers from problems in image quality and color limitations owing to its use of vertically stacked, inorganic p-n junctions, the authors note.
This photo shows an experimental prototype of the sensor. The tandem structure consists of six polarization-sensitive organic photovoltaics and four polymer retarder films cascaded in series along the same optical axis. [Image: Ali Altaqui]
The SIMPOL sensor, by contrast, employs semitransparent P-OPV detectors with compact FRs. Spectral detection is achieved, the researchers say, by using the FRs with panchromatic P-OPV cells to form a filter that enables color selectivity through polarization interference.
“The use of polarization to control the spectral distribution of light, through a tandem detector structure overcomes the spatial and temporal sampling errors of current SPI sensors and enables light to efficiently transmit further into the tandem stack,” the authors write.
And because the light’s spectrum is detected through polarization control, the incident polarization state can be measured, the authors note. In all, they say, the sensor’s design freedom allows for broad control of the device’s spectral and polarimetric sensitivity, with the bonus of polymer semiconductors rather than light-squelching inorganic silicon.
Applications on Earth and beyond
Kudenov says the team sees potential applications for the device in agriculture, and has been working with faculty at NCSU’s College of Agriculture and Life Sciences to develop methods for screening or grading plants using machine learning algorithms to measure crop insect damage, fungal spread, size and other parameters. The algorithms use both color and spatial information for classification, Kudenov says, but can also work for other types of information, such as polarization, which might improve the classification accuracy.
By combining such algorithms with advanced sensor technology, he adds, one hope is that plant breeders will more easily and quickly be able to create next-generation plant cultivars that are, for example, more drought and heat tolerant. “This is especially important given that climate resiliency in our food supply will be key for the coming decades,” he says.
The team also envisions applications for its organic photodetector in chemical imaging, where, according to Altaqui, speed, high spatial resolution and versatile spectral selectivity are paramount. And he says that it might be used in astronomy, to acquire spatially resolved spectra of objects and clusters at great distances from the Earth, for studies in the distribution, collision and evolution of galaxies, for compositional studies of distant planets, and to quantify the sun’s radiation and magnetic field.
Meeting future challenges
Altaqui says one of the notable challenges for the NCSU team’s sensor is that its spectral performance can be affected by off-axis light due to the stacked FR elements. However, he says, recent research has demonstrated better off-axis performance with the use of liquid-crystal retarders.
Future challenges, Altaqui says, include monolithically integrating the sensor in a compact design and realizing a pixelated 2D array.
“We are working on another paper related to this research,” Altaqui says, “employing liquid-crystal multi-twist retarders as well as a different modality to detect the polarization states of light by leveraging additional mantis shrimp eye features.”