Getting started with on-sensor polarization
Many vision systems struggle to overcome the effects of reflections and glare on reflective surfaces like glass, plastic and metal. Sony’s newest sensor technology can solve this problem with its pixel-level polarizer structure. This technology enables Sony polarized sensors to detect the amount and angle of polarized light across a scene.
IMX250MYR: USB3, GigE
Introducing the IMX250MZR and the IMX250MYR
Like wavelength, polarization is a key property of light. Sensing polarized light can enable entirely new machine vision applications and improve the performance of existing ones. The FLIR Blackfly S Machine Vision camera, featuring Sony’s new IMX250MZR (Mono) and IMX250MYR (Color) image sensors with polarizing filters, makes it easier than ever to take advantage of polarized light for your application.
This guide provides everything you need to get started with the Blackfly S. It covers four main topics:
- What is polarized light?
- What can you do with it?
- How does the IMX250MZR work?
- How can polarization data be interpreted?
Enabling new applications with polarized light
The IMX250MZR’s on-sensor polarimetry, high-speed global shutter readout, and compact, low-power package is ideal for a wide range of applications.
Unmanned Ariel Systems (UAS) typically operate outdoors in uncontrolled lighting conditions. The IMX250MZR enables quick adjustment of polarization parameters to compensate for changing lighting conditions and the relative movement and orientation of the UAS.
Polarizing filters eliminate unwanted reflections, making it easy for traffic enforcement systems to see through car windshields to detect seatbelt and mobile device violations. On-sensor polarization enables quick system installation and automatic adjustment based on time of day and direction of vehicles.
Polarimetry is ideal for detecting man-made objects that would otherwise be difficult to identify using traditional visible or thermal imagery. Camouflaged vehicles or structures still reflect polarized light oriented parallel to the surface that stands out clearly in AoLP mode.
Decluttering images by removing unwanted glare and reflections can simplify the training of deep learning systems. This is particularly useful in the high-glare environments encountered by autonomous vehicles and marine Unmanned Surface Vehicles (USVs).
The high resolution and low read noise of the IMX250MZR enable a wide field of view to be analysed with standard microscopy equipment. For example, the polarizing properties and optical activity of biological compounds can differentiate healthy and diseased tissues.
What Is polarized light?
Light is a transverse electromagnetic wave. As it propagates, it oscillates perpendicular to the direction of propagation. Most light sources emit unpolarized light, with all the waves oscillating at random angles. When light is aligned so that most waves oscillate at a common angle, it is said to be polarized. Circular polarization is also possible, though it is beyond the scope of this guide.
Unpolarized light oscillating at random angles vs. Polarized light aligned on an angle
Polarizing filters form the foundation of most polarized light technologies. By aligning a series of narrow slits, polarizing filters pass light that is oscillating parallel to the slits while blocking light oscillating perpendicular to them.
The polarizing filter passes the yellow beam that is aligned to the angle of the slits, and blocks the blue beam aligned perpendicular to them.
Industrial applications frequently rely on a pair of polarizing filters; one that creates a polarized light source and another that passes only polarized light aligned to a specific orientation. These systems typically require precisely aligned filters and highly controlled lighting. They are only sensitive to one angle of polarized light.
As a polarizing filter is rotated, the intensity of the light it passes will increase as it comes into alignment and decrease as it is moves beyond the angle of alignment.
As a polarizing filter aligned to the angle of the blue wave is rotated, it begins to block the blue wave and pass the orange one.
When plotted, the change in intensity relative to polarizer orientation is like a sine function. The ratio between the highest and lowest intensities is called the extinction ratio.
Due to the transverse nature of light, angles of polarization cannot exceed 180°. As the slits in a polarizing filter are all parallel, rotating a filter by 180° will return it to its original orientation. This explains why the intensity peaks and falls off twice as the filter is rotated by 360°.
How does light become polarized?
Light can become polarized when emitting directly from a coherent source, passing though a polarizing filter, or reflecting off a non-metallic surface. The angle of polarized light reflected off water or a polished surface is parallel to surface’s angle.
The angle of polarized light can change as it passes though certain optically active materials, such as biological molecules and pharmaceuticals.
Introducing the IMX250MZR
Sony’s new IMX250MZR sensor is based on their popular five-megapixel IMX250 Pregius global shutter CMOS sensor. On the IMX250MZR, each individual pixel has its own polarizing filter. These filters are oriented to 0°, 45°, 90° and 135° and arranged in repeating two-pixel blocks.
Each pixel’s polarizing filter (C) is coated with an anti-reflective layer (B) and is positioned between the microlens (A) and the light sensitive photodiode (E).
The IMX250MZR sensor has features that minimize the impact of reduced quantum efficiency (QE) resulting from adding polarizing filters to pixels. The polarizing filters of the IMX250MXR have a 4:1 extinction ratio, which is high enough to deliver accurate polarimetric data without blocking cross-polarized light. This ensures that when filter alignment passes a minimal amount of light, enough will reach the light-sensitive photodiode to capture useful images. Like the IMX250 it is based on, the IMX250MZR has a very low read noise of 2.3e-. This allows it to capture low-noise images even in challenging conditions requiring gain to compensate for reduced QE.
The IMX250MZR can be used in applications that rely on a fixed polarizing filter and provide high-speed, affordable on-sensor polarimetry for new applications. Systems that rely on multiple cameras and filters behind a beam-splitting prism, or a single camera with a rotating filter or filter wheel, are large, complicated, and slow. By simultaneously sensing the angle and intensity of all polarized light across the sensor, the BFS-U3-51S5P-C delivers increased speed and greatly reduced size, mass, and power consumption compared to existing solutions.
Interpreting polarization data
To characterize the polarization parameters of light requires measurements from all four angles of polarization. To achieve this for each pixel on the sensor requires an interpolation process where data from adjacent pixels is combined. This is analogous to how data from adjacent red, green, and blue pixels is combined on color sensors to product RGB values for each pixel. This process is supported by FLIR’s Spinnaker SDK.
The four Stokes parameters are a convenient way of describing the polarization state of a light beam. Stokes parameters are the basis of many polarimetry calculations and algorithms. Users wishing to adapt existing techniques or create their own should be familiar with how to determine the Stokes parameters on the IMX250MZR.
S0 is the intensity of light beam. On the IMX250MZR, this is calculated by adding the intensities of the vertically and horizontally polarized pixels.
S1 is the difference between the horizontal and vertical components. Positive values are horizontally linearly polarized while negative ones are vertically linearly polarized.
S2 is the 45° component. Positive values are 45° linearly polarized. Negative values are -45° linearly polarized.
S3 is the circular polarization component. Though this parameter is not measured by the IMX250MZR, it can often be accurately estimated. In outdoor and passively illuminated environments, S3 is assumed to be 0 since sunlight is unpolarized and reflection or scattering of sunlight only imparts linear polarization. In environments with controlled active illumination, it is possible to eliminate any sources of unpolarized light, making it possible to characterize the circular component.
The S1, S2, and S3 Stokes parameters that ignore the intensity of light are frequently represented as a set of spherical coordinates mapped to a Poincaré sphere. This notation is a convenient way of understanding the relative contribution of each of the polarized components of a light beam to its overall polarization state.
Poincaré sphere. Outdoors, the S3 component can reliably be assumed to be 0. The intensity, Ip is equal to 1.
Stokes parameters can be used to compute polarimetric parameters and greatly enhance visible spectrum imagery.
Degree of Linear Polarization
The Degree of Linear Polarization (DoLP) is the most basic way to interpret polarization data. DoLP is the proportion of light that is polarized at a given pixel. A perfectly polarized light source would have a DoLP of 100%, while unpolarized light would have a DoLP of 0%.
Angle of Linear Polarization
The Angle of Linear Polarization (AoLP) is the average polarization angle of the light at a given pixel. If the DoLP is low, only a small amount of light will be polarized. In this case, the resulting AoLP values will show clear spatial and temporal noise. This is analogous to a low intensity signal being amplified with high gain. As the DoLP increases, AoLP values will become less noisy.
Using the polarimetric measurements, FLIR’s Spinnaker SDK can dynamically reduce reflections from non-metallic surfaces.
Combining Polarization and Color
The IMX250MYR sensor adds a color filter array to the sensor below the polarizing filters. This sensor uses a unique quad-Bayer pattern which prioritizes spatial resolution of the polarization domain over spatial resolution of color information.
Fig. 17. RGB pixels are rearranged into 2x2 “super-pixels”. Each super-pixel has one polarizing filter of each orientation.