What Can Flat Optics Do For Depth Sensing?

By Ram Narayanswamy, Fellow, Head of Product Marketing, NIL Technology

Depth sensing is becoming a critical enabling technology in various applications, including autonomous robots, virtual and augmented reality, face recognition, and logistics. The technology enables devices and machines to sense their surroundings, change how digital systems understand environments, and automate processes that typically would require human handling or observation. This requires depth measurements and 3D perception.

Depth measurements are increasingly measured by Time-of-Flight (ToF). In a ToF system, the distance from the sensor to the object is determined by measuring the time it takes light to travel from a source to the objects-in-the-scene and then back to the sensor. In such applications, illuminating the scene with an array of precise dots instead of flood illumination is desirable, because it enables measurements of depth reliably over longer distances for a given amount of battery power. 

On the receiver side, it is desirable for a camera that has a low F-number and high relative illumination. This enables accurate measurements. Flat optics-based dot pattern generators and imaging lenses, customized to the VCSEL source and the ToF sensor, can make for a highly compact, efficient, and robust system. 

What are the different ways to measure depth?

Stereoscopic camera systems, structured light patterns, and time of flight are three technologies for depth sensing. The advent of vertical-cavity surface-emitting laser (VCSEL) with its narrow spectrum and sophisticated driver chips are making time of flight a desirable method for the handset, XR, and room-scale depth-sensing applications.

Stereo camera pair with active illumination

In a stereo camera with active illumination, the scene is illuminated with a random dot pattern to provide feature points on the scene. The stereo pair of infra-red cameras image the scene, and depth is extracted by how much the features of interest shift between the two cameras (disparity).

Illuminating the scene with structured light

By illuminating the scene with structured light, you have a scene that is lit up with a pre-determined pattern of dots. The depth is triangulated using the pre-calibrated spatial relationship between the projector and the infrared camera, and the position of each dot in the captured image

Time of Flight (ToF) system

In a ToF system, the scene is illuminated in infrared using a flood illuminator or an array of dots. Direct time of flight uses pulsed light and measures the time between the emitted signal and the reflected signal. Indirect time of flight illuminates the scene with a temporal periodic signal; Depth is then extracted by measuring the phase difference between emitted and reflected signal.

Design considerations of a depth measurement system 

Various design considerations need to be considered, both on the transmitter (TX) and the receiver (RX) sides.

TX side considerations

• Given a battery budget, the optical power should be concentrated in the dots while minimizing the power between the dots.
• The dot divergence should be minimized, and this is typically achieved when the design is diffraction limited. 
• The dots should maintain their local spatial relationship in a predictive manner to be accounted for algorithmically.
• In a flood illuminator, the roll-off profile should be sharp like a step-function so that power is not wasted outside the field of illumination.
• The field of illumination should be large.

RX side considerations

• The lens should have a low F/number to capture the faintest dot reflected from the scene.
• The lens should have high relative illumination to the full field of view.
• The lens should have spatially consistent lens distortion.

Overall System considerations

• The system should offer low module height. 
• The system should deliver consistent performance over operating temperature and other environmental conditions.

What are flat optics? 

Flat optics is a new class of optics built using diffractive optics (DOE) and meta-optics (MOE) technology.

Diffractive optics are optical components made of structures with dimensions comparable to the optical wavelengths set in complex arrangements. They can manipulate the light’s phase and amplitude to create the desired output pattern with unique functionality. DOEs can also be made flat, thin, and multifunctional. This combination is ideal for NIR solutions used in 3D sensing and LiDAR. 

Meta optics, or metalenses, are created using meta-atoms that can bend and steer the light even more efficiently than DOEs. They are entirely flat but effectively produce the same results as traditional refractive lens elements with a curved surface. A single metalens can replace several refractive lenses stacked on top of each other. These reduced meta-elements have multiple benefits: multiple functionalities in a single element, easier assembly, and more compact module form factors.

Flat optics have the potential to disrupt numerous applications, such as simplified time-of-flight (ToF) systems, lidar, ultra-compact near-infrared (NIR) cameras used for eye-tracking, driver monitoring systems, object detection/avoidance in robotics, shortwave infrared (SWIR) machine vision, and wearable medical imaging.

What can flat optics do for depth measurement systems?

The depth measuring system typically consists of a transmit side and a receive side. The transmit side illuminates the scene. The receiver camera images the scene, which is used for depth extraction. The RX and TX are typically co-designed for compatibility. Flat optics can play a significant role in the following ways:

• TX Dot projectors 
  • Collimate and fan-out a VCSEL array and turn it into a large array of dots for ToF.  
  • Project a random dot pattern for active stereo systems.
• TX Diffusers
  • Illuminate the entire scene with flood-illuminator.
• RX IR camera optics 
  • Capture the scene optimized for optimum depth extraction.
     

Flat optics offers many options both in the TX and RX modules of depth measurement systems. Functionalities like collimation and fan-out can be implemented in a single element. The DOE and MOE are fabricated on glass and amorphous silicon to make them robust and cost-effective.

How is NILT moving this technology toward mass production?

NIL Technology (NILT) is an advanced optics company focusing on commercializing flat-optics for near-infra-red consumer applications. NILT has extensive design, modeling, fast-prototyping, and mass production capabilities for MOE and DOE elements and components. All processes are in-house. This large toolbox of abilities allows us to rethink how optical systems are made and harvest the benefits of significantly simpler, smaller, and better-performing systems. NILT will make flat optics readily available. 

About the Author

Ram Narayanswamy is Strategic Technology Advisor at NIL Technology, a Coppenhagen-based optical solutions company that designs, develops, and manufactures optical elements and components using high-precision nanoscale features.