The Great Debate: LiDAR Or Camera-Based Systems For AVs

By John Oncea, Editor

It’s time to decide. LiDAR or camera-based systems for autonomous vehicles. As a public service, Photonics Online is happy to host a debate between the two, the results of which may surprise you.

We’re live from Pennsylvania, a key battleground state in the race for the autonomous vehicle industry’s technology of choice. In just moments, LiDAR will debate camera-based systems to determine which technology AVs should use.

We want to welcome our readers in the U.S. and around the world to our office in Erie to this, the Photonics Online autonomous vehicle (AV) technology debate. This is a pivotal moment between LiDAR and camera-based systems as they face off for the AV industry’s support. Each will make its case to the industry, hoping to become the go-to technology of automakers, engineers, and decision makers.

I’m John Oncea, editor of Photonics Online, and I will be your moderator. My job is to facilitate a debate between the two candidates and, before I introduce them, I want to share the rules of the debate with the readers. LiDAR will answer first, and camera-based systems second. A coin toss determined the order.

Each candidate will answer a question and time will be set aside for response and rebuttal at the moderator’s discretion. The opponent's microphones will be turned off during a candidate’s response so any interruption will be muted and difficult to understand.

Pre-written notes, props, or contact with campaign staff are not permitted by the debate. By accepting my invitation to debate both candidates and their campaigns agreed to accept these rules. Now, please welcome LiDAR and camera-based systems to the stage.

Dramatic musical interlude.

Let The Debate Begin

JO: LiDAR, camera-based systems, thanks so much for being here. Let’s begin the debate by making the case for you as the technology of choice for AVs, starting with LiDAR.

LiDAR: I’ve emerged as a crucial component for AVs because I create highly accurate 3D representations of the AV’s surroundings. To do this, I emit laser pulses and measure their reflections which allows for detailed environmental mapping, which is essential for safe navigation.

I also offer better accuracy and range compared to other sensing technologies like radar and sonar. I can detect objects and obstacles with greater precision, which is critical for avoiding collisions. And I can do this in various conditions. I perform well in different lighting and weather conditions, making me more reliable than camera-based systems in challenging environments.

I also can quickly process vast amounts of data in real time, enabling AVs to make split-second decisions, as well as provide a 360-degree, comprehensive view of the AV’s surroundings, improving the accuracy and quality of safety alerts.

I integrate and work well in conjunction with other sensing technologies like radar and cameras, creating robust navigation systems. Finally, I come with the potential for miniaturization and cost reduction. I’m currently developing technology that will lead to smaller, more affordable systems that can be easily integrated into vehicle designs.

While some companies, like Tesla, have opted for camera-based approaches, many AV developers consider me essential for achieving higher levels of autonomy safely. My ability to provide detailed, accurate environmental data makes me a valuable tool for ensuring the safety and functionality of self-driving cars.

JO: Thank you. So, to summarize, LiDAR can:

• map environments with sub-millimeter accuracy, providing highly detailed 3D images,
• immediately determine the distance and direction of objects, requiring less computational power than camera systems,
• work effectively in any lighting condition, including total darkness, and
• detect subtle details like pedestrian-facing directions and hand signals from cyclists.

Camera based-systems – same question.

Camera-based systems: Let me begin by saying while I do offer certain advantages for AVs, I should not be the sole technology used. Instead, a multi-sensor approach that includes cameras along with other technologies like radar and, yes, LiDAR is generally considered more robust and reliable.

That said, I do excel at visual object detection and classification, allowing AVs to interpret road signs, lane markings, and traffic signals in a way that mimics human vision and provides rich contextual information. I’m also less expensive than LiDAR, making me more economical to implement at scale.

I’m also an established technology. I am more mature and widely produced compared to LiDAR, benefiting from years of development in the consumer electronics industry.

Again, I’m a candidate that is going to unite technologies. I believe in a multi-sensor approach that combines me with radar and with my opponent, LiDAR. This fusion of technologies provides redundancy, improves accuracy across various conditions, and enhances overall safety. How? LiDAR will provide precise 3D mapping and object detection, while radar will provide long-range detection and poor weather performance.

JO: Thank you. To summarize, camera-based systems are:

• generally more affordable than LiDAR systems, making them an attractive option for mass-market vehicles,
• can interpret colors, read street signs, and see the world similarly to human vision, and
• be easily incorporated into vehicle designs, making them more aesthetically pleasing for consumer vehicles.

Spill The Tea On Your Opponent

JO: You’ve identified your strengths, take a moment now to tell me about your opponent’s weaknesses. LiDAR?

LiDAR: Look, camera-based systems have some advantages but there are several reasons why they should not be the sole AV technology, starting with weather and lighting limitations. Cameras struggle in poor visibility conditions like rain, fog, snow, or low light, which can significantly compromise safety and reliability.

They also have difficulty accurately measuring precise distances – crucial for safe navigation and collision avoidance – compared to me. They have a shorter effective range than me, something which is particularly important for high-speed driving scenarios.

My opponent is vulnerable to occlusion and objects can be partially or fully hidden from their view, potentially leading to missed detections and safety risks. And don’t even get me started on their high processing demands. You know, interpreting complex visual data from cameras requires significant computational power and sophisticated AI algorithms, which can be challenging to implement effectively.

They struggle with extreme contrasts, such as direct sunlight or brightly lit areas at night, which are common in driving scenarios. They can easily become contaminated by dirt, debris, or weather conditions, reducing their effectiveness. Finally, they may have difficulty distinguishing between similar-looking objects or structures, such as lane markings versus other linear features in the environment.

But they’re not all bad. It’s just that instead of relying solely on camera-based systems, we should maybe consider a multi-sensor approach that combines the two of us with radar. My opponent suggested this earlier, and I do agree that this fusion of the three of us will provide redundancy, improve accuracy across various conditions, and enhance overall safety.

JO: Camera-based systems? What do you see as LiDAR’s failings?

Camera-based systems: There are several reasons why LiDAR shouldn't be the sole or primary technology used. Let’s start with the fact that LiDAR systems are currently expensive, which can significantly increase the overall cost of AVs. This high cost hinders widespread adoption and makes AVs less accessible to consumers.

And, while I hate to use the word “compromised,” in certain types of weather LiDAR’s performance can be just that. Weather conditions such as heavy rain, fog, or snow can reduce the reliability and safety of AVs. LiDAR is vulnerable to interference beyond weather, too. External light sources, reflective surfaces, or other LiDAR-equipped vehicles can potentially lead to inaccurate readings or false detections.

My opponent, to put it delicately, can be bulky, something that can pose design and integration challenges for vehicle manufacturers trying to maintain aesthetics and aerodynamics. And yes, while LiDAR excels at detecting objects and creating 3D maps, it may struggle with identifying and classifying specific objects, which is crucial for making informed driving decisions.

Let’s not forget that LiDAR generates massive amounts of data that require significant computational power to process in real time, which can be challenging for onboard systems. Finally, LiDAR sensors contain moving parts and delicate components that may require more frequent maintenance and be more susceptible to damage compared to other sensing technologies.

And even if I lose this round, my fellow cameras and AI technologies are rapidly evolving and there remains a strong possibility that a combination of cameras and cheap radar could potentially render LiDAR technology obsolete.

The Choice Is Yours

JO: Thank you to both candidates, as well as to you, the reader, for your time today. It’s clear to me that both LiDAR and camera-based systems have seen significant advancements in recent years. Camera-based systems have improved in object detection and environmental understanding, while LiDAR has become more affordable, compact, and higher resolution.

Perhaps rather than running against each other the two of you should unite. Many experts and companies advocate for a combined approach, known as sensor fusion, which utilizes both LiDAR and cameras along with other sensors like radar. This multi-sensor approach provides redundancy and a more comprehensive view of the AV’s surroundings.

Ultimately, the choice between LiDAR and camera-based systems will come down to specific use cases, vehicle types, and company philosophies. While some companies like Tesla focus solely on camera-based systems, others like Waymo and many traditional automakers opt for a combination of sensors including LiDAR. As both of you continue to evolve, the debate over which is better may shift toward finding the optimal combination of sensors to achieve safe and reliable autonomous driving.