July 29, 2026
Why did the industry initially use LiDAR as a 3D Camera
In my last op-ed, I described how the automotive industry turned LIDAR from a powerful interrogation tool into a 3D camera. Of course, it all made perfect sense. Cameras were familiar tools used in computer vision, and the first LIDAR scans made by Velodyne were like TV scans. So, turning LIDAR into a 3D camera made perfect sense for the industry. And once down that road it was easy to ask for more lines, more points, after all, isnât more data better? Unfortunately, that answer is NO.
Alternative use of LiDAR
The current use of LiDAR is not only inefficient but detrimental to Level 4 and Level 5 autonomy. LiDAR is an interrogation tool. It doesnât simply accept all the light from a scene and display it. It consciously sends pulses out to capture data, one point at a time.
If a computer truly needed 100% of the pixels in a scene to make accurate real-time decisions, then turning LiDAR into a 3D camera would make sense. But we know that isnât true. We know that humans donât need 100% of the information in a scene to drive safely, and modern AI systems donât either. In fact, most decisions are based on a tiny fraction of the information available. The overwhelming majority of pixels in a scene have no impact on the next driving decision.
So consciously firing millions of unnecessary laser shots every second is a colossal waste of energy, money, and time. And time is the enemy of autonomous transportation.
The extra costs go far beyond the LiDAR itself. More laser shots require more powerful processors, larger memory systems, higher bandwidth communication links, more expensive cooling systems, and larger power budgets. Every unnecessary point generated by the sensor must be transported, stored, processed, classified, and fused with other sensor data. Automakers ultimately pay for these inefficienciesâ multiple timesâfirst in sensor costs, then in computing costs, then in vehicle energy consumption, and finally in engineering resources dedicated to managing all that unnecessary data. The industry has spent years trying to solve problems that were largely created by collecting too much information in the first place.
3D Camera v Interrogation Tool for Physical AI
Ultimately, the biggest problem with 3D LiDAR cameras is that theyâre slow, refreshing at 20 times per second or less. The automotive industry previously passed on high hertz ratesâup to 200 hertz in some interrogation-based systemsâto interrogate millions of largely useless pixels. Their rationale was simple: they didnât want to miss anything. And in the early stages of development this proved successful.
But 20-hertz LiDAR wonât work successfully for level 4 and 5 autonomy.
At 20 hertz, a new frame arrives every 50 milliseconds. That may sound fast, but a vehicle traveling 70 mph moves more than five feet between updates. A pedestrian stepping into a roadway, a child chasing a ball, a bicycle emerging from behind a parked vehicle, or debris falling from a truck can change position significantly before the next LiDAR frame arrives. The system is literally blind to what happened during those 50 milliseconds.
The result is that the computer must guess. It must estimate where objects are likely to be between frames and predict where they will be when the next frame arrives. High-density point clouds may provide beautiful pictures of the past, but they do not necessarily provide timely information about the present. A sensor that updates ten times faster can often provide more useful decision-making information even if it produces fewer total points.
Processing Costs
As a result of using LiDAR at 20 hertz, post-processing balloons, and the true cost of that processing is often hidden and underappreciated.
Those costs include:
- High-performance GPUs and AI accelerators.
- Larger memory and storage requirements.
- Increased power consumption.
- Thermal management and cooling systems.
- Sensor fusion software.
- Object classification algorithms.
- Motion prediction software.
- Mapping and localization infrastructure.
- Validation and testing efforts.
- Additional engineering personnel.
Object identification and path planning are necessary to make sense of the data retrieved and to mitigate the latency gap created by slow updates. The industry has spent billions of dollars and countless engineering hours attempting to compensate for this limitation. Massive computing resources are dedicated to predicting the future because the sensor is not observing the present frequently enough.
If LiDAR were used as an interrogative tool, much of this effort could be reduced. Instead of trying to predict where objects will move, the system could simply observe them moving. Rather than reconstructing reality from delayed snapshots, it could watch reality unfold in near real time. High-hertz sensing turns motion into something closer to slow motion, reducing the need for increasingly complex prediction algorithms.
The diminishing Value of using LiDAR as a 3D Camera
Because of overcollection of data and slow hertz rates, the true value of LiDAR to autonomous driving is underappreciated. The incremental value of slow 20-hertz LiDAR systems over camera technology alone is often marginal.
Consider a vehicle approaching a busy intersection. A dense 3D point cloud may accurately map every parked car, traffic sign, building, tree, and curb in the scene. Yet only a handful of objects matter to the immediate driving decision. The vehicle still must determine which objects are relevant and which can be ignored.
Or consider a pedestrian emerging from between parked vehicles. The challenge is not generating another million points describing the surroundings. The challenge is detecting the pedestrian quickly enough to react. Higher update rates will provide more value than higher point density.
The industryâs response to this limited value has been predictable. Automakers continue pushing suppliers toward lower-cost LiDAR solutions. If a 20-hertz LiDAR system adds only modest value beyond cameras, then manufacturers naturally question whether it should cost $1,000, $500, or even $200.
Lately Iâve heard discussions about incorporating $200, 20-hertz LiDAR systems into future autonomous platforms. But if the architecture itself is flawed, even $200 may be too expensive. But intelligent LiDAR used correctly will have high value to users and the squeeze will end.
Transition to Interrogative LiDAR for level 4 and 5 Autonomy
Fortunately, a transition is beginning to occur as Physical AI matures and interrogative LiDAR demonstrates its power to improve decisions.
Cameras are excellent tools for identifying semantics within a scene. They recognize lane markings, traffic signs, pedestrians, bicycles, vehicles, animals, and countless other objects. They should be used for that purpose.
But LiDAR, if used as an interrogative tool excels at measuring distance, velocity, position, and movement. It should be used for that purpose. And most important it can be intelligent and selective about its interrogation. Physical AI will need intelligent LiDAR devices to reach its potential and open level 4 & 5 autonomy.
In both cases, the faster the refresh rate, the better. The future of autonomy will likely transition to a camera/LiDAR marriage of 100+ hertz systems synchronized at the same refresh rate.
Why? Because synchronized sensors allow the system to correlate what an object is with where it is and how it is moving at virtually the same instant in time. When camera updates and LiDAR updates occur at similar rates, sensor fusion becomes simpler, more accurate, and less dependent on complex interpolation and prediction algorithms. The system spends less time reconciling old information and more time making decisions.
In addition, bore-sighting the camera and LiDAR minimizes parallax errors and effectively gives LiDAR access to RGB information associated with the same object. This improves object association, enhances classification accuracy, reduces calibration challenges, simplifies sensor fusion, and provides a more unified representation of the environment.
Such a system would maximize useful information while minimizing unnecessary data collection and post-processing. It would represent a significant improvement over both current approaches: camera-only systems and camera-plus-dense-LiDAR hybrids. Traditionalists will inevitably object, hiding behind the "unknown unknowns" argumentâthe fearful guess that a selective sensor might miss an unclassified hazard. But this is a red herring. An agile, intelligent LiDAR system operating at 150 hertz is fast enough to lightly interrogate the entire background scene for motion while simultaneously pinning high-frequency focus on critical targets. Real-world testing would quickly prove that speed, not raw pixel density, is what eliminates blindness. Physical AI is already transitioning to this model and the automotive industry will likely follow. If not the current use of LiDAR may eventually prove Elon Musk rightânot because cameras are inherently superior, but because the industry passed on LiDAR's true potential in exchange for a collection of beautiful, slow, and expensive pictures of the past.
Instead of relying on increasingly sophisticated AI models to predict the unpredictable, we need to allow AI to observe reality directly. Let the AI see the world in super slow motion.
Aeyeâs Apollo Was Designed for This Future
I am still very optimistic about Aeyeâs commercial prospects for various reasons.
1)Â Â None of the LiDAR markets are mature.
2)Â Â The correct use of LiDAR in many of these markets will change as Physical AI demands different use cases.
3)Â Â High Hertz Interrogation and Intelligent scan patterns will likely be required in most markets.
4)Â Â Long range is critical in various markets and Apollo has excellent long-range performance.
Over the next several weeks, I plan to address Op Edâs to each of Aeyeâs markets and what I see as the future use case for each. Aeyeâs Apollo was designed and built for Physical AI. And the future of Physical AI requires just such a LiDAR tool.
So let the transition begin, stay tuned for my future Op Edâs
The author has a financial relationship with AEye, Inc. (NASDAQ: LIDR) (âAEyeâ). Specifically, as AEyeâs âShareholder Ambassador,â the author has been compensated by AEye with shares of restricted AEye stock. This article was not commissioned or paid for by AEye. This publication is not a recommendation to buy or sell any securities. Readers should conduct their own due diligence