How Computer Vision Makes Drones Better
By John Oncea, Editor
The integration of computer vision into drone technology has significantly enhanced their performance, making them more autonomous, efficient, and capable of handling complex tasks across various industries.
Unmanned Aerial Vehicles (UAVs), more commonly known as drones, have been around since the late 18th century, World War I, or any point in between, depending on how you define them. Most definitions, including The Free Dictionary, agree that a drone is a powered, aerial vehicle that does not carry a human operator, uses aerodynamic forces to provide vehicle lift, can fly autonomously or be piloted remotely, and can be expendable or recoverable.
However drones are defined, they’re everywhere. The Federal Aviation Administration (FAA) notes there are more than a million drones registered* and 427,335 remote pilots certified in the U.S. Recreational pilots are operating their drones at or below FAA-authorized altitudes in controlled airspace (Class B, C, D, and surface Class E designated for an airport) only with prior FAA authorization by using LAANC or DroneZone, or at or below 400 feet in Class G (uncontrolled) airspace.
Commercial pilots must be at least 16 years old, able to read, write, speak, and understand English, and be in a physical and mental condition to safely fly a drone. They must also pass a Knowledge Test before receiving certification.
Flying drones as a hobby and using them commercially for photography, mapping, inspection, and delivery do make up a chunk of what drones are known for. But they are arguably better known for their use by the military to carry sensors, target designators, offensive ordnance, or electronic transmitters designed to interfere with or destroy enemy targets.
Whatever their use, drones are omnipresent. From the War in Ukraine to the Super Bowl to hazing wolves, not a day goes by that drones aren’t in the news. And now, with the integration of computer vision (CV), the future role of drones is limitless.
* Estimated number: Recreational flyers may use one registration number on multiple drones.
A Brief History Of Drones
The history of pilotless aerial vehicles can be traced to 1783 when French inventors Joseph-Michel and Jacques-Étienne Montgolfier experimented with unmanned hot air balloons, notes Drone Launch Academy.
“Their linen and silk balloon, fueled by a stove burning wool and straw, ascended roughly 6,000 feet and traveled over a mile in 10 minutes,” Drone Launch Academy writes. “This event marked the beginning of humanity’s exploration of unmanned flight and the first time in recorded history that humans have used armed drones.”
Some 50 years later, the first occurrence of the military’s use of drones occurred when the Austrians used explosive-laden, unmanned balloons to attempt to attack Venice, Italy. Flying without direction, these balloons were blown off course by a shift in the wind and most missed their target.
1898 marked the first demonstration of a radio-controlled craft, but it took place on the water and not in the air. Still, Nicola Tesla’s three-foot-long boat laid the groundwork for what we today call a drone.
As noted earlier, to be a drone the craft needs to use aerodynamic forces to provide vehicle lift and fly autonomously while being piloted remotely. This first happened during World War I when, in 1917, the first radio-controlled aircraft, known as the Aerial Target, was developed by the British for training anti-aircraft gunners.
At the same time, Charles Kettering “created the first unmanned aerial torpedo for the U.S. Army,” writes Drone Launch Academy. “It was known as the Kettering Aerial Torpedo, or ‘Bug,’ and had a maximum speed of approximately 50 mph and a maximum range of approximately 75 miles. Test flights gave mixed results, and the Bug was never used in combat.”
The mid-1930s witnessed the alleged birth of the term drone, applied to the British Queen Bee, a remotely controlled, converted biplane used for anti-aircraft training. “The metaphor of a ‘bee’ is thought to contribute to the use of the word ‘drone’ to describe UAVs,” Drone Launch Academy writes, adding, “However, this term has been in popular use throughout history.”
While the British were rolling out Queen Bee, the U.S. was developing its drone program, launching remote-controlled propeller aircraft for use in training anti-aircraft gunners. More than 9,400 Radioplane QO-3s, also known as the Curtiss N2C-2, were produced during World War II and, after the war, another 60,000 Radioplane BTTs were manufactured.
Drones Become Commercial
Drones remained largely in the domain of the military until 2006 when the FAA began allowing the use of them under specific regulations in civilian airspace. The first instance of a certificate of authorization was to permit a drone to be used to search for survivors of disasters after requests in 2005 failed to be approved and no drones were used following Hurricane Katrina.
The years that followed this ruling saw drone applications outside the military emerge to serve private businesses, the public sector, and hobbyists. Far different from military drones, these relied on horizontally oriented propellers for lift, piloted by the user on the ground, and powered by rechargeable batteries.
“Drones may survey weather and other environmental conditions, such as this water sampling drone does,” writes Britannica. “They can film dramatic events from the air (and) are valuable in remote or dangerous areas.
In everyday applications, drones can be used to drive away birds and other animals from sensitive areas, such as airports, perform precise crop dusting, conduct crop surveys, and provide wireless internet and telephone services.
Drones are also used by law enforcement, a fact that has generated significant interest and some controversy. Many people believe that drones can help police departments monitor traffic law violations and surveil dangerous criminals. However, there are concerns that police departments and government agencies could misuse drones to unnecessarily track the activities of law-abiding citizens.
Meanwhile, drones continue to evolve with the most recent step being the integration of artificial intelligence (AI), specifically CV.
Introducing Computer Vision
Artificial intelligence (AI) is, according to IBM, “Technology that enables computers and machines to simulate human learning, comprehension, problem solving, decision making, creativity, and autonomy.
“Applications and devices equipped with AI can see and identify objects. They can understand and respond to human language. They can learn from new information and experience. They can make detailed recommendations to users and experts. They can act independently, replacing the need for human intelligence or intervention (a classic example being a self-driving car).”
CV is a subset of AI that enables machines to interpret and understand visual information from the world, such as images and videos. While it draws on techniques from various fields, including machine learning, deep learning, and image processing, its primary goal is to mimic human vision and perception.
In short, IBM writes, “If AI enables computers to think, CV enables them to see, observe, and understand.” It accomplishes this by mimicking human vision, but while humans rely on a lifetime of experience, machines learn through data, cameras, and algorithms. Once trained, CV systems can surpass human capabilities by analyzing thousands of products per minute and detecting minute defects that are often imperceptible.
CV requires large datasets to train models that can recognize and differentiate images. For instance, to identify automobile tires, the system must analyze extensive images of tires and related objects to learn subtle distinctions.
Machine learning models allow computers to learn from data rather than being explicitly programmed. Deep learning, a subset of machine learning, uses layered neural networks to enhance this process, enabling systems to improve their accuracy over time.
Finally, Convolutional Neural Networks (CNNs) enable image analysis by breaking images into pixels and assigning labels. Through convolution – a mathematical operation – CNNs predict what they see by iterating over data, refining their accuracy with each pass.
Initially, they detect simple shapes and edges, gradually recognizing more complex patterns, similar to how humans first perceive broad shapes before finer details. While CNNs are ideal for single-image recognition, recurrent neural networks (RNNs) analyze video sequences, understanding relationships between consecutive frames.
Using Computer Vision To Improve Drone Performance
CV significantly improves drone performance by allowing them to perceive and understand their surroundings through image analysis, enabling autonomous navigation, obstacle avoidance, object detection, and real-time decision making, explains Ultralytics. This leads to more efficient and precise drone operations in complex environments without human intervention.
One example of this is autonomous navigation. Drones equipped with CV can interpret their surroundings using cameras and sensors, enabling them to navigate complex environments without human intervention. Algorithms like SLAM (Simultaneous Localization and Mapping) allow drones to map unfamiliar areas and dynamically adjust flight paths in real-time, improving safety and efficiency.
Through advanced imaging processing techniques, CV-enabled drones can identify obstacles like trees, buildings, or people, enabling immediate course adjustments to avoid collisions. And, by tracking specific objects in the field of view, drones can follow targets, monitor movements, and perform tasks like surveillance or inspection with high accuracy.
CV enables drones to generate accurate 3D maps of an area by stitching together images, facilitating detailed analysis and data collection for various applications like surveying and inspection. At the same time, CV algorithms can automatically identify and classify objects within captured images, allowing for faster and more efficient data analysis compared to manual methods. Finally, by processing visual information in real time, drones can react to changing conditions and adjust their flight patterns dynamically.
The benefits that come by adding CV to drones can be gained by many industries, including:
- Agriculture: Drones use CV for precision farming tasks like crop health monitoring, pest detection, and irrigation management.
- Logistics: In warehouses or delivery scenarios, drones equipped with CV can scan barcodes, track inventory, or autonomously deliver packages to precise locations.
- Disaster Management: CV-powered drones assist in rapid damage assessment by creating real-time maps of disaster zones or identifying survivors during search-and-rescue missions.
- Security and Surveillance: Drones equipped with CV monitor large areas for potential threats or anomalies, enhancing security operations in critical infrastructure or remote areas.
- Environmental Monitoring: CV enables drones to track wildlife populations or monitor deforestation and other ecological changes effectively.
Challenges And Future Prospects
While computer vision is promising to take drone performance to the next level, technological challenges and environmental factors (e.g., lighting conditions) remain, including the drone’s limited battery life, something that is further strained by the high computational demands of computer vision algorithms. Real-time processing of complex tasks like object detection and 3D mapping can overwhelm onboard hardware, necessitating efficient algorithm design and hardware optimization.
In addition, transferring the vast amounts of visual data generated by drones to ground stations for real-time analysis faces bandwidth and latency limitations. While edge computing offers a solution, it adds complexity to drone systems.
Concerning environmental factors, adverse conditions such as poor lighting, fog, rain, or occlusions reduce the accuracy of computer vision algorithms. These challenges demand adaptive models capable of handling variations in environmental conditions.
Regulatory and ethical concerns need to be addressed as well. Compliance with airspace regulations and privacy laws complicates drone deployment while ethical concerns around data collection, storage, and potential misuse must be addressed to ensure responsible use of computer vision-enabled drones.
As with any technology, security risks abound. Drones are vulnerable to adversarial attacks that manipulate data or the environment to deceive computer vision systems. This poses risks in critical applications like surveillance or disaster response.
However, ongoing advancements are expected to further enhance drone autonomy and intelligence in the coming years. Research into lightweight and efficient algorithms will help overcome computational limitations, enabling real-time processing on drones with constrained resources.
Advances in adaptive learning techniques, such as transfer learning, will improve drone performance in diverse conditions by fine-tuning models for specific environments while the adoption of 5G networks will enhance real-time data transmission capabilities, while advancements in AI and machine learning will refine autonomous navigation and decision-making processes.
Improved security protocols can be achieved via enhanced encryption methods and secure data transmission technologies will address privacy concerns and bolster trust in drone applications across industries.
Addressing these challenges through innovation in hardware, algorithms, and regulatory frameworks, as well as the refinement of regulations for beyond visual line-of-sight (BVLOS) operations and urban air mobility (UAM), will allow the integration of CV into drones to unlock transformative applications across industries.