Drone Companion Computers: UAV Onboard Computing Guide

In today's drone-based solutions, UAVs perform much more complex tasks than just photography and videography. This is driven by the shift away from ground control in favor of companion computers. We'll discuss these technologies in more detail below.

UAV Computing Architecture Overview

The conventional architecture based on a flight controller transmitting data via radio is gradually becoming obsolete, due to high latency caused by the fact that datasets come to  the ground station and only then, after their processing, commands are returned.

Modern autonomous drone systems, however, already have two layers:

• The flight control layer, responsible for low-level stability loops with a sampling rate of 400Hz+. At this layer, latency is unacceptable, and, therefore, it cannot be overloaded with high-level computing tasks.
• The companion layer, running on top of the flight controller via MAVLink/ROS 2, is responsible for resource-intensive tasks.

As a result, this separation of processes solves the problem of UAV safety. Even if the drone companion computer freezes due to overheating or a library error, the UAV will still continue to fly stably in failsafe mode.

Role of a Companion Computer in UAV Systems

The primary task of the onboard computer is edge computation, as it often operates in jammed or complex urban environments without sacrificing decision-making speed. In particular, unlike ground-based solutions, the computer has direct access to sensors, enabling it to perform SLAM and obstacle avoidance without a data link. 

This is what the data flow looks like in modern UAV-based solutions: first, data is collected by the sensor->it’s followed by AI inference->decision-making logic is executed->finally, a MAVLink command is sent to the fly-by-wire controller.

It's important to understand that the cornerstone of the effective implementation of this chain isn't computation speed. The problem lies in bus bandwidth and driver latency when transferring the video stream between the GPU and the processor. To overcome this challenge, we implement optimization through DMA (meaning direct memory access) and use libraries like TensorRT.

Hardware Stack for Drone Companion Computers

Choosing UAV hardware typically requires achieving the best balance between performance and power efficiency.

For example, General-purpose CPUs are inferior to platforms with specialized accelerators, as floating-point density is crucial for tasks like computer vision. Meanwhile, SoC-based systems with integrated GPUs deliver up to 200+ trillions of operations per second while consuming only 15-30 watts.

It's also important to consider thermal constraints, which also determine performance. Since air density changes during flight, and convection is minimal during hover, it makes sense to design the case as a single heatsink, with the companion computer integrated directly into the UAV frame. This ensures effective heat dissipation through the structural elements.

Finally, when choosing between a NUC and embedded modules, you must consider vibration resistance. Mechanical connectors like HDMI/USB-A are susceptible to degradation due to motor vibrations. That's why it's better to solder the interfaces or immediately give preference to JST-GH connectors for the highest priority nodes.

Real-Time Data Processing in Flight Environments

During flight, data processing requires microsecond accuracy. Therefore, software engineers typically use decoders like NVDEC/NVENC for video streaming processing, freeing up CPU resources for implementing control logic. It also makes sense to use zero-copy buffers, as transferring data from the camera and GPU memory through system RAM provokes a CPU scheduling issue (due to Memory Copying/Memcpy).

Moreover, you have to understand that sensor fusion, implemented in a standard camera-LiDAR-IMU system, requires timestamping. This means that if the IMU outputs data at a rate of 1 kHz and the LiDAR at 10 Hz, you'll need to implement a Kalman filter with a high prediction rate – only this will allow you to compensate for drift during abrupt UAV maneuvers. Otherwise, if the drone reaches a speed of 15 m/s, it will encounter a latency of 100 ms or more, which will inevitably lead to a positioning error of 1.5 meters, making obstacle avoidance impossible without the operator's assistance.

AI and Computer Vision on Board Drones

As our practice shows, YOLOv8-tiny architectures and NanoDet are best suited for implementing AI and computer vision with object detection/tracking capabilities, enabling inference of 30-60 FPS directly on the GPU. As for simultaneous mapping/localization, here, we use inertial-visual odometry, with the integrated IMU data directly into graph optimization.

Quantization is best suited for solving the neural network inference problem, which means abandoning FP32 formats in favor of INT8. This speeds up inference by 3-4 times with an error of only 1-2% higher than the initial. And, of course, you shouldn't forget about TensorRT as the main assistant in optimizing neural network graphs for specific UAV hardware.

Communication Between Companion Computer and Flight Controller

The companion computer+flight controller connection requires MAVLink protocols, but implementing it via an UART carries some risks, such as:

• Command priorities – here, we mean that the companion computer must be responsible for high-level tasks only (while the flight controller must take on the UAV’s stabilization).
• UAV safety – the companion computer should never work without a failsafe. Specifically, if the signal is lost for 200+ ms, the controller should immediately turn to the holding-position/auto-return mode. Actually, that’s why we use the MAVROS+MAVSDK combo, thereby ensuring command queuing to prevent the autopilot<->higher-level logic collisions.

Edge Computing vs Cloud Processing in UAVs

Still considering the cloud, given the development of drone edge computing technologies? The choice in favor of the latter ones is obvious, and here's the reason. Standard cloud computing involves first transmitting video to the server and then waiting for a command, which introduces unacceptable latency for a maneuvering drone. Furthermore, bandwidth is limited, imposing communication stability, which is minimal in densely populated areas or electronic warfare.

Furthermore, in autonomous missions, the drone doesn't have to wait for a command – it requires implementing inference directly onboard, with communication with ground stations only for high-level telemetry processing or, for example, navigation mission changes. 

If you can't abandon cloud computing completely, you can consider a hybrid approach where decisions are made at the edge, and metadata is sent to the cloud. This allows the drone to compress keyframes and send them to the ground station for monitoring while it performs the task autonomously.

Software Stack for Companion Computers

The architecture of onboard computer drone custom builds must be fault-tolerant and deterministic. Here are the technological solutions we use to achieve this:

• ROS 2. Generally speaking, ROS 2 is considered the gold standard for robotics middleware (unlike ROS, this version supports quality-of-service profiles to prioritize critical queues), as it provides real-time data processing and eliminates single points of failure. For critical mission control nodes, the PREEMPT_RT Kernel is recommended. This will minimize task scheduler latency, eliminating the risk of system stalls during peak CPU loads, which is crucial for maintaining a stable control loop frequency.
• AI frameworks. PyTorch/TensorFlow are ineffective here (due to interpreter redundancy). Therefore, we convert neural network weights to formats suitable for edge devices (such as ONNX/TensorRT Engine). When it comes to frameworks themselves, TensorFlow Lite and PyTorch Mobile come in handy, as they enable hardware acceleration. However, when using NVIDIA Jetson, TensorRT remains the optimal choice, as it can perform layer fusion and select optimal kernels.
• Middleware and Drone SDKs. SDKs like PX4 Autopilot/DJI Onboard SDK are also often used in development. It's important to understand that the interaction between software layers should be built on Pub/Sub principles, allowing you to replace modules without rewriting the entire code. This is why we often use MAVSDK for high-level management and build microservices using Docker containers. This ensures process isolation and CI/CD.

Industrial Use Cases of Companion Computing in Drones

Here are the industrial sectors where UAV companion computer development can bring the most benefits:

• Agriculture monitoring. In this sector, onboard computing is used to assess vegetation indices directly during flight missions. In particular, UAVs with multispectral cameras can classify crop areas affected by diseases or pests. Additionally, the processors of such UAVs can perform image segmentation, autonomously making decisions about targeted fertilizer application to save up to 30-40% of chemicals.
• Infrastructure inspections. Aerial inspections of power lines and pipelines require the highest precision, and this is where V-SLAM makes sense, allowing the drone to maintain its position even if the GPS signal is lost. Onboard AI detects corrosion and other deformations, transmitting only the defect coordinates and type to the ground instead of terabytes of 4K video.
• Defense and surveillance. In defense missions, drones must remain stealthy, so it is crucial that they operate in radio silence mode. This is only possible with full autonomy, which makes it useful to use neural networks to detect camouflaged objects. This, in turn, requires optimizing models for specific thermal signatures.
• Logistics and delivery. End-to-end automation in logistics can also rely on the onboard computer, where it takes on the task of dynamic path planning. In such projects, we typically use algorithms like A/D-Lite, which recalculate the trajectory in real time when new obstacles appear, while the onboard computer itself analyzes the battery status and available landing sites.

Challenges in Drone Companion Computer Development

The development of UAV onboard computing systems is limited by the following aspects:

• Power. In UAVs, every watt consumed by the companion computer reduces mission time, so drone companion computer developers are tasked with finding ARM platforms with the best TOPS/Watt ratio. Furthermore, this problem is exacerbated by peak loads, as when performing heavy AI inference, processor power consumption often spikes, causing voltage drops and, consequently, flight controller reboots.
• Thermal management. Unlike server-based solutions, engineers can't use active cooling with fans due to their form factor, so it makes sense to replace them with composite frames with integrated anodized aluminum heatsinks. They transfer heat from the chip to the case, which is then ventilated by the airflow. However, even this approach has risks, as the effectiveness of such cooling decreases during hover, necessitating the implementation of software thermal throttling algorithms that reduce the GPU frequency when overheating.
• Weight. Every extra gram of the companion computer reduces its payload, so it's best to use boards with high-density interconnect, unpackaged components, and flexible printed circuits.
• Real-time processing. Often, the problem of high latency between object capture and response lies in the bandwidth of the DRAM and data buses. Multithreading can help here, but it requires careful adjustment of RTOS priorities to ensure that path planning always takes precedence over secondary logging.

Future Trends in UAV Onboard Computing

As our ongoing monitoring of drone technology trends suggests, the future of a companion computer drone integration service lies in:

• Fully autonomous drone swarms. New generations of UAVs will communicate via semantic map sharing instead of traditional raw data, redistributing computational loads among themselves via high-speed short-range protocols based on each UAV's configuration.
• On-device generative AI. Models like vision transformers will soon be used to complete maps of the environment, meaning drones will be able to predict the appearance of objects hidden behind obstacles using only visible fragments.
• Specialized low-power AI chips. General-purpose GPUs will be replaced by specialized NPUs with RISC-V architecture, tailored for specific operations and consuming 5-10 times less power.
• 6G-enabled drone networks. 6G will make integrated sensing and communication technologies a reality, with the network acting as a distributed radar and transmitting data directly to the drone with minimal latency.

FAQ