From Sensing to Sense: Deterministic Fusion for Real Robots
Version: 1.0
Author: Adam Taylor
Adiuvo Engineering
2. Change Log Version
1.0 Initial Issue
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN Company Number: 9350139
Page 2 of 10
3. Introduction
Robots are no longer single-purpose machines bolted in place. They are autonomous systems that must perceive, decide, and act in real-time, often around people, where mistakes could cause damage or loss of life. At the heart of that autonomy is sensor fusion: combining high-rate, heterogeneous data streams (such as vision, LiDAR, radar, IMUs, force/torque, proximity, and audio), into a consistent model of its environment the robot can trust. Doing this well is hard Doing it deterministically, safely, and with the ability to scale is even harder.
Roboticists are faced with several challenges when trying to address these problems
The first problem is the data volume. Contemporary robots ingest gigabits per second from multicamera inputs and 3D sensors while also reading microsecond-sensitive encoders and force sensors. Turning those raw streams into time-aligned features requires synchronization, low latency, and high processing bandwidth, both on-board and potentially across the network. When the fusion stack slips (because of jitter, queuing, or cache stalls), the consequences cascade: state estimators diverge, planners throttle back, and the system either slows down or violates safety margins. The core challenges are therefore time bound, bringing with them predictable deadlines
The second problem is that safety and security are non-negotiable. A perception error can translate directly into unsafe motion. Modern systems demand deterministic, multi-axis control with continuous situational awareness, designed under functional-safety and cybersecurity disciplines Fusion pipelines must defend against accidental faults (for example, blinded sensors or partial sensor blockages) and error conditions (for example, incorrect or malformed frames). These considerations are not add-ons. They shape the architecture from the outset and must be considered from the start of the project.
The third problem is that power and thermal limits collide with ambitions. Edge-deployed robots rarely enjoy data center cooling or power. However, edge workloads are increasingly heavily compute based (e.g. SLAM) with learned features, learned depth, segmentation, tracking, and intent recognition. Compute demand has risen while the power budget stays flat. This tension forces careful partitioning into what runs on general-purpose CPUs, and what is offloaded to deterministic real-time cores, and what is accelerated in programmable logic to minimize latency.
The fourth problem is connectivity and complexity. Fusion is not just about sensors on one board. It is about reliable, and real-time communication across many nodes (sensors, actuators, edge gateways), often under ROS 2/DDS with mixed QoS and time-sensitive networking on the factory floor. Roboticists must integrate heterogeneous processors and networks without turning integration tests into the project. Even small architectural choices, such as how you schedule kernels or bridge between microcontrollers and application processors can have system-level impacts.
Finally, development timescales, teams must prototype, iterate, and verify quickly while keeping the ability to easily integrate new sensors, new AI models, and evolving safety goals. Practical experience shows that heterogeneous computing, which combines application processors, real-time cores, and programmable logic can tame latency and jitter when orchestrated correctly. Isolating time-critical sensor and actuator loops on lightweight soft or real-time cores, while higher-level fusion, planning, and ROS 2 nodes run on Linux, reduce interference and improve system visibility. This pattern underpins recent AMD Kria™ KR260 robotics starter kit-based projects that spin up on-demand AMD MicroBlaze™ processor soft cores under remoteproc control, fully isolated from the application OS.
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139
Page 3 of 10
Robotics teams are also converging on ROS 2 end to end, from microcontrollers to edge servers. This raises new concerns (such as transport QoS, discovery overheads), but brings a unifying communications model. Micro-ROS pushes ROS 2 down into embedded cores, allowing critical sensor pipelines to publish with predictable timing while Linux hosts subscribe and fuse at higher levels. This approach has already been demonstrated on KR260 robotics starter kit designs with MicroBlaze processor clients that communicate with an external agent over Ethernet.
This white paper opens with the fusion problem space: latency, determinism, safety and security, power and thermal constraints, connectivity, complexity, and AI pressure, all grounded in the realities of modern deployments. It then outlines a practical architecture that blends ROS 2 with heterogeneous compute and acceleration so that fusion remains fast, explainable, and certifiable. We close with application patterns inside and beyond classical robotics (for example, machine vision and smart infrastructure) and a call to action for building fusion stacks that are robust by design
4. AMD Kria™ KR260 Robotics Starter Kit
This white paper is focused on robotic solutions based on using the AMD Kria™ K26 System-onModule mounted on the KR260 robotics starter kit.
The KR260 robotics starter kit is a carrier board with a K26 System-on-Module and active cooling in a compact 119 × 140 × 36 mm package. The KR260 robotics starter kit is intended to target both robotics and industrial applications and ships with a software support, including Ubuntu, ROS 2, and the Kria Robotics Stack. This makes it an ideal base for projects that need low latency control and straightforward integration with modern robotics frameworks.
At the heart of the K26 SOM is an AMD Zynq™ UltraScale+™ MPSoC that combines four Arm® Cortex®-A53 application cores with two Cortex-R5F real-time cores, closely coupled with AMD UltraScale+™ device programmable logic. The programmable logic provides substantial acceleration resources with 256K system logic cells, 1.2K DSP slices, 64 UltraRAM blocks, and 144 Block RAM blocks. The device also includes a hardware H.264 and H.265 video codec unit for efficient video capture and streaming.
On-module memory consists of 4 GB DDR4 with 512 Mb QSPI and 16 GB eMMC for non-volatile storage, while the carrier adds a microSD slot. Networking is extensive with four 10/100/1000 Ethernet ports and a 10 GbE SFP+ cage. Time-sensitive networking support is available on two of the four Ethernet ports, including an integrated switch and IEEE 802.1AS time synchronization for coordinated, deterministic traffic.
The board includes high-speed vision interfaces such as an SLVS-EC Gen2 two-lane sensor connector and the SFP+ port that can serve 10GigE Vision applications. Display output is available through DisplayPort 1.2a up to 1080p60. There are four USB ports for peripherals.
For expansion, the KR260 robotics starter kit offers four 12-pin Pmods and a Raspberry Pi HAT header with 26 general purpose I/Os. Developers get direct JTAG and an FTDI USB-UART for bring up and debug, along with microSD boot support, a 12 V power input, and a controllable fan header. On the processing system side common peripherals such as CAN, SPI, UART and GPIO are also provided.
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139
Page 4 of 10
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139
Page 5 of 10
5. Architectural Concepts and System Composition
The KR260 robotics starter kit and its K26 SOM enable advanced robotics architectures by combining heterogeneous compute engines, programmable logic, and unified communications. To build predictable, scalable robotic systems, each task must run on the compute engine that fits it best, while the system as a whole must remain modular, observable, and fault-tolerant.
1) Right Engine for the Right Task
• Task-to-engine fit: Place tight control loops, sensor I/O, and pre-processing on real-time compute close to the pins (e.g., Arm Cortex-R5F or AMD MicroBlaze processor soft cores in programmable logic). Use the Arm Cortex-A53 cluster for orchestration, user-space tools, and non-real-time workloads.
• Benefits: Predictable latency where it matters, reduced pressure on application processors, and spare capacity for future growth.
2) Software-Defined Hardware with Clear Boundaries
• Replaceable offload engines: Treat each real-time engine or accelerator as a managed service with a defined memory region, I/O, boot procedure, health interface, and lifecycle control.
• Lifecycle management: Standardize start, stop, update, crash recovery, and logging to minimize custom tooling and maximize reuse.
3) Unified Communications End-to-End
• ROS 2 and micro-ROS fabric: Use a single publish/subscribe model from embedded nodes to applications and, where needed, to the cloud.
• Tooling continuity: Unified messaging ensures consistent debugging, visualization, and replay across embedded and application domains, shortening integration and test cycles.
4) Determinism First, Targeted Acceleration
• Measure and optimize critical paths: Identify latency/jitter bottlenecks and selectively offload them to hardware accelerators in programmable logic or dedicated real-time cores.
• Predictability over raw speed: Acceleration is only valuable if it preserves timing guarantees under load.
• Examples of offload: Filters, feature extraction, timestamp alignment, and transport bridges.
5) System Composition Principles
• Edge Processing Modules: Group sensors, DMA paths, and accelerators into repeatable units with a clear ROS 2 API.
• Orchestration Layer: Run Ubuntu for service management, configuration, and developer utilities. Use containers where appropriate but keep real-time workloads outside container boundaries.
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139
Page 6 of 10
• Data and Time Contracts: Timestamp every sample at the source, maintain a unified time base, and define message schemas with versioning.
• Observability by Design: Build in structured logs, metrics, trace IDs, and event correlation spanning embedded and application domains.
• Fault Containment and Safety: Partition memory and I/O, use watchdogs, health checks, and fallback paths. Design for graceful recovery and isolation.
• Continuous Integration and Delivery: Reproducible builds for firmware, FPGA images, and applications, with hardware-in-the-loop testing to validate timing and interfaces before release.
Implications for Sensor Fusion and Edge AI
By combining these architectural and compositional principles:
• Pre-processing near the sensor reduces bandwidth and upstream compute load.
• Deterministic inference scheduling ensures AI tasks do not violate real-time control budgets.
• Partitioning by function allows low-latency loops to run close to the hardware while higherlevel fusion, planning, and AI inference operate on the application cores.
• Expected outcomes include predictable behavior under load, faster iteration with safer changes, unified observability, and a clear path to scaling sensor fusion and AI workloads at the edge.
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139
Page 7 of 10
6. Why This Scales to Real Robotic Workloads
In the established KR260 robotics starter kit architecture, time-sensitive control paths are isolated from user-facing tasks so that real-time engines never compete with caches, schedulers, or interrupt lines used by user application and logging. Each engine runs headless with a strict interface to the application processor, which treats these engines like peripherals. This separation keeps jitter low and response times bounded as functionality scales and more nodes are added to the system.
Unified Communications Across the Stack
A single communications fabric simplifies everything from bring-up to scaling. Using ROS 2 end-toend and micro-ROS on embedded engines removes the seam between application code and embedded firmware: one message schema provides a direct path to hardware acceleration within the ROS 2 computational graph. On the KR260 robotics starter kit, micro-ROS clients on MicroBlaze processor engines publish to an agent, allowing you to validate discovery and topic flow with standard ROS 2 tools before any behavior is layered on.
Reproducible Deployment
The xmutil app model keeps the bitstream, overlay, and metadata bundled, while remoteproc provides a single runtime to start, stop, and update each engine deterministically. Firmware updates become atomic swaps of ELF images, and rollbacks are as simple as restarting an engine with a previous artifact an approach that aligns with factory update pipelines and DevOps-style validation used during KR260 robotics starter kit development and test.
Clear Path to Acceleration
When CPU-hosted nodes become critical, kernels can migrate using HLS/RTL and present as ROS 2accelerated nodes without changing the surrounding graph. Alternatively, you can dedicate a MicroBlaze processor to the function and preserve the same ROS 2 interface, keeping teams focused on behavior rather than plumbing. This pattern aligns with ROS-native hardware acceleration practices that deliver deterministic, low-latency execution for perception and control while preserving developer ergonomics.
Applying the Approach to Sensor Fusion
Sensor-fusion pipelines map naturally onto this architecture. Fabric engines handle timestamp alignment, per-sensor calibration, and first-stage filtering, such as IMU bias correction, anti-alias filtering, camera line-rate preprocessing, and lidar packet reordering, so that raw bandwidth and timing sensitivity stay off the application CPU. The application processor then performs state estimation and multi-sensor fusion (e.g., EKF/UKF), manages maps, and coordinates with planning. ROS 2 carries synchronized topics with QoS tuned per stream: high-rate data uses best-effort with keep-last policies; low-rate, safety-critical feedback uses reliable delivery with tight histories. The result is a fusion node that consumes clean, aligned data with stable timestamps, improving consistency and observability across the stack.
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139 Page 8 of 10
Demonstration: Applying the Architecture on AMD Kria™ KR260 Robotics Starter Kit
To demonstrate the techniques discussed in this white paper, we will outline a concept design based on the KR260 robotics starter kit: partitioning by function, explicit lifecycle control, standardized communications, and secure packaging. The objective is to show how heterogeneous engines can be combined into a single, deterministic system that remains observable, recoverable, and production ready.
System Under Testing
The KR260 robotics starter kit programmable logic hosts two MicroBlaze processor “engines” that execute time-critical I/O framing and first-stage filtering. Each engine runs a minimal micro-ROS client that publishes at a fixed rate into the system’s DDS network. The Arm Cortex-A53 application processors run Ubuntu and orchestrate the engines at runtime using the Linux remoteproc framework. A micro-ROS or ROS 2 agent runs on a networked host to provide discovery, bridging, and test tooling. This composition mirrors the paper’s principle of the right engine for the right task, with low-level determinism in the fabric and higher-level orchestration on the application cores.
Lifecycle, Isolation, and Recovery
Engine binaries are deployed as firmware objects and started or stopped through remoteproc state transitions. Each engine has a dedicated memory carveout, and memory is cleared when an engine is stopped so stale state does not leak across runs. GPIO-controlled power or reset lines can be asserted to guarantee a known start condition. These mechanisms deliver fault containment in practice: a misbehaving engine can be halted, reset, and relaunched without disturbing the second engine or the application environment.
Deterministic Control Paths
Control loops and framing execute on engines that are not preempted by the application operating system. This reduces jitter compared to running the same loops on the A53 cores, and it shortens the hot path from pins to publish. The application cores then consume already-structured messages, perform fusion and coordination, and schedule actuation with predictable timing. The outcome is a control surface that remains stable as computational graphs grow.
Unified Communications and Observability
All components communicate as ROS 2 or micro-ROS nodes. That choice standardizes interfaces and enables a single introspection workflow throughout development and into field test. Operators bring up the system with familiar tools, list and echo topics, verify rates and QoS, and watch nodes cleanly drop and rejoin during lifecycle events. Because communications are unified, engineers do not need bespoke probes for the fabric engines, which shortens debug cycles and improves repeatability.
Security and Deployment Posture
The demonstration packages the programmable logic as a signed bitstream and loads the devicetree overlay and engine firmware through controlled paths. Engine images are stored under rootcontrolled directories and version pinned, which gives a clear provenance chain from source to system. The same transport fabric that carries telemetry can be configured for authenticated links,
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139
Page 9 of 10
which supports a credible path from lab to pilot deployments without changing the fundamental architecture.
Method and Flow
The bring-up sequence aligns with standard KR260 robotics starter kit workflows. The bitstream and overlay are loaded, engine firmware images are placed in the firmware directory, and each remoteproc instance is pointed to its image. Engines are started, messages appear at the agent, and tooling confirms that rates, topics, and QoS match expectations. To demonstrate isolation, one engine is stopped while the other continues publishing, then the stopped engine is restarted and immediately resumes participation on the bus. This exercise is repeatable and scriptable, so it is suitable for continuous integration and regression testing.
Results and Takeaways
The demonstration highlights four themes that run through this paper. First, moving first-stage processing close to the pins can help reduce latency and contention, which can help improves determinism for the overall control loop. Second, per-engine lifecycle control containing memory carveouts and reset lines provide practical recovery without system-wide disruption. Third, a uniform ROS 2 and micro-ROS fabric enables end-to-end observability using standard tools, lowering the cost of integration and diagnosis. Fourth, secure packaging and controlled loading can create a deployment posture that translates cleanly from prototypes to products.
7. Wrap Up
Modern robotics demands architectures that are deterministic, safe, scalable, and adaptable to rapid changes in sensors and AI workloads. The AMD Kria™ KR260 robotics starter kit provides a strong foundation for meeting these challenges by combining heterogeneous compute, programmable logic, and a unified ROS 2 / micro-ROS communication framework.
By partitioning workloads to the right compute engines, enforcing clear lifecycle and safety boundaries, and building observability into the design, roboticists can create systems that remain predictable under load while still supporting fast iteration and future growth.
The approaches outlined in this paper included sensor pre-processing near the source, unified messaging, software-defined hardware, reproducible deployment, and demonstrate a practical path from prototype to production. With these principles, teams can confidently build robotic platforms that are responsive, certifiable, and ready for the increasing demands of real-world applications.
AMD sponsored this white paper, including engineering hours. AMD, and the AMD Arrow logo, Kria, Microblaze, UltraScale+, Zynq, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.
Registered Office: 1 Bunting Street, Newhall, Harlow, Essex, CM17 9GN
Company Number: 9350139 Page 10 of 10