Sensor fusion for efficient human-robot collaboration

Reliable person tracking: the key to the next generation of mobile robots

  • Robotics
  • Intralogistics
Autonomer mobiler Roboter in einem Lagergang. Sensorlinien visualisieren die Umgebungserfassung und erkennen eine vorausgehende Person sowie die Regale auf beiden Seiten.
Reliable person tracking is essential for safe collaboration between humans and mobile robots. The research project by KEBA and Hagenberg University of Applied Sciences demonstrates why sensor fusion is crucial in this context and how these findings are paving the way for the next generation of intelligent assistance systems in intralogistics.

Intralogistics is undergoing a paradigm shift. While conventional automation systems rely on rigid processes, demand is growing for flexible, adaptive solutions. Mobile robots in particular are playing an increasingly important role – not as a replacement for people, but as intelligent assistance systems that provide targeted support for employees in their day-to-day work.

Person tracking is a key enabler of this development. Only when a mobile robot can reliably determine a person’s location can it be used effectively in the immediate working environment – for example, in a so-called follow-me mode, in which the robot autonomously follows an employee while taking care of transport tasks.

This vision was investigated in the EU research project AI-PRISM, in which KEBA participated as a technology partner. The project clearly highlighted the limitations of existing technologies.

The challenge: reliable tracking in real-world environments

As part of AI-PRISM, a range of person-tracking technologies were initially evaluated. The results showed that although each individual solution has specific strengths, it also reaches its limits in real-world production and logistics environments.

  • UWB (ultra-wideband) enables robust position determination, but requires additional infrastructure and employees to wear a tag.

  • Camera-based systems do not require this infrastructure and can use artificial intelligence to detect people. However, their reliability decreases significantly as soon as a person is occluded or leaves the camera’s field of view.

Such situations occur frequently in intralogistics. Dynamic workflows, changing visibility conditions and obstacles such as racks, pallets or transport containers make it difficult to track people continuously and reliably.

The conclusion: no single technology is sufficient on its own to ensure reliable person tracking in an industrial environment.

A warehouse worker wearing an orange safety vest walks alongside an autonomous mobile robot transporting stacked boxes in a modern warehouse environment.

Learn more about the EU research project AI-Prism

Click here for the blog post

The solution: sensor fusion instead of a single technology

To address this challenge, KEBA launched a follow-up research project together with the University of Applied Sciences Upper Austria, Hagenberg Campus. The central question was how the strengths of different sensor technologies could be combined to enable reliable person tracking under real-world conditions.

The central approach: Combining multiple sensors through intelligent data fusion

The solution combines the following technologies:

  • LiDAR sensors, which capture the environment with high precision and detect geometric structures such as the characteristic leg patterns of people

  • Camera systems, which use AI-based object detection to identify people in an image and provide the semantic information confirming that the detected object is in fact a person.

Fusing these data sources creates a significantly more robust overall system. LiDAR continues to provide reliable distance information even when a person is outside the camera’s direct field of view or temporarily occluded, while the camera ensures unambiguous identification. The result is more stable tracking – even in dynamic environments with changing visibility conditions.

How sensor fusion delivers lasting improvements in person tracking

The strength of this approach lies in its ability to systematically compensate for the weaknesses of individual technologies:

© KEBA
© KEBA

The benefits for intralogistics

Stable person tracking makes follow-me robots suitable for industrial use for the first time. This enables reliable automation of applications in order picking, material transport and production assistance. The robot follows employees precisely, transports goods, materials or tools, and relieves them of time-consuming walking and carrying tasks. This reduces unproductive travel time, improves workplace ergonomics and increases the efficiency of logistics processes. People remain at the heart of the process at all times, while the robot supports them according to the situation.

What makes this approach stand out:

  • Integration of multiple sensor technologies instead of a single-sensor approach

  • Use of advanced AI models for person detection

  • Combination of geometric and semantic information

  • Real-time processing on industrial hardware

The approach developed in the project differs from many existing solutions by seamlessly combining three core technologies: sensing, artificial intelligence and data fusion. The LiDAR sensors provide precise geometric information about the position and movement of objects. The camera’s AI-based image processing reliably detects and classifies people as human beings. Data fusion brings both sources of information together in real time and processes them into a shared model of the environment. This allows the system to continue tracking people reliably even when individual sensor data is temporarily incomplete or uncertain.

The innovation lies not only in combining these technologies, but also in integrating them in real time on a common platform suitable for industrial use. The result is a significantly more robust person-tracking system than would be possible with individual sensors or isolated AI solutions.

Insights from the research project

The project demonstrated that the quality of a person-tracking system is determined not by individual sensors or algorithms, but by how effectively they work together. Only the fusion of camera and LiDAR data enables robust and reliable person detection under real-world conditions.

Precise sensor calibration, accurate data fusion and careful integration of all system components are equally important. Validation against defined reference points – the ground truth – ensured the accuracy of the results.

Working with real hardware also highlighted that a functioning overall system requires far more than powerful algorithms. In day-to-day project work, factors such as a stable power supply, reliable cabling and precise coordination between the sensors proved just as important as the software development itself.

The project results demonstrate the potential of the solution, but also make it clear that reliable person tracking remains a demanding field of research. Despite the progress achieved, challenges persist – for example in very crowded environments, under extreme lighting conditions, when the target person disappears completely from view, or when the system must re-identify that person after a longer interruption.

Modern production hall featuring machines, a robotic arm, an employee at a control panel, and graphically represented connections between the systems.

Kemro X | Platform for modular software and hardware

More about Kemro X

Outlook: from research project to industrial application

The insights gained are now being incorporated into the further development of intelligent mobile robotics solutions at KEBA. Integrating the sensor-fusion solution into the Kemro X automation platform brings sensing, artificial intelligence and control together in a powerful overall system – an important step towards industrial application.

The vision extends well beyond conventional material transport. Mobile robots are increasingly evolving into intelligent assistance systems that collaborate flexibly with people, make processes more efficient and help companies meet the growing demands of modern production and logistics operations.

Conclusion: Sensor fusion as a key technology

The results show that reliable person tracking is a key technology for mobile robotics. A single sensor technology is not enough: only the combination of camera and LiDAR enables robust and precise person detection, even under challenging conditions.

For KEBA, the project provides valuable input for the development of intelligent assistance systems. Combining sensing, AI and the Kemro X automation platform creates the foundation for safe and efficient human-robot collaboration in intralogistics.

Andreas Reinguber
Andreas Reinguber Section Manager Intralogistics | KEBA Industrial Automation [email protected]
Please select your preferred language
Your browser is out of date
Internet Explorer is no longer supported. Please switch to a current browser to use keba.com to its fullest extent.

Edge

Chrome

Safari

Firefox