Overview
Mobile robots, including logistics robots and autonomous vehicles, must estimate their motion and adapt to different surfaces, wheels, and control inputs. This project investigated how robots can learn these motion characteristics through interaction with their environment instead of relying entirely on fixed, manually calibrated models.
Led by Jörg Stückler, the research focused on self-supervised learning of mobility affordances—learning how a robot’s actions affect its movement. The team combined camera images with inertial measurements to develop visual-inertial odometry and online calibration methods for wheeled robots.
Key results
- Developed methods that calibrate a robot’s kinematic or dynamic motion model while the robot is operating.
- Combined visual-inertial odometry with models of how control commands, such as velocity or steering inputs, affect robot motion.
- Improved motion prediction compared with unadapted or purely visual-inertial models, without requiring wheel encoders in some settings.
- Demonstrated that online calibration can adapt to changes in wheels, ground surfaces, and terrain, and can improve both motion prediction and tracking accuracy.
- Applied the methods to different platforms and environments, including indoor wheeled robots and car-like ground vehicles operating on varied terrain.
- Developed an event-camera method for tracking six-degree-of-freedom object motion. It combines high-frequency event data with conventional image frames to improve tracking during rapid motion and reduce the effects of motion blur.
Why it matters
Robots need accurate motion models to navigate, plan movements, and respond to changes in their surroundings. By learning these models online, the research reduces reliance on fixed calibration and enables robots to adapt more flexibly to different surfaces and operating conditions. The results are relevant to logistics, autonomous vehicles, and other forms of mobile robotics.
Publications and outputs
- Li, H., Stueckler, J. Tracking 6-DoF Object Motion from Events and Frames. In Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA), 2021. DOI: 10.1109/ICRA48506.2021.9561760
- Li, H., Stueckler, J. Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models. IEEE Robotics and Automation Letters (RA-L), 2022. DOI: 10.1109/LRA.2022.3169837
- Li, H., Stueckler, J. Online Calibration of a Single-Track Ground Vehicle Dynamics Model by Tight Fusion with Visual-Inertial Odometry, 2024. DOI: doi:10.1109/ICRA57147.2024.10610157.
- Li, H., Stueckler, J. Observability Analysis of Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models abs/2204.06651, CoRR/arxiv, 2022
