WildCap – Autonomous Animal Motion Capture for Wildlife Conservation

Overview

Understanding animal behaviour at large spatial and temporal scales is crucial for ecology and conservation, but it is difficult to monitor wild animals continuously without interfering with them. Traditional approaches often rely on GPS collars, body-mounted sensors, or stationary camera traps, which are hard to scale and can affect the animals.

Led by Aamir Ahmad at the University of Stuttgart’s Institute of Flight Mechanics and Flight Control, the WildCap project (2021–2026) developed autonomous aerial robots that can detect, track, and follow animals in their natural habitat. Using onboard cameras and AI, the system estimates behaviour, 3D pose, and body shape without attaching any sensors or markers to the animals.

Key results

• Built custom autonomous aerial platforms, including octocopter drones and an airship, equipped with perception-driven autonomy pipelines for tracking and following animals.
• Developed vision-based methods for detecting, tracking, and following multiple species, demonstrated in field missions at the Mpala Conservancy in Kenya and Hortobágy National Park in Hungary.
• Created algorithms for multi-view 3D pose and shape estimation of animals from unsynchronised aerial images, enabling continuous “4D motion capture” (pose and shape over time).
• Advanced cooperative control methods for multiple aerial robots, allowing teams of drones or airships to coordinate their positions to maximise observation quality.
• Developed synthetic data generation pipelines and learning-in-simulation methods to train perception and control algorithms before deployment in the wild.
• Produced peer-reviewed publications on topics including aerial 3D pose and shape estimation, autonomous blimp control, simulation and control of deformable airships, and real-time animal re-identification from coat patterns.linkedin+1
• Released open-source tools such as RAPID, a real-time algorithm for identifying individual animals by their coat patterns, designed to run on edge devices like drones or camera traps.

Why it matters

WildCap demonstrates how autonomous robots and AI can enable non-invasive, large-scale monitoring of animal behaviour and movement. This could support wildlife conservation, ecological research, and veterinary applications by providing continuous, high-resolution data without disturbing the animals. The project’s methods for multi-robot coordination, 3D pose estimation, and real-time individual identification may also inform other domains, such as environmental monitoring and search-and-rescue.

This project produced the following peer-reviewed publications: 

• Price, E., Khandelwal P. C., Rubenstein D. I., & Ahmad, A (2025).  A Framework for Fast, Large-scale, Semi-Automatic Inference of Animal Behavior from Monocular Videos, Methods in Ecology and Evolution (Wiley) (Accepted, doi awaited); BioArXiv version (2023):  https://doi.org/10.1101/2023.07.31.551177 
• [Award Paper] Price, E., Black, M., & Ahmad, A. (2023) Viewpoint-driven Formation Control of Airships for Cooperative Target Tracking, IEEE Robotics and Automation Letters, vol. 8, no. 6, pp. 3653-3660, June 2023. doi: https://doi.org/10.1109/LRA.2023.3264727 (Best Faculty Paper Award 2023: Best paper in 2023 from the Faculty of Aerospace Engineering and Geodesy at the University of Stuttgart)
• Bonetto, E. & Ahmad, A. (2023) Synthetic Data-based Detection of Zebras in Drone Imagery, IEEE European Conference on Mobile Robots (ECMR 2023), 1-8 . https://doi.org/10.1109/ECMR59166.2023.10256293
• Price, E. & Ahmad, A. (2023) Accelerated Video Annotation driven by Deep Detector and Tracker, 18th International Conference on Intelligent Autonomous Systems (IAS 18). https://link.springer.com/chapter/10.1007/978-3-031-44981-9_12
•  Zuo, Y., Liu, Y. T. and Ahmad A. (2023), Autonomous Blimp Control via H-infinity Robust Deep Residual Reinforcement Learning, 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE 2023), Auckland, New Zealand, 2023, pp. 1-8. https://doi.org/10.1109/CASE56687.2023.10260561
• Liu, Y. T., Price, E., Black, M.J., Ahmad, A. (2022) Deep Residual Reinforcement Learning based Autonomous Blimp Control, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 12566-12573, 2022. https://doi.org/10.1109/IROS47612.2022.9981182
• Saini, N., Bonetto, E., Price, E., Ahmad, A., & Black, M. J. (2022). AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation, IEEE Robotics and Automation Letters, 7(2), 4805–4812. https://doi.org/10.1109/LRA.2022.3145494
• Price, E., Liu, Y. T., Black, M. J., & Ahmad, A. (2022). Simulation and Control of Deformable Autonomous Airships in Turbulent Wind. In: Ang Jr, M.H., Asama, H., Lin, W., Foong, S. (eds) Intelligent Autonomous Systems 16. IAS 2021. Lecture Notes in Networks and Systems, vol 412. Springer, Cham. https://doi.org/10.1007/978-3-030-95892-3_46
• Goldschmid, P. & Ahmad, A. (2024) Reinforcement learning based autonomous multi-rotor landing on moving platforms, Autonomous Robots, Springer. vol. 48, no. 13, June 2024. doi: https://doi.org/10.1007/s10514-024-10162-8
• [Award Paper] Price, E. & Ahmad, A. (2024) Airship Formations for Animal Motion Capture and Behavior Analysis,  In 2nd International Conference on Design and Engineering of Lighter-Than-Air systems (DELTAs 2024), Mumbai, India. Preprint: https://arxiv.org/abs/2404.08986 (Awarded the ‘Best Paper Award’ for papers presented on Day 2 of the conference).
• Bonetto, E. & Ahmad, A. (2025) GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research with Isaac Sim, The International Journal of Robotics Research (IJRR), 2025;0(0). doi:10.1177/02783649251346211, April 2025.