Opportunity
Approach
Ecosystem Catalysis
External Collaborators
To understand our planet is to understand its inhabitants. Yet the most species-rich group of animals, the arthropods, remains our largest blind spot in how we measure the living world. They pollinate our crops, cycle our soils and hold food webs together, yet some overwhelm us entirely while others vanish, both faster than we can count them. Studying their 3D shape, movement and behaviour still depends on costly equipment and thousands of hours of manual annotation, which keeps modern AI almost entirely out of field biology and biodiversity monitoring, exactly where it is needed most. Today, every new species, lab or question means starting digitisation and labelling again from scratch.
I am building open infrastructure that takes a researcher from a physical specimen to a trained AI model in hours rather than months, without machine learning expertise. Building both the hardware and software, I am designing a system to digitise organisms in 3D, generate highly versatile labelled training data synthetically, and quantify movement and behaviour from raw footage, removing almost entirely the manual annotation that gates the field today. Because the underlying macro 3D scanner, scAnt, is highly automated and low cost, it will make digitisation viable at the scale of whole collections, not single labs. Over the fellowship I want to harden this into a pipeline anyone can run and begin digitising at scale, the groundwork for models that generalise across species rather than being rebuilt for each one.
The opportunity space of Engineering Ecosystem Resilience is most closely aligned with my efforts with the shared belief that our tools to measure, predict and manage ecosystems are insufficient, and that closing that gap needs robotics, novel sensors and AI. The measurement layer is exactly what my project builds. By making digitisation open, automated and affordable, it lets many groups contribute to and draw from a shared, growing dataset instead of each starting from scratch, so every specimen added compounds the value for the next. The aim is not one tool but common infrastructure an emerging community can build on.
Fabian Plum is a computer vision researcher and robotics engineer building tools that close the gap between the physical world and AI, with a focus on biology. He is CEO of scAnt UG and a postdoctoral researcher at the Institute for Advanced Simulation at Forschungszentrum Jülich, where he works on species-agnostic parametric models of animal shape and pose and on deep reinforcement learning for multi-agent simulation. During his PhD at Imperial College London, funded by a President's PhD Scholarship, he created scAnt (an open-source macro 3D scanner), replicAnt (a synthetic training-data pipeline) and OmniTrax (an animal tracking and pose-estimation tool). His work has been recognised with a European Research Council Proof-of-Concept grant, a BBSRC International Partnering Award and an Amazon Robotics PhD Prize. He builds full stacks, hardware, software and models alike, and cares most about making advanced tools genuinely accessible to the labs that need them.