News Category: Press Release

The course content includes research areas in radiation therapy, non-invasive imaging and tumor and radiation biology as well as machine learning, computer vision, basics for a better understanding, analysis and visualization of scientific data.

AI methods are already being used in cancer treatment, including for the evaluation of complex data or in therapy decisions. The knowledge acquired in the master’s profile can be used, for example, to optimize imaging for radiation planning, to contour organs and tumors using deep learning methods, or to better control radiation treatments. Graduates will also be able to contribute to the further development of individualized diagnostic and therapeutic procedures through AI in the future, both in research and in industry.

Researchers from the Faculty of Medicine and the Faculty of Mathematics and Natural Sciences at the University of Tübingen and the University Hospital Tübingen as well as from Cyber Valley will teach in this profile area. Cyber Valley is Europe’s largest research consortium in the field of AI and it connects partners from politics, science, industry and society. This ecosystem in southwestern Germany is continuously growing with additional multidisciplinary initiatives, making the Stuttgart-Tübingen region a hotspot for research and innovation in the fields of machine learning, robotics and computer vision.

In addition, the master’s profile is part of the AI in medicine initiative, in which the University of Tübingen, the Faculty of Medicine and the Max Planck Institutes, together with other numerous Cyber Valley partners, are advancing the use of AI in medicine and the life sciences.

Each process in nature happens at its own pace: neurons take milliseconds to signal to each other, demographic changes occur over years, and climate change is a matter of decades or even millennia. The period that a process typically needs to undergo a change is known as its timescale, and understanding the timescale is paramount for a deep understanding of the process itself.

How long does change typically take?

Statisticians make this idea of a timescale very precise. In the example of neural processes, the state of your brain right now is influenced by the state some time ago. The extent of this influence is captured by the temporal correlation. This correlation decreases over time: the longer the elapsing time, the weaker the dependency. The timescale of your neural processes is just this: a number that indicates how fast this correlation typically decreases and thus how fast the brain forgets its state.

In practice, determining the timescale from empirical data can be tricky. Typically, one would calculate correlations of the measured brain activity at different times; the decreasing dependency then indicates the timescale.

Recreating the cake to determine its sugar content

“Unfortunately, this method of determining timescales is flawed and can create misleading results,” explains Roxana Zeraati, researcher at the University of Tübingen and at the Max Planck Institute for Biological Cybernetics. “The problem is that the empirical data is always measured over finite, often short, time. Because of this, the average dependency between what happens at different points in time is systematically underestimated.” Zeraati stresses how important a correct measurement of timescales is: “A lot of conclusions in neuroscience rely on precise estimates of timescales. To give just one example, we think that aberrant timescales are linked to autism. But people never realized before that they might be estimating timescales incorrectly.”

This is why Zeraati and her collaborators came up with a novel idea to determine timescales: the scientists propose to generate new, artificial data from a computer model that match closely the empirical data. Anna Levina, assistant professor in Tübingen and Zeraati’s PhD advisor, explains the method with a simple analogy: “If chemically analyzing the sugar content in a cake your grandmother made is tricky – possibly because sugar from the frosting seeps into the cake and messes with your measurements – you can instead try to recreate the cake several times with different amounts of sugar, and the cake that comes closest to the original tells you how much sugar was in your grandmother’s, precisely because you know how much sugar you used for your cakes.”

Analyses of memory

The researchers subsequently tested the potential of their new method on data from neural recordings of the visual cortex from a previous study. “We wondered if spontaneous fluctuations of neural activity were ruled by a single timescale, or if maybe the process we observed was the sum of several processes with different timescales,” says Levina. “In that particular case, we saw that the process in the brain involved two different intrinsic timescales – something that never before has been reported.”

Zeraati adds: “In a similar way, we can analyze almost any intrinsic process in the brain, for example how neurons remember the past. This makes our new estimation method an incredibly valuable and powerful tool for neuroscientists and other researchers.”

Powered exoskeletons are usually not ready for immediate use but have to be adjusted by an expert, so that the different characteristics of the human body, the biomechanics of gait and the wearer’s preferences can be taken into account. Alternatively, this can be done by automatically evaluating quantifiable data such as metabolic rate or muscle activity in order to minimize energy consumption, for example. Or the wearer can be asked to repeatedly compare between different settings in order to find out which one is the most comfortable. However, the setting that leads to a minimum of energy consumption is not necessarily the most comfortable. Letting users choose between numerous settings takes time. Moreover, it does not reveal how the setting parameters interact with each other and affect the user experience.

The focus is on the user as a person

If a Tyrannosaurus Rex living 66 million years ago featured a similar leg structure as an ostrich running in the savanna today, then we can assume bird legs stood the test of time – a good example of evolutionary selection.

Graceful, elegant, powerful – flightless birds like the ostrich are a mechanical wonder. Ostriches, some of which weigh over 100 kg, run through the savanna at up to 55 kph. The ostriches outstanding locomotor performance is thought to be enabled by the animal’s leg structure. Unlike humans, birds fold their feet back when pulling their legs up towards their bodies. Why do the animals do this? Why is this foot movement pattern energy-efficient for walking and running? And can the bird’s leg structure with all its bones, muscles, and tendons be transferred to walking robots?

Alexander Badri-Spröwitz has spent more than five years on these questions. At the Max Planck Institute for Intelligent Systems (MPI-IS), he leads the Dynamic Locomotion Group. His team works at the interface between biology and robotics in the field of biomechanics and neurocontrol. The dynamic locomotion of animals and robots is the group’s main focus.

Together with his doctoral student Alborz Aghamaleki Sarvestani, Badri-Spröwitz has constructed a robot leg that, like its natural model, is energy-efficient: BirdBot needs fewer motors than other machines and could, theoretically, scale to large size. On March 16th, Badri-Spröwitz, Aghamaleki Sarvestani, the roboticist Metin Sitti, a director at MPI-IS, and biology professor Monica A. Daley of the University of California, Irvine, published their research in the renowned journal Science Robotics.

Compliant spring-tendon network made of muscles and tendons

When walking, humans pull their feet up and bend their knees, but feet and toes point forward almost unchanged. It is known that Birds are different — in the swing phase, they fold their feet backward. But what is the function of this motion? Badri-Spröwitz and his team attribute this movement to a mechanical coupling. “It’s not the nervous system, it’s not electrical impulses, it’s not muscle activity,” Badri-Spröwitz explains. “We hypothesized a new function of the foot-leg coupling through a network of muscles and tendons that extends across multiple joints. These multi-joint muscle-tendon coordinate foot folding in the swing phase. In our robot, we have implemented the coupled mechanics in the leg and foot, which enables energy-efficient and robust robot walking. Our results demonstrating this mechanism in a robot lead us to believe that similar efficiency benefits also hold true for birds.”

The coupling of the leg and foot joints and the forces and movements involved could be the reason why a large animal like an ostrich can not only run fast but also stand without tiring, the researchers speculate. A person weighing over 100 kg can also stand well and for a long time, but only with the knees ‘locked’ in an extended position. If the person were to squat slightly, it becomes strenuous after a few minutes. The bird, however, does not seem to mind its bent leg structure; many birds even stand upright while sleeping. A robotic bird's leg should be able to do the same: no motor power should be needed to keep the structure standing upright.

Robot walks on a treadmill

To test their hypothesis, the researchers built a robotic leg modeled after the leg of a flightless bird. They constructed their artificial bird leg so that its foot features no motor, but instead a joint equipped with a spring and cable mechanism. The foot is mechanically coupled to the rest of the leg’s joints through cables and pulleys. Each leg contains only two motors— the hip joints motor, which swings the leg back and forth, and a small motor that flexes the knee joint to pull the leg up. After assembly, the researchers walked BirdBot on a treadmill to observe the robot’s foot folding and unfolding. “The foot and leg joints don’t need actuation in the stance phase,” says Aghamaleki Sarvestani. “Springs power these joints, and the multi-joint spring-tendon mechanism coordinates joint movements. When the leg is pulled into swing phase, the foot disengages the leg’s spring – or the muscle-tendon spring, as we believe it happens in animals,” Badri-Spröwitz adds. A video shows BirdBot walking in the research group’s laboratory.

Zero effort when standing, and when flexing the leg and knee

When standing, the leg expends zero energy. “Previously, our robots had to work against the spring or with a motor either when standing or when pulling the leg up, to prevent the leg from colliding with the ground during leg swing. This energy input is not necessary in BirdBot’s legs,” says Badri-Spröwitz and Aghamaleki Sarvestani adds: “Overall, the new robot requires a mere quarter of the energy of its predecessor.”

The treadmill is now switched back on, the robot starts running, and with each leg swing, the foot disengages the leg’s spring. To disengage, the large foot movement slacks the cable and the remaining leg joints swing loosely. This transition of states, between standing and leg swing, is provided in most robots by a motor at the joint. And a sensor sends a signal to a controller, which turns the robot’s motors on and off. “Previously, motors were switched depending on whether the leg was in the swing or stance phase. Now the foot takes over this function in the walking machine, mechanically switching between stance and swing. We only need one motor at the hip joint and one motor to bend the knee in the swing phase. We leave leg spring engagement and disengagement to the bird-inspired mechanics. This is robust, fast, and energy-efficient,” says Badri-Spröwitz.

Monica Daley observed in several of her earlier biology studies that the bird’s leg structure not only saves energy during walking and standing but is also adapted by nature so that the animal hardly stumbles and injures itself. In experiments with guineafowls running over hidden potholes, she quantified the birds’ remarkable locomotion robustness. A morphological intelligence is built into the system that allows the animal to act quickly – without having to think about it. Daley had shown that the animals control their legs during locomotion not only with the help of the nervous system. If an obstacle unexpectedly lies in the way, it is not always the animal’s sense of touch or sight that comes into play.

“The structure with its multi-jointed muscle-tendons and its unique foot movement can explain why even heavy, large birds run so quickly, robustly, and energy-efficient. If I assume that everything in the bird is based on sensing and action, and the animal steps onto an unexpected obstacle, the animal might not be able to react quickly enough. Perception and sensing, even the transmission of the stimuli, and the reaction cost time,” Daley says.

Yet Daley’s work on running birds over 20 years demonstrates that birds respond more rapidly than the nervous system allows, indicating mechanical contributions to control. Now that the team developed BirdBot, which is a physical model that directly demonstrates how these mechanisms work, it all makes more sense: the leg switches mechanically if there is a bump in the ground. The switch happens immediately and without time delay. Like birds, the robot features high locomotion robustness.

Whether it’s on the scale of a Tyrannosaurus Rex or a small quail, or a small or large robotic leg. Theoretically, meter-high legs can now be implemented to carry robots with the weight of several tons, that walk around with little power input.

The knowledge gained through BirdBot developed at the Dynamic Locomotion Group and the University of California, Irvine, leads to new insights about animals, which are adapted by evolution. Robots allow testing and sometimes confirming hypotheses from Biology, and advancing both fields.

In a paper published on February 23, 2022 in Nature Machine Intelligence, a team of scientists at the Max Planck Institute for Intelligent Systems (MPI-IS) introduce a robust soft haptic sensor named “Insight” that uses computer vision and a deep neural network to accurately estimate where objects come into contact with the sensor and how large the applied forces are. The research project is a significant step toward robots being able to feel their environment as accurately as humans and animals. Like its natural counterpart, the fingertip sensor is very sensitive, robust, and high resolution.

The thumb-shaped sensor is made of a soft shell built around a lightweight stiff skeleton. This skeleton holds up the structure much like bones stabilize the soft finger tissue. The shell is made from an elastomer mixed with dark but reflective aluminum flakes, resulting in an opaque greyish color which prevents any external light finding its way in. Hidden inside this finger-sized cap is a tiny 160-degree fish-eye camera which records colorful images illuminated by a ring of LEDs.

When any objects touch the sensor’s shell, the appearance of the color pattern inside the sensor changes. The camera records images many times per second and feeds a deep neural network with this data. The algorithm detects even the smallest change in light in each pixel. Within a fraction of a second, the trained machine-learning model can map out where exactly the finger is contacting an object, determine how strong the forces are and indicate the force direction. The model infers what scientists call a force map: it provides a force vector for every point in the three-dimensional fingertip.

Figure 1 illustrates the sensor, the camera image generated inside the sensor, and the output of the deep neural network, which shows how accurately the position of contact, the force strength and the force direction can be estimated across the contacting areas. A video also shows the sensor’s functionality.

“We achieved this excellent sensing performance through the innovative mechanical design of the shell, the tailored imaging system inside, automatic data collection, and cutting-edge deep learning,” says Georg Martius, Max Planck Research Group Leader at MPI-IS, where he heads the Autonomous Learning Group. His Ph.D. student Huanbo Sun adds: “Our unique hybrid structure of a soft shell enclosing a stiff skeleton ensures high sensitivity and robustness. Our camera can detect even the slightest deformations of the surface from one single image.” Indeed, while testing the sensor, the researchers realized it was sensitive enough to feel its own orientation relative to gravity.

The third member of the team is Katherine J. Kuchenbecker, the Director of the Haptic Intelligence Department at MPI-IS. She confirms that the new sensor will be useful: “Previous soft haptic sensors had only small sensing areas, were delicate and difficult to make, and often could not feel forces parallel to the skin, which are essential for robotic manipulation like holding a glass of water or sliding a coin along a table,” says Kuchenbecker.

But how does such a sensor learn? Huanbo Sun designed a testbed to generate the training data needed for the machine-learning model to understand the correlation between the change in raw image pixels and the forces applied. The testbed probes the sensor all around its surface and records the true contact force vector together with the camera image inside the sensor. In this way, about 200,000 measurements were generated. It took nearly three weeks to collect the data and another one day to train the machine-learning model. Surviving this long experiment with so many different contact forces helped prove the robustness of Insight’s mechanical design, and tests with a larger probe showed how well the sensing system generalizes.

Another special feature of the thumb-shaped sensor is that it possesses a nail-shaped zone with a thinner elastomer layer. This tactile fovea is designed to detect even tiny forces and detailed object shapes. For this super-sensitive zone, the scientists choose an elastomer thickness of 1.2 mm rather than the 4 mm they used on the rest of the finger sensor.

“The hardware and software design we present in our work can be transferred to a wide variety of robot parts with different shapes and precision requirements. The machine-learning architecture, training, and inference process are all general and can be applied to many other sensor designs,” Huanbo Sun concludes.