publications

2025

1. 

   CausalRivers - Scaling up benchmarking of causal discovery for real-world time-series

   Gideon Stein, Maha Shadaydeh, Jan Blunk, and 2 more authors

   In The Thirteenth International Conference on Learning Representations, Feb 2025

   (accepted at ICLR 2025 as a spotlight poster)

2. 

   Self-Attention for Medical Imaging - On the need for evaluations beyond mere benchmarking

   Tristan Piater, Niklas Penzel, Gideon Stein, and 1 more author

   Communications in Computer and Information Science, Jan 2025

   (in press)

   Abs DOI

   A considerable amount of research has been dedicated to creating systems that aid medical professionals in labor-intensive early screening tasks, which, to this date, often leverage convolutional deep-learning architectures. Recently, several studies have explored the application of self-attention mechanisms in the field of computer vision. These studies frequently demonstrate empirical improvements over traditional, fully convolutional approaches across a range of datasets and tasks. To assess this trend for medical imaging, we enhance two commonly used convolutional architectures with various self-attention mechanisms and evaluate them on two distinct medical datasets. We compare these enhanced architectures with similarly sized convolutional and attention-based baselines and rigorously assess performance gains through statistical evaluation. Furthermore, we investigate how the inclusion of self-attention influences the features learned by these models by assessing global and local explanations of model behavior. Contrary to our expectations, after performing an appropriate hyperparameter search, self-attention-enhanced architectures show no significant improvements in balanced accuracy compared to the evaluated baselines. Further, we find that relevant global features like dermoscopic structures in skin lesion images are not properly learned by any architecture. Finally, by assessing local explanations, we find that the inherent interpretability of self-attention mechanisms does not provide additional insights. Out-of-the-box model-agnostic approaches can provide explanations that are similar or even more faithful to the actual model behavior. We conclude that simply integrating attention mechanisms is unlikely to lead to a consistent increase in performance compared to fully convolutional methods in medical imaging applications.

3. 

   Change Penalized Tuning to Reduce Pre-trained Biases

   Niklas Penzel, Gideon Stein, and Joachim Denzler

   Communications in Computer and Information Science, Jan 2025

   (in press)

   Abs DOI

   Due to the data-centric approach of modern machine learning, biases present in the training data are frequently learned by deep models. It is often necessary to collect new data and retrain the models from scratch to remedy these issues, which can be expensive in critical areas such as medicine. We investigate whether it is possible to fix pre-trained model behavior using very few unbiased examples. We show that we can improve performance by tuning the models while penalizing parameter changes. Hence, we are keeping pre-trained knowledge while simultaneously correcting the harmful behavior. Toward this goal, we tune a zero-initialized copy of the frozen pre-trained network using strong parameter norms. Secondly, we introduce an early stopping scheme to modify baselines and reduce overfitting. Our approaches lead to improvements in four datasets common in the debiasing and domain shift literature. We especially see benefits in an iterative setting, where new samples are added continuously. Hence, we demonstrate the effectiveness of tuning while penalizing change to fix pre-trained models without retraining from scratch.

4. 

   Seasonal shifts in plant diversity effects on above-ground–below-ground phenological synchrony

   Ana E Bonato Asato, Claudia Guimarães-Steinicke, Gideon Stein, and 8 more authors

   Journal of Ecology, Jan 2025

2024

1. 

   Embracing the black box: Heading towards foundation models for causal discovery from time series data

   Gideon Stein, Maha Shadaydeh, and Joachim Denzler

   arXiv preprint arXiv:2402.09305, Jan 2024

2. 

   Investigating the effects of plant diversity on soil thermal diffusivity using Physics-Informed Neural Networks

   Gideon Stein, Sai Karthikeya Vemuri, Yuanyuan Huang, and 4 more authors

   In ICLR 2024 Workshop on AI4DifferentialEquations In Science, Jan 2024

3. 

   Data-Driven Prediction of Large Infrastructure Movements Through Persistent Scatterer Time Series Modeling

   Gideon Stein, Jonas Ziemer, Carolin Wicker, and 8 more authors

   In IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium, Jan 2024

4. 

   Reducing Bias in Pre-trained Models by Tuning while Penalizing Change

   Niklas Penzel, Gideon Stein, and Joachim Denzler

   arXiv preprint arXiv:2404.12292, Jan 2024

5. 

   When Medical Imaging Met Self-Attention: A Love Story That Didn’t Quite Work Out

   Tristan Piater, Niklas Penzel, Gideon Stein, and 1 more author

   arXiv preprint arXiv:2404.12295, Jan 2024

6. 

   Enhanced stability of grassland soil temperature by plant diversity

   Yuanyuan Huang, Gideon Stein, Olaf Kolle, and 8 more authors

   Nature Geoscience, Jan 2024

2023

1. 

   Plant diversity stabilizes soil temperature

   Yuanyuan Huang, Gideon Stein, Olaf Kolle, and 8 more authors

   bioRxiv, Jan 2023

2022

1. 

   Outcome Prediction and Murmur Detection in Sets of Phonocardiograms by a Deep Learning-Based Ensemble Approach

   Sven Festag, Gideon Stein, Tim Büchner, and 3 more authors

   In 2022 Computing in Cardiology (CinC), Jan 2022
2. KI4KI: Neues Projekt zur regelmäßigen Überwachung von Stauanlagen aus dem All

   Clemence Dubois, Jannik Jänichen, Maha Shadaydeh, and 6 more authors

   In Messtechnische Überwachung von Stauanlagen; XII. Mittweidaer Talsperrentag, Jan 2022

2020

1. 

   Stabilizing Transformer-Based Action Sequence Generation For Q-Learning

   Gideon Stein, Andrey Filchenkov, and Arip Asadulaev

   arXiv preprint arXiv:2010.12698, Jan 2020

2. 

   Exploring and exploiting conditioning of reinforcement learning agents

   Arip Asadulaev, Igor Kuznetsov, Gideon Stein, and 1 more author

   IEEE Access, Jan 2020

2019

1. Jacobian Policy Optimizations.

   Arip Asadulaev, Gideon Stein, Igor Kuznetsov, and 1 more author

   CoRR, Jan 2019
2. Conditioning of Reinforcement Learning Agents and its Policy Regularization Application

   Arip Asadulaev, Igor Kuznetsov, Gideon Stein, and 1 more author

   arXiv preprint arXiv:1906.05437, Jan 2019