What we are interested in
Machine learning has revolutionised the process of analyzing data and has led to new insights and applications. However, one of the key short comings of this field is its use of black box models which may lack explainability. A key aspect of our research is to develop interpretable machine learning algorithms. We do so by using techniques such as adversarial optimization, sparsity, etc. We also focus on the interplay between learning and optimization by developing new algorithms to train machine learning models and exploting machine learning models to improve the process of optimization. Finally, we also aim to identify the theoretical reasons behind the success of such models.
đ§âđ Members
sadiku (at) zib.de
turan (at) zib.de
meise (at) zib.de
wagner (at) zib.de
mundinger (at) zib.de
roux (at) zib.de
graczyk (at) zib.de
urbano (at) zib.de
chitranshi (at) zib.de
pelleriti (at) zib.de
birsan (at) zib.de
asadull (at) zib.de
đŹ Projects
Preserving global vegetation is crucial for addressing and mitigating climate change. Accurate, up-to-date forest health data is essential. AI4Forest aims to develop advanced AI methods to monitor forests using satellite imagery, including radar and optical data. The project will create scalable techniques for detailed, high-resolution maps of the globe, e.g., to monitor canopy height, biomass, and to track forest disturbances.
In this project, we study domain decomposition approaches for optimal control in gas transport networks. Our goal is to couple space-time-domain decomposition with machine learning and mixed-integer programming. We will develop NeTI (Network Tearing and Interconnection), a data-driven and physics-informed algorithm combining mixed-integer nonlinear programming, surrogate model learning, and graph decomposition strategies.
Existing approaches for interpreting Neural Network classifiers that highlight features relevant for a decision are based solely on heuristics. We introduce a theory that allows us to bound the quality of the features without assumptions on the classifier model by relating classification to Interactive Proof Systems.
Heuristics play a crucial role in exact solvers for Mixed Integer Programming (MIP). However, the question of how to manage multiple MIP heuristics in a solver has not received sufficient attention. This project addresses the strategic management of primal heuristics in MIP solvers, aiming to replace static, hard-coded rules with dynamic, self-improving procedures.
The performance of modern mixed-integer program solvers is highly dependent on a number of interdependent individual components. Using tools from machine learning, we intend to develop an integrated framework that is able to capture interactions of individual decisions made in these components with the ultimate goal to improve performance.
Training artificial neural networks is a key optimization task in deep learning. To improve generalization, robustness, and explainability, we aim to compute globally optimal solutions. We will use integer programming methods, exploiting mixed-integer nonlinear programming and enhancing solving techniques like spatial branch-and-cut. Additionally, we'll leverage symmetry to reduce computational burden and ensure symmetry in solutions, and incorporate true sparsity using a mixed-integer nonlinear programming framework.
đŹ Talks and posters
Conference and workshop talks
- Nov 2024
- A Combinatorial Comparison of Boosting and Repeated Boosting by Ingo Meise
Annual Meeting, SPP Theoretical Foundations of Deep Learning, Tutzing - Oct 2024
- Extending the Continuum of Six-Colorings by Christoph Spiegel
41st Kolloquium ĂŒber Kombinatorik (KolKom), Heidelberg [PDF] - Jul 2024
- Extending the Continuum of Six-Colorings by Christoph Spiegel
13th Discrete Mathematics Days (DMD), AlcalĂĄ de Henares [PDF] - Sep 2022
- Wavelet-based Low Frequency Adversarial Attacks by Shpresim Sadiku
3rd BMS-BGSMath Junior Meeting, Barcelona [PDF] - Mar 2021
- Neural Network Approximation Theory by Shpresim Sadiku
BMS Conference [PDF] - Dec 2019
- Tensor-based Algorithms for Image Classification by Patrick GelĂ
Mathematics of Deep Learning
Research seminar talks
- Dec 2024
- Coloring the Plane with Neural Networks by Christoph Spiegel
Research Seminar Combinatorics, Berlin [PDF] - May 2024
- Learning Operators Via Hypernetworks by Konrad Mundinger
Research Seminar Numerical Analysis of Stochastic and Deterministic Partial Differential Equations, Berlin - Aug 2023
- Minimally Distorted Explainable Adversarial Attacks by Shpresim Sadiku
AIP Seminar, RIKEN Center for Advanced Intelligence Project, Tokyo - Feb 2023
- What Is Backpropagation? by Shpresim Sadiku
"What is ...?" seminar [PDF]
Poster presentations
- Jul 2024
- Estimating Canopy Height at Scale by Max Zimmer
41st International Conference on Machine Learning (ICML), Vienna - Jul 2024
- Unified Taxonomy in AI Safety: Watermarks, Adversarial Defenses, and Transferable Attacks by Berkant Turan
Workshop on Theoretical Foundations of Foundation Models (TF2M) @ ICML 2024, Vienna - May 2024
- Sparse Model Soups a Recipe for Improved Pruning Via Model Averaging by Max Zimmer
12th International Conference on Learning Representations (ICLR), Vienna - May 2024
- Interpretability Guarantees with Merlin-Arthur Classifiers by Berkant Turan
27th AISTATS Conference, ValĂšncia - May 2024
- Neural Parameter Regression for Explicit Representations of PDE Solution Operators by Konrad Mundinger
Workshop on AI4DifferentialEquations in Science @ ICLR 2024, Vienna - Mar 2024
- Group-wise Sparse and Explainable Adversarial Attacks by Shpresim Sadiku
Deep Learning: Theory, Applications, and Implications, RIKEN Center for Advanced Intelligence Project, Tokyo - Jul 2023
- Extending Merlin-Arthur Classifiers for Improved Interpretability by Berkant Turan
The 1st World Conference on eXplainable Artificial Intelligence - May 2023
- How I Learned to Stop Worrying and Love Retraining by Max Zimmer
11th International Conference on Learning Representations (ICLR), Kigali - Mar 2023
- Wavelet-based Low Frequency Adversarial Attacks by Shpresim Sadiku
Workshop on Optimization and Machine Learning, Waischenfeld - Mar 2023
- How I Learned to Stop Worrying and Love Retraining by Max Zimmer
Workshop on Optimization and Machine Learning, Waischenfeld - Dec 2021
- Learning to Schedule Heuristics in Branch-and-Bound by Antonia Chmiela
conference on neural information processing systems (NeurIPS)
đ Publications and preprints
- WĂ€ldchen, S., Sharma, K., Turan, B., Zimmer, M., and Pokutta, S. (2024). Interpretability Guarantees with Merlin-Arthur Classifiers. Proceedings of the International Conference on Artificial Intelligence and Statistics.
[arXiv]
[BibTeX]
- Zimmer, M., Spiegel, C., and Pokutta, S. (2024). Sparse Model Soups: A Recipe for Improved Pruning Via Model Averaging. Proceedings of the International Conference on Learning Representations.
[URL]
[arXiv]
[BibTeX]
- GöĂ, A., Martin, A., Pokutta, S., and Sharma, K. (2024). Norm-induced Cuts: Optimization with Lipschitzian Black-box Functions.
[URL]
[arXiv]
[BibTeX]
- Goerigk, M., Hartisch, M., Merten, S., and Sharma, K. (2024). Feature-Based Interpretable Optimization.
[arXiv]
[BibTeX]
- GĆuch, G., Turan, B., Nagarajan, S. G., and Pokutta, S. (2024). The Good, the Bad and the Ugly: Watermarks, Transferable Attacks and Adversarial Defenses.
[arXiv]
[BibTeX]
- Mundinger, K., Pokutta, S., Spiegel, C., and Zimmer, M. (2024). Extending the Continuum of Six-Colorings. Geombinatorics Quarterly, XXXIV.
[URL]
[arXiv]
[BibTeX]
- Mundinger, K., Pokutta, S., Spiegel, C., and Zimmer, M. (2024). Extending the Continuum of Six-Colorings. Proceedings of the Discrete Mathematics Days.
[URL]
[arXiv]
[BibTeX]
- Mundinger, K., Zimmer, M., and Pokutta, S. (2024). Neural Parameter Regression for Explicit Representations of PDE Solution Operators.
[arXiv]
[BibTeX]
- Pauls, J., Zimmer, M., Kelly, U. M., Schwartz, M., Saatchi, S., Ciais, P., Pokutta, S., Brandt, M., and Gieseke, F. (2024). Estimating Canopy Height at Scale. Proceedings of the International Conference on Machine Learning.
[arXiv]
[code]
[BibTeX]
- Haase, J., and Pokutta, S. (2024). Human-AI Co-Creativity: Exploring Synergies Across Levels of Creative Collaboration.
[arXiv]
[BibTeX]
- VuâHan, T. L., Sunkara, V., BermudezâSchettino, R., Schwechten, J., Runge, R., Perka, C., Winkler, T., Pokutta, S., WeiĂ, C., and Pumberger, M. (2024). Feature Engineering for the Prediction of Scoliosis in 5qâSpinal Muscular Atrophy. Journal of Cachexia, Sarcopenia and Muscle.
DOI: 10.1002/jcsm.13599
[URL]
[BibTeX]
- Roux, C., Zimmer, M., and Pokutta, S. (2024). On the Byzantine-resilience of Distillation-based Federated Learning.
[arXiv]
[BibTeX]
- Sadiku, S., Wagner, M., Nagarajan, S. G., and Pokutta, S. (2024). S-CFE: Simple Counterfactual Explanations.
[arXiv]
[BibTeX]
- Kevin-Martin, A., BĂ€rmann, A., Braun, K., Liers, F., Pokutta, S., Schneider, O., Sharma, K., and Tschuppik, S. (2023). Data-driven Distributionally Robust Optimization Over Time. INFORMS Journal on Optimization, 5(4), 376â394.
DOI: 10.1287/ijoo.2023.0091
[URL]
[arXiv]
[BibTeX]
- Bienstock, D., Muñoz, G., and Pokutta, S. (2023). Principled Deep Neural Network Training Through Linear Programming. Discrete Optimization, 49.
DOI: 10.1016/j.disopt.2023.100795
[arXiv]
[summary]
[BibTeX]
- Zimmer, M., Spiegel, C., and Pokutta, S. (2023). How I Learned to Stop Worrying and Love Retraining. Proceedings of the International Conference on Learning Representations.
[URL]
[arXiv]
[code]
[BibTeX]
- Kruser, J., Sharma, K., Holl, J., and Nohadani, O. (2023). Identifying Patterns of Medical Intervention in Acute Respiratory Failure: A Retrospective Observational Study. Critical Care Explorations.
[BibTeX]
- Sadiku, S., Wagner, M., and Pokutta, S. (2023). Group-wise Sparse and Explainable Adversarial Attacks.
[arXiv]
[BibTeX]
- Thuerck, D., Sofranac, B., Pfetsch, M., and Pokutta, S. (2023). Learning Cuts Via Enumeration Oracles. Proceedings of the Conference on Neural Information Processing Systems.
[arXiv]
[BibTeX]
- Zimmer, M., Andoni, M., Spiegel, C., and Pokutta, S. (2023). PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs.
[arXiv]
[code]
[BibTeX]
- Macdonald, J., Besançon, M., and Pokutta, S. (2022). Interpretable Neural Networks with Frank-Wolfe: Sparse Relevance Maps and Relevance Orderings. Proceedings of the International Conference on Machine Learning.
[arXiv]
[poster]
[video]
[BibTeX]
- Tsuji, K., Tanaka, K., and Pokutta, S. (2022). Pairwise Conditional Gradients without Swap Steps and Sparser Kernel Herding. Proceedings of the International Conference on Machine Learning.
[arXiv]
[summary]
[slides]
[code]
[video]
[BibTeX]
- Kossen, T., Hirzel, M. A., Madai, V. I., Boenisch, F., Hennemuth, A., Hildebrand, K., Pokutta, S., Sharma, K., Hilbert, A., Sobesky, J., Galinovic, I., Khalil, A. A., Fiebach, J. B., and Frey, D. (2022). Towards Sharing Brain Images: Differentially Private TOF-MRA Images with Segmentation Labels Using Generative Adversarial Networks. Frontiers in Artificial Intelligence.
DOI: 10.3389/frai.2022.813842
[BibTeX]
- Nohadani, O., and Sharma, K. (2022). Optimization Under Connected Uncertainty. INFORMS Journal on Optimization.
DOI: 10.1287/ijoo.2021.0067
[arXiv]
[BibTeX]
- WĂ€ldchen, S., Huber, F., and Pokutta, S. (2022). Training Characteristic Functions with Reinforcement Learning: XAI-methods Play Connect Four. Proceedings of the International Conference on Machine Learning.
[arXiv]
[poster]
[video]
[BibTeX]
- Zimmer, M., Spiegel, C., and Pokutta, S. (2022). Compression-aware Training of Neural Networks Using Frank-Wolfe.
[arXiv]
[BibTeX]
- Pokutta, S. (2021). Mathematik, Machine Learning Und Artificial Intelligence. Mitteilungen Der DMV.
[URL]
[BibTeX]
- Carderera, A., Pokutta, S., SchĂŒtte, C., and Weiser, M. (2021). CINDy: Conditional Gradient-based Identification of Non-linear Dynamics â Noise-robust Recovery.
[arXiv]
[BibTeX]
- Ziemke, T., Sering, L., Vargas Koch, L., Zimmer, M., Nagel, K., and Skutella, M. (2021). Flows Over Time As Continuous Limits of Packet-based Network Simulations. Transportation Research Procedia, 52, 123â130.
DOI: 10.1016/j.trpro.2021.01.014
[URL]
[BibTeX]
- Combettes, C., Spiegel, C., and Pokutta, S. (2020). Projection-free Adaptive Gradients for Large-scale Optimization.
[arXiv]
[summary]
[code]
[BibTeX]
- Pokutta, S., Spiegel, C., and Zimmer, M. (2020). Deep Neural Network Training with Frank-Wolfe.
[arXiv]
[summary]
[code]
[BibTeX]
- Ziemke, T., Sering, L., Vargas Koch, L., Zimmer, M., Nagel, K., and Skutella, M. (2020). Flows Over Time As Continuous Limits of Packet-based Network Simulations. Proceedings of the EURO Working Group on Transportation Meeting.
[BibTeX]
- Klus, S., and GelĂ, P. (2019). Tensor-based Algorithms for Image Classification. Algorithms, 12(11), 240.
DOI: 10.3390/a12110240
[URL]
[arXiv]
[BibTeX]
- Arumugam, K., Kadampot, I., Tahmasbi, M., Shah, S., Bloch, M., and Pokutta, S. (2017). Modulation Recognition Using Side Information and Hybrid Learning. Proceedings of the Proceedings of IEEE DySPAN.
[BibTeX]
- Roy, A., Xu, H., and Pokutta, S. (2017). Reinforcement Learning Under Model Mismatch. Proceedings of the Conference on Neural Information Processing Systems.
[arXiv]
[BibTeX]