
Karl Welzel
I am interested in Frank-Wolfe methods for mixed-integer problems as well as explaining the success of LMO-based first-order methods, such as Muon, for LLM pre-training and machine learning. Previously, I investigated local optimization algorithms that have access to third and higher-order derivatives.
📬 Contact
- office
- Room 3105 at ZIB
- welzel (at) zib.de
- homepage
- karl.welzel.one
- languages
- German and English
🎓 Curriculum vitae
- 2026 to 2028
- Researcher at ZIB
- Jul 2026
- Ph.D. in Mathematics at University of Oxford
- 2022 to 2025
- Tutor at University of Oxford
- Jun 2021
- M.Sc. in Mathematics at University of Oxford
- Sep 2020
- B.Sc. in Mathematics at University of Bonn
- 2020
- Research Assistant at University of Bonn
- 2019 to 2020
- System Administrator at University of Bonn
📝 Publications and preprints
Preprints
- Cartis, C., Jerad, S., and Welzel, K. (2025). On Global Rates for Regularization Methods based on Secant Derivative Approximations.
[arXiv]
[BibTeX]
@misc{2025_CartisJeradWelzel_Secantregularization_2509-07580, archiveprefix = {arXiv}, eprint = {2509.07580}, arxiv = {arXiv:2509.07580}, primaryclass = {math.OC}, year = {2025}, author = {Cartis, Coralia and Jerad, Sadok and Welzel, Karl}, title = {On Global Rates for Regularization Methods based on Secant Derivative Approximations}, date = {2025-09-09} } - Welzel, K., Liu, Y., Hauser, R. A., and Cartis, C. (2025). Local Convergence of Adaptively Regularized Tensor Methods.
[arXiv]
[BibTeX]
@misc{2025_WelzelLiuHauserCartis_Adaptivetensorconvergence_2510-25643, archiveprefix = {arXiv}, eprint = {2510.25643}, arxiv = {arXiv:2510.25643}, primaryclass = {math.OC}, year = {2025}, author = {Welzel, Karl and Liu, Yang and Hauser, Raphael A. and Cartis, Coralia}, title = {Local Convergence of Adaptively Regularized Tensor Methods}, date = {2025-10-29} }
Full articles
- Cartis, C., Hauser, R., Liu, Y., Welzel, K., and Zhu, W. (2026). Efficient Implementation of Third-order Tensor Methods with Adaptive Regularization for Unconstrained Optimization. Mathematical Programming Computation.
DOI: 10.1007/s12532-026-00313-6
[arXiv]
[BibTeX]
@article{2024_CartisEtAl_TensormethodsRegularization_2501-00404, year = {2026}, journal = {Mathematical Programming Computation}, date = {2026-03-24}, doi = {10.1007/s12532-026-00313-6}, archiveprefix = {arXiv}, eprint = {2501.00404}, arxiv = {arXiv:2501.00404}, primaryclass = {math.OC}, author = {Cartis, Coralia and Hauser, Raphael and Liu, Yang and Welzel, Karl and Zhu, Wenqi}, title = {Efficient Implementation of Third-order Tensor Methods with Adaptive Regularization for Unconstrained Optimization} } - Welzel, K., and Hauser, R. A. (2024). Approximating Higher-Order Derivative Tensors Using Secant Updates. SIAM Journal on Optimization, 34(1), 893–917.
DOI: 10.1137/23m1549687
[arXiv]
[BibTeX]
@article{2023_WelzelHauser_Higherordersecant_2301-11678, year = {2024}, journal = {SIAM Journal on Optimization}, volume = {34}, number = {1}, pages = {893--917}, doi = {10.1137/23m1549687}, archiveprefix = {arXiv}, eprint = {2301.11678}, arxiv = {arXiv:2301.11678}, primaryclass = {math.OC}, author = {Welzel, Karl and Hauser, Raphael A.}, title = {Approximating Higher-Order Derivative Tensors Using Secant Updates} }
🔬 Projects
MINO MATH+ AA-Mobil-1
Solution Techniques for Mixed-Integer Nonconvex Optimization
This project investigates (mixed-)integer optimization with a non-convex, differentiable objective within a Branch-and-Bound framework utilizing Frank-Wolfe methods as node solvers. The focus is on incorporating spatial branching and comparing it to convexification strategies.
MINO MATH+ AA-Mobil-1
Sep 2025 to Aug 2028
4
2
💬 Talks and posters
Conference and workshop talks
- Jun 2026
- Challenges and Potential of Higher-Order Optimization Methods in Machine Learning by Karl Welzel
SIAM conference on optimization (SIAMOP), Edinburgh [PDF] - Jul 2025
- Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
8th International Conference on Continuous Optimization (ICCOPT), Los Angeles [PDF] - Jul 2025
- Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
22nd Conference on Advances in Continuous Optimization (EUROPT), Southampton [PDF] - Sep 2024
- Quasi-Tensor Methods by Karl Welzel
66th OR66, Bangor [PDF] - Jul 2024
- Approximate Derivatives for Tensor Methods by Karl Welzel
25th International Symposium on Mathematical Programming (ISMP), Montréal [PDF] - Aug 2023
- Approximating Higher-Order Derivative Tensors Using Secant Updates by Karl Welzel
10th ICIAM Conference, Tokyo [PDF] - Jun 2023
- Generalizing Quasi-Newton Updates to Higher-Order Derivative Tensors by Karl Welzel
29th Biennial Numerical Analysis Conference, Glasgow [PDF] - Jun 2023
- Generalizing Quasi-Newton Updates to Higher-Order Derivatives by Karl Welzel
SIAM conference on optimization (SIAMOP), Seattle [PDF]
Research seminar talks
- Apr 2026
- Do Third Derivatives Accelerate Unconstrained Continuous Optimization? by Karl Welzel
IOL Research Seminar (IOL), Berlin [PDF] - Nov 2024
- Local Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
Numerical Analysis Group Internal Seminar, Oxford [PDF] - Feb 2023
- Generalizing Quasi-Newton Updates to Higher-Order Derivatives by Karl Welzel
Numerical Analysis Group Internal Seminar, Oxford [PDF]
Poster presentations
- Jul 2026
- Local Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
10th Foundations of Computational Mathematics (FoCM), Vienna [PDF] - Jun 2026
- Local Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
SIAM UKIE Conference, Leicester [PDF]
📅 Event Attendance
- Jul 2026
- 10th Foundations of Computational Mathematics (FoCM), Vienna
- Jun 2026
- SIAM UKIE Conference, Leicester
- Jun 2026
- SIAM conference on optimization (SIAMOP), Edinburgh
- Jul 2025
- 8th International Conference on Continuous Optimization (ICCOPT), Los Angeles
- Jun 2025
- 22nd Conference on Advances in Continuous Optimization (EUROPT), Southampton
- Sep 2024
- 66th OR66, Bangor
- Jul 2024
- 25th International Symposium on Mathematical Programming (ISMP), Montréal
- Aug 2023
- 10th ICIAM Conference, Tokyo
- Jun 2023
- 29th Biennial Numerical Analysis Conference, Glasgow
- May 2023
- SIAM conference on optimization (SIAMOP), Seattle
👨🏫 Teaching
- 2025
- Tutor for B6.3 Integer Programming at University of Oxford
- 2025
- Tutor and Teaching Assistant for C6.2 Continuous Optimization at University of Oxford
- 2024
- Tutor and Teaching Assistant for B6.3 Integer Programming at University of Oxford
- 2024
- Demonstrator for Computational Mathematics in Python at University of Oxford
- 2024
- Tutor and Teaching Assistant for C6.2 Continuous Optimization at University of Oxford
- 2023
- Tutor and Teaching Assistant for B6.3 Integer Programming at University of Oxford
- 2022
- Teaching Assistant for B6.3 Integer Programming at University of Oxford
- 2022
- Teaching Assistant for C6.2 Continuous Optimization at University of Oxford