Karl Welzel

I am interested in Frank-Wolfe methods for mixed-integer problems as well as explaining the success of LMO-based first-order methods, such as Muon, for LLM pre-training and machine learning. Previously, I investigated local optimization algorithms that have access to third and higher-order derivatives.

📬 Contact

office
Room 3105 at ZIB
e-mail
homepage
karl.welzel.one
languages
German and English

🎓 Curriculum vitae

2026 to 2028
Researcher at ZIB
Jul 2026
Ph.D. in Mathematics at University of Oxford
2022 to 2025
Tutor at University of Oxford
Jun 2021
M.Sc. in Mathematics at University of Oxford
Sep 2020
B.Sc. in Mathematics at University of Bonn
2020
Research Assistant at University of Bonn
2019 to 2020
System Administrator at University of Bonn

📝 Publications and preprints

Preprints

  1. Cartis, C., Jerad, S., and Welzel, K. (2025). On Global Rates for Regularization Methods based on Secant Derivative Approximations. [arXiv]
    [BibTeX]
    @misc{2025_CartisJeradWelzel_Secantregularization_2509-07580,
      archiveprefix = {arXiv},
      eprint = {2509.07580},
      arxiv = {arXiv:2509.07580},
      primaryclass = {math.OC},
      year = {2025},
      author = {Cartis, Coralia and Jerad, Sadok and Welzel, Karl},
      title = {On Global Rates for Regularization Methods based on Secant Derivative Approximations},
      date = {2025-09-09}
    }
  2. Welzel, K., Liu, Y., Hauser, R. A., and Cartis, C. (2025). Local Convergence of Adaptively Regularized Tensor Methods. [arXiv]
    [BibTeX]
    @misc{2025_WelzelLiuHauserCartis_Adaptivetensorconvergence_2510-25643,
      archiveprefix = {arXiv},
      eprint = {2510.25643},
      arxiv = {arXiv:2510.25643},
      primaryclass = {math.OC},
      year = {2025},
      author = {Welzel, Karl and Liu, Yang and Hauser, Raphael A. and Cartis, Coralia},
      title = {Local Convergence of Adaptively Regularized Tensor Methods},
      date = {2025-10-29}
    }

Full articles

  1. Cartis, C., Hauser, R., Liu, Y., Welzel, K., and Zhu, W. (2026). Efficient Implementation of Third-order Tensor Methods with Adaptive Regularization for Unconstrained Optimization. Mathematical Programming Computation. DOI: 10.1007/s12532-026-00313-6 [arXiv]
    [BibTeX]
    @article{2024_CartisEtAl_TensormethodsRegularization_2501-00404,
      year = {2026},
      journal = {Mathematical Programming Computation},
      date = {2026-03-24},
      doi = {10.1007/s12532-026-00313-6},
      archiveprefix = {arXiv},
      eprint = {2501.00404},
      arxiv = {arXiv:2501.00404},
      primaryclass = {math.OC},
      author = {Cartis, Coralia and Hauser, Raphael and Liu, Yang and Welzel, Karl and Zhu, Wenqi},
      title = {Efficient Implementation of Third-order Tensor Methods with Adaptive Regularization for Unconstrained Optimization}
    }
  2. Welzel, K., and Hauser, R. A. (2024). Approximating Higher-Order Derivative Tensors Using Secant Updates. SIAM Journal on Optimization, 34(1), 893–917. DOI: 10.1137/23m1549687 [arXiv]
    [BibTeX]
    @article{2023_WelzelHauser_Higherordersecant_2301-11678,
      year = {2024},
      journal = {SIAM Journal on Optimization},
      volume = {34},
      number = {1},
      pages = {893--917},
      doi = {10.1137/23m1549687},
      archiveprefix = {arXiv},
      eprint = {2301.11678},
      arxiv = {arXiv:2301.11678},
      primaryclass = {math.OC},
      author = {Welzel, Karl and Hauser, Raphael A.},
      title = {Approximating Higher-Order Derivative Tensors Using Secant Updates}
    }

🔬 Projects

MINO MATH+ AA-Mobil-1
Solution Techniques for Mixed-Integer Nonconvex Optimization

This project investigates (mixed-)integer optimization with a non-convex, differentiable objective within a Branch-and-Bound framework utilizing Frank-Wolfe methods as node solvers. The focus is on incorporating spatial branching and comparing it to convexification strategies.

MINO MATH+ AA-Mobil-1
Sep 2025 to Aug 2028
4
2

💬 Talks and posters

Conference and workshop talks

Jun 2026
Challenges and Potential of Higher-Order Optimization Methods in Machine Learning by Karl Welzel
SIAM conference on optimization (SIAMOP), Edinburgh [PDF]
Jul 2025
Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
8th International Conference on Continuous Optimization (ICCOPT), Los Angeles [PDF]
Jul 2025
Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
22nd Conference on Advances in Continuous Optimization (EUROPT), Southampton [PDF]
Sep 2024
Quasi-Tensor Methods by Karl Welzel
66th OR66, Bangor [PDF]
Jul 2024
Approximate Derivatives for Tensor Methods by Karl Welzel
25th International Symposium on Mathematical Programming (ISMP), Montréal [PDF]
Aug 2023
Approximating Higher-Order Derivative Tensors Using Secant Updates by Karl Welzel
10th ICIAM Conference, Tokyo [PDF]
Jun 2023
Generalizing Quasi-Newton Updates to Higher-Order Derivative Tensors by Karl Welzel
29th Biennial Numerical Analysis Conference, Glasgow [PDF]
Jun 2023
Generalizing Quasi-Newton Updates to Higher-Order Derivatives by Karl Welzel
SIAM conference on optimization (SIAMOP), Seattle [PDF]

Research seminar talks

Apr 2026
Do Third Derivatives Accelerate Unconstrained Continuous Optimization? by Karl Welzel
IOL Research Seminar (IOL), Berlin [PDF]
Nov 2024
Local Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
Numerical Analysis Group Internal Seminar, Oxford [PDF]
Feb 2023
Generalizing Quasi-Newton Updates to Higher-Order Derivatives by Karl Welzel
Numerical Analysis Group Internal Seminar, Oxford [PDF]

Poster presentations

Jul 2026
Local Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
10th Foundations of Computational Mathematics (FoCM), Vienna [PDF]
Jun 2026
Local Convergence of Adaptively Regularized Tensor Methods by Karl Welzel
SIAM UKIE Conference, Leicester [PDF]

📅 Event Attendance

👨‍🏫 Teaching

2025
Tutor for B6.3 Integer Programming at University of Oxford
2025
Tutor and Teaching Assistant for C6.2 Continuous Optimization at University of Oxford
2024
Tutor and Teaching Assistant for B6.3 Integer Programming at University of Oxford
2024
Demonstrator for Computational Mathematics in Python at University of Oxford
2024
Tutor and Teaching Assistant for C6.2 Continuous Optimization at University of Oxford
2023
Tutor and Teaching Assistant for B6.3 Integer Programming at University of Oxford
2022
Teaching Assistant for B6.3 Integer Programming at University of Oxford
2022
Teaching Assistant for C6.2 Continuous Optimization at University of Oxford