Geometric Neuroevolution for Fine-tuning Automatic Speech Recognition ongoing
This project aims to develop geometry-aware neuroevolution by integrating information geometry into evolutionary algorithms for neural network training, applied to fine-tuning ASR foundation models for audiovisual transcription.
🧑🎓 Project Members
Principal Investigator
🪙 Funding
This project is being funded by the Berlin Mathematics Research Center MATH+ (project ID PaA-9), itself funded by the German Research Foundation (DFG) under Germany's Excellence Strategy (EXC-2046/1, project ID 390685689) from September 2025 to August 2028.
🔬 Project Description
Backpropagation faces challenges with saddle points and error plateaus in high-dimensional weight spaces. Natural gradient methods address this by treating weight space as a Riemannian manifold (the neuromanifold) equipped with the Fisher information metric.
This project aims to develop geometric memetic algorithms exploiting this structure through Riemannian genetic operators: mutation via exponential map perturbations and crossover via exponential barycenters. Applied to fine-tuning Whisper for oral history transcription at FU Berlin library, targeting reduced entity recognition errors in names, organizations, and locations.