Senior AI Researcher- Pre-training (f/m/d)
Level inferred from job title
Job description
Senior AI Researcher in the Pre-training team, owning architecture, optimization, stability and scaling of next-generation foundation models. The role combines machine learning research with engineering, from small-scale proxy experiments to multi-thousand-GPU training runs.
At a glance
- Work mode
- Hybrid · Heidelberg, Germany
- Languages
- English
What you'll do
- Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency
- Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU behavior and de-risk major training decisions
- Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication bottlenecks, race conditions, and synchronization errors
- Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies
What we're looking for
- Proficient in Python and deeply familiar with PyTorch-based training workflows
- Strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research
- Strong mathematical foundation and comfortable reasoning formally about optimisation, scaling behaviour, and training dynamics
- Deep understanding of transformer training dynamics, optimisation, and the behaviour of large distributed training jobs
- Can design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions
- Hands-on experience pre-training large models (e.g., 7B+ parameters) on substantial infrastructure (e.g., 100+ GPU clusters)
- Applies strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows
- Able to implement complex model architectures efficiently and reliably and to debug complex issues across model code, training dynamics, and distributed systems
- Collaborates effectively within a research and engineering team and communicates clearly about work across Pre-training and the broader AAR/AA organization
- Able to work in Germany and collaborate regularly on site in Heidelberg as part of the Pre-training team
Nice to have
- Hands-on experience training LLMs or multimodal models on large GPU clusters using distributed frameworks (e.g., Megatron-LM, DeepSpeed, torchtitan)
- Familiarity with scaling laws, hyperparameter transfer, or methods for predicting large-scale training behavior from smaller proxy runs
- Experience profiling distributed jobs and diagnosing training anomalies like loss spikes, numerical instability, or optimizer pathologies
- Exposure to sparse training approaches (e.g., Mixture-of-Experts) and an understanding of their routing and systems trade-offs
- Demonstrated research excellence through top-tier publications (NeurIPS, ICML, ICLR), impactful open-source contributions, or significant shipped technical work
- Strong curiosity about the hardware and systems constraints that shape scale
Your team
Pre-training
Team focused on advancing the architecture and training of next-generation foundation models; culture built on ownership, autonomy, and empowerment with a flat organisational structure and close collaboration on complex technical problems.
Benefits
- 30 days of paid vacation
- Access to a variety of fitness & wellness offerings via Wellhub
- Mental health support through nilo.health
- Substantially subsidized company pension plan
- Subsidized Germany-wide transportation ticket
- Budget for additional technical equipment
- Flexible working hours and hybrid working model
- JobRad® Bike Lease
Apply
Direct apply at Alephalpha
Opens the job posting at Alephalpha
About Aleph Alpha
Our customers - in finance, manufacturing, public administration - need models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings. We're building that in Heidelberg.
Mission
Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training like our recently released Kolibri-1 model.
Key facts
- Heidelberg
- Recently released Kolibri-1 model
- Customers in finance, manufacturing, public administration
- Training models on a large scale cluster (thousands of NVIDIA Blackwell GPUs)
What they value
- Ownership
- Autonomy
- Empowerment
- Open communication
- Shared purpose