Software Engineer, Distributed Training

What you'll do

• Develop distributed training for dense and mixture-of-experts models, including low-rank adapter training.
• Improve reinforcement-learning pipelines and optimize GPU memory, parallelism, communication, checkpointing, and recovery.
• Validate training behavior, diagnose execution failures, and work with researchers to productionize new algorithms through the River API.

What they ask for

• Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical industry experience.
• Hands-on experience building or substantially improving distributed model-training systems.
• Strong proficiency in Python and a modern deep-learning framework, such as PyTorch or JAX.
• Understanding of backpropagation, optimizers, mixed-precision training, GPU memory management, and debugging distributed execution.

Nice to have

• Experience with reinforcement-learning infrastructure, rollout generation, or asynchronous training.
• Familiarity with parallelism methods, LoRA, mixture-of-experts training, distributed optimizers, activation checkpointing, NCCL, or C++, Rust, or CUDA.

Pay

$200,000 - $420,000 USD per year (base pay)

Tools

Python, PyTorch, JAX, NCCL, C++, Rust, CUDA

AI focus

Build distributed training infrastructure for fine-tuning and reinforcement learning, and bring new learning algorithms into production.