Member of Technical Staff, Training

What you'll do

• Design and optimize diffusion language model architectures, training objectives, and loss functions.
• Research controlled text generation, constraint satisfaction, and multimodal integration for diffusion models.
• Improve training and inference efficiency and develop post-training methods to align model behavior.

What they ask for

• At least 2 years of experience working on ML projects in PyTorch (or equivalent).
• Excellent familiarity with transformers and core LLM concepts, including autoregressive pretraining, instruction tuning, in-context learning, and KV caching.
• Familiarity with training and inference in diffusion models.
• Experience training deep learning models at scale in distributed computing environments.

Nice to have

• Extensive experience training transformer-based language models from scratch.
• Experience with multimodal learning and cross-modal architectures.

Tools

PyTorch, vLLM, SGLang, TensorRT

AI focus

Research and develop diffusion-based LLMs, including training techniques, parallel token generation, controlled generation, multimodal integration, efficiency, and post-training alignment.