Member of Technical Staff - ML Research Engineer, Pre-Training Data

What you'll do

• Source, filter, deduplicate, and decontaminate web-scale and public datasets; build production pipelines with traceable data lineage.
• Design pre-training and midtraining data mixtures, curricula, and synthetic data generation systems.
• Run proxy experiments and ablations, build trustworthy evaluation sets, and determine which data changes improve model quality.

What they ask for

• Hands-on experience with pre-training data work, including sourcing and curation, synthetic data generation, mixing and curriculum design, deduplication, and decontamination.
• Ability to reason about how data decisions affect training and model quality, and design experiments to test those decisions.
• Experience building and owning production data pipelines.
• ML fundamentals, including training, evaluating, and iterating on models in Python (PyTorch preferred).

Nice to have

• Distillation, long-context, or architecture-specific data strategy; evaluation frameworks and benchmarks for pretrained models.
• Data work across pre-training and post-training; small, efficient models or on-device deployment; startup experience or connecting model work to product needs.

Tools

Python, PyTorch, LLMs

AI focus

Pre-training data research and engineering for foundation models, including synthetic data generation with LLMs, data mixture and curriculum design, and experiments measuring effects on model quality.