Machine Learning Engineer

What you'll do

• Define and maintain model evaluation metrics and success criteria.
• Build evaluation pipelines for data handling, labeling, versioning, reruns, and automated judging.
• Analyze model traces and results, create simulated test environments, and deliver findings and proposed fixes.

What they ask for

• Experience designing, building, and running complex evaluations or simulators for reinforcement learning post-training, such as LLM-as-judge setups, RL environments, or simulations.
• Peer-reviewed research at leading machine learning venues or independently released open-source projects of similar caliber.
• Production-quality Python, including testing, maintenance, and a regular release cadence.
• Working knowledge of PyTorch.

Nice to have

• Hands-on experience creating artificial test agents or mock user populations to assess or tune language models.
• Advanced study in a numerate discipline; machine-learning research open-source contributions; personalization, memory, or identity modeling for language models; or data pipeline tooling and SQL.

Pay

$220K - $300K (base salary)

Tools

Python, PyTorch, SQL

AI focus

Evaluate and improve machine-learning models, including reinforcement learning post-training and language models, using automated judging and simulated users or environments.