Evaluation & Insights Machine Learning Engineer

What you'll do

• Design and run rigorous evaluations for LLMs and multimodal models, including safety, factuality, robustness, and human-centered quality.
• Build automated scoring, inference, and regression-testing pipelines; use evaluation findings to guide model and product improvements.
• Analyze model failures and collaborate across research, engineering, and product teams to develop actionable improvements.

What they ask for

• Bachelor’s or Master’s degree in a related technical field.
• 8+ years of relevant industry experience in ML Engineering or Applied Research.
• Advanced proficiency in Python and modern deep learning ecosystems; experience building scalable ML inference and evaluation workflows.
• Hands-on experience with LLMs, multimodal models, or NLP systems, and familiarity with AI quality metrics, model alignment, and evaluation frameworks.

Nice to have

• Knowledge of human factors, HCI, or cognitive science methodologies applied to AI system design.

Pay

$184,700 and $324,800 USD base salary

Tools

Python, PyTorch, JAX, Hugging Face, Ray, vLLM, MLflow, Weights & Biases

AI focus

Evaluation and improvement of LLMs, multimodal models, and NLP systems, including model quality, safety, alignment, and human-centered behavior.