Machine Learning Engineer - AI & ML Evaluation Frameworks

What you'll do

• Design evaluation frameworks and metrics for traditional ML systems and foundation models, including multimodal models.
• Conduct failure analysis and build tools to detect hallucinations, reasoning flaws, edge cases, and demographic bias.
• Develop synthetic-data pipelines, sensor-data adaptors, and visualizers; translate evaluation findings into engineering improvements.

What they ask for

• BS in Computer Science, Machine Learning, Statistics, or a related field.
• 3+ years of experience in ML Engineering or Applied ML.
• Strong experience evaluating supervised, unsupervised, LLM, and deep learning models; hands-on failure analysis and model improvement.
• Proficiency in Python and production-grade code (OOP, CI/CD, Git); experience building data pipelines, inference frameworks, and automated evaluation systems.

Nice to have

• MS/PhD in Computer Science, Machine Learning, Statistics, or a related field.
• Experience with LLM or agentic-system evaluation, synthetic data, prompt engineering, parallel data processing, privacy-preserving ML, AI Safety, interpretability, or adversarial testing.

Pay

$150,400 and $277,600 USD base salary

Tools

Python, OOP, CI/CD, Git, Spark, Kubernetes, Airflow, Federated Learning

AI focus

Evaluate traditional ML, LLMs, diffusion models, and multimodal foundation models; conduct failure analysis, detect clinical hallucinations and reasoning flaws, generate synthetic data, and measure demographic equity and safety.