Machine Learning Engineer, Core Evaluations

What you'll do

• Build evaluation pipelines and automated dashboards for models in development and production; design user studies and measurable metrics.
• Train models to support new evaluation metrics and use results to guide model, data, and product decisions.
• As the founding evaluation team member, help grow the team and lead it in the future.

What they ask for

• Strong experience designing model-performance metrics and user studies using Mechanical Turk or similar platforms.
• Strong experience training and fine-tuning models for evaluation, with statistical expertise to compare results and make decisions.
• Very strong engineering and programming skills.
• Experience training ASR and TTS models and working on large-scale ML problems involving >3B models and >1m hours of data.

Tools

Mechanical Turk, ASR, TTS

AI focus

Evaluates speech generation and recognition models and trains models to capture additional evaluation metrics.