Duolingo
Proctor Quality and Learning Manager, Scaling Operations
$143K–$193K
New York, NY +3 · Manager · Full-time · Posted Sep 30, 2026
- Location
- New York, NY
Pittsburgh, PA
New York, New York, United States
Pittsburgh, Pennsylvania, United States - Experience
- 5+ years; no alternative stated.
About the job
Owns the quality standards, calibration, adjudication, and learning programs that help Duolingo English Test proctors make accurate, consistent decisions, including in AI-assisted workflows.
What you'll do
- Maintain proctoring rubrics, decision rules, examples, and auditable guidance; lead certification, calibration, and adjudication.
- Develop and measure onboarding, ongoing training, and remediation; use quality data to target interventions and improve decisions.
- Guide Training Specialists and adjudicators, and partner across teams to validate human and AI-assisted workflows and preserve decision quality.
What they ask for
- 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or another field involving complex, consequential judgments, including calibration programs.
- Experience running adult-learning programs spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement.
- Experience designing quality standards and rubrics, and defining benchmarks and validation methods for human–technology workflows, including AI-assisted processes.
- Strong analytical, communication, judgment, and cross-functional leadership skills; experience directing contractors or partners.
Nice to have
- Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention.
- Experience with distributed reviewer populations, assessment concepts, quality systems, or learning, audit, and analytics tools.
Pay
$142,800 — $193,200 USD base salary
AI focus
Establish benchmarks for AI-assisted proctoring workflows and validate automated capabilities against real sessions before deployment.
Model-written summary · the original posting is authoritative