Prolific
Human Data Quality Engineer (Founding Team)
Mexico +1 · Full-time · Posted Jan 6, 2026
- Location
- Mexico
New York, New York, United States - Experience
- 5+ years building quality, evaluation, or annotation systems within AI, machine learning, LLMs, or human-data environments.
About the job
Build quality frameworks and infrastructure for human-data programs, partnering with clients and engineering teams to ensure data supports advanced AI model training and evaluation.
What you'll do
- Design quality frameworks, evaluation rubrics, and launch-readiness processes for human-data programs.
- Work with clients to shape data requirements and schemas, and build quality into workflows from participant recruitment through delivery.
- Develop scalable quality systems, automated checks, dashboards, and reporting with product and engineering teams.
What they ask for
- Strong Python and SQL skills.
- Solid understanding of machine-learning pipelines and how human data affects model performance.
- Strong analytical and statistical thinking, including experience designing scalable quality frameworks.
- Ability to influence stakeholders and translate data analysis and quality methods into buildable product and engineering requirements.
Nice to have
- Experience with LLM evaluation, RLHF, AI safety, or red teaming.
- Experience with human annotation programs or human-data operations.
Tools
Python, SQL
AI focus
Defines and builds quality and evaluation systems for human data used in advanced AI models, including work across frontier use cases.
Model-written summary · the original posting is authoritative