Perplexity
Member of Technical Staff (Answer Quality & Evals)
$200K–$350K
San Francisco +1 · Hybrid · Staff · Full-time · Posted Apr 13, 2026
- Location
- San Francisco
Palo Alto - Experience
- 4+ years of software, data, or machine learning engineering experience shipping and operating production systems.
About the job
Build evaluation infrastructure and agent-trace replay tools to measure, diagnose, and improve Perplexity’s answer quality.
What you'll do
- Build shared evaluation infrastructure for running evals, analyzing results, and informing product and model decisions.
- Develop systems to replay and analyze agent traces and reproduce production behavior.
- Build and operate scalable systems for processing, storing, and monitoring interaction, trace, and evaluation data.
What they ask for
- Strong proficiency in Python and SQL, with solid system design, data modeling, and distributed systems fundamentals.
- Experience building big-data systems, including distributed compute, large-scale storage, and high-volume pipelines.
- Demonstrated ownership of ambiguous technical projects from design through production operation.
- Ability to work effectively with data scientists, engineers, and product partners.
Nice to have
- Experience building evaluation, experimentation, observability, or machine learning infrastructure.
- Familiarity with LLM and agent systems, or platforms such as Databricks, Snowflake, or ClickHouse.
Pay
$200K – $350K (base salary; compensation type not specified)
Tools
Python, SQL, Databricks, Snowflake, ClickHouse
AI focus
Build evaluation infrastructure and agent-trace replay and analysis systems that help assess and improve LLM- and agent-powered answer quality.
Model-written summary · the original posting is authoritative