Apple
Senior Software Development Engineer in Test — LLM Evaluation & Automation, T3E
$175K–$325K
Cupertino, California +1 · Senior · Full-time · Posted Aug 3, 2026
- Location
- Cupertino, California
San Diego, California - Experience
- Three years of relevant industry experience in test automation, software development, or related areas.
- Education
- BS in Computer Science, Mathematics, or a related field (or equivalent practical experience).
About the job
Leads automated evaluation of generative AI models, building reusable testing tools and CI pipelines to detect and report quality regressions before rollout.
What you'll do
- Build and maintain LLM-as-a-judge evaluation harnesses and integrate them into CI and automation pipelines.
- Create evaluation sets, rubrics, quality gates, and reporting; run and triage evaluation jobs to identify actionable regressions.
- Develop reusable component-level and end-to-end test coverage for generative AI models and frameworks.
What they ask for
- BS in Computer Science, Mathematics, or a related field, or equivalent practical experience.
- Three years of relevant industry experience in test automation, software development, or related areas.
Nice to have
- Practical Python and data-pipeline experience, including JSON/YAML and REST APIs.
- Experience with LLM-as-a-judge evaluation and rubric design, CI/CD integration, or generative model evaluation.
Pay
$175,000 to $308,500 USD base salary; $184,700 to $324,800 USD base salary
Tools
Python, JSON/YAML, REST APIs, CI/CD, Jupyter, SQL, Xcode
AI focus
Automates LLM-as-a-judge evaluation of generative AI outputs, including image generation, natural-language generation, and generated summaries; detects model regressions and supports quality measurement.
Model-written summary · the original posting is authoritative