Senior Software Development Engineer in Test (SDET), QA - AI Systems

What you'll do

• Build test frameworks, evaluation harnesses, simulation environments, and CI/CD quality gates for AI agents.
• Perform exploratory testing, review agent traces, and verify releases, including manual red-teaming and regression testing.
• Define quality metrics and release criteria, manage evaluation datasets, and investigate failures across models, prompts, tools, and orchestration.

What they ask for

• 5+ years as an SDET, QA engineer, or software engineer with a strong testing focus, owning test strategy and automation for complex distributed systems.
• Experience with both manual and automated testing, including exploratory testing, test planning, and bug reporting.
• Strong programming skills; experience building test frameworks and infrastructure from the ground up.
• Experience testing LLM-based products or agents, using CI/CD and containerized test environments, and testing APIs and microservices.

Nice to have

• Experience with human-in-the-loop evaluation, performance or load testing, or adversarial testing of ML systems.
• Experience with observability tools or testing industrial, IoT, or operational technology systems.

Tools

Python, GitHub Actions, GitLab CI, Jenkins, Docker, Kubernetes, promptfoo, DeepEval

AI focus

Test and evaluate LLM agents, including reasoning, tool use, task completion, safety, and multi-turn behavior; build evaluation and simulation systems and assess agent failures.