Palo Alto Networks
Principal AI Tool Engineer for Software Automation/Testing
$156K–$253K
Office - USA - CA - Headquarters · No · Principal · Full-time · Posted Oct 1, 2026
- Experience
- 8+ years in software engineering, QA automation, or ML engineering.
- Education
- Bachelor’s or Master’s degree in Computer Science, AI, Machine Learning, or related field.
- Visa sponsorship
- Immigration sponsorship: Yes
About the job
Designs and develops automated testing and evaluation systems for AI/ML models and LLM applications, from validation and benchmarking to production monitoring and safety testing.
What you'll do
- Build testing frameworks, automated validation pipelines, evaluation datasets, and test scenarios.
- Develop metrics and tools for assessing model accuracy, robustness, latency, safety, and output quality.
- Monitor production model performance, debug failures, and support ethical AI, fairness, and bias testing.
What they ask for
- Bachelor’s or Master’s degree in Computer Science, AI, Machine Learning, or a related field.
- Strong programming skills in Python (preferred) or similar languages.
- Experience with testing frameworks, machine learning concepts, APIs, distributed systems, and CI/CD pipelines.
- Knowledge of data structures, algorithms, and software design principles.
Nice to have
- Experience with LLMs and prompt engineering; familiarity with LangChain, OpenAI Evals, or similar.
- Knowledge of AI safety, bias detection, adversarial testing, cloud platforms, observability, or synthetic data.
Pay
$156,400.00 - $253,000.00/yr (USD, base salary)
Tools
Python, PyTest, unittest, LangChain, OpenAI Evals, AWS, GCP, Azure
AI focus
Develops testing, evaluation, benchmarking, monitoring, and safety capabilities for AI/ML models and LLM applications.
Model-written summary · the original posting is authoritative