Senior Agentic AI Engineer - Core Platforms

What you'll do

• Defines evaluation strategies, benchmarks, acceptance criteria, and scorecards for AI solutions; builds automated and human-in-the-loop evaluation pipelines.
• Establishes regression testing, release gates, and continuous evaluation, and analyzes failures to recommend remediation.
• Integrates AI models and evaluation results into production systems, CI/CD, monitoring, and governance processes while mentoring engineers.

What they ask for

• 5+ years' experience in software development, including hands-on experience in machine learning, AI technologies, and agentic solutions.
• Knowledge of Python and data processing technologies such as Hadoop or Spark; experience with LLM evaluation harnesses and automated test pipelines for LLM, RAG, or agentic applications.
• Experience designing human-in-the-loop and model-based evaluation workflows, including calibration, error analysis, and evaluation-quality validation.
• Experience delivering production AI systems and integrating evaluation into CI/CD, release readiness, production monitoring, and governance evidence.

Nice to have

• Advanced degree preferred.
• Knowledge of NLP techniques such as fuzzy wuzzy and models such as BERT and Transformer models.

Pay

$123000 - $215250 annually (base salary)

Tools

Python, Hadoop, Spark, BERT, Transformer models, GPT-3.5/4, R, Java

AI focus

Builds and evaluates LLM, RAG, agentic, and multimodal AI systems, including evaluation pipelines, human review, model-based grading, regression testing, and continuous evaluation.