Senior Engineer – AI Monitoring & Control Capabilities

What you'll do

• Deliver vendor applications from build through production support on OpenShift; build APIs and integrations with enterprise AI Agent solutions.
• Manage monitoring, model evaluation, dashboards, and alerting for AI performance, safety, quality, guardrails, and audit traceability.
• Troubleshoot production issues, improve reliability and scalability, and lead and mentor engineers across delivery teams.

What they ask for

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, a related field, or equivalent practical experience.
• 13+ years of software engineering experience designing, building, and scaling production applications.
• Strong proficiency in Python; hands-on experience deploying and operating containerized workloads on OpenShift or Kubernetes.
• Production experience with LLM- or RAG-based applications, APIs and third-party integrations, and AI Agents development including OpenAI APIs and LangChain.

Nice to have

• Experience delivering technology in a large, complex, highly regulated enterprise, preferably financial services.
• Experience with annotation pipelines, feedback loops, fine-tuning, model alignment, prompt lifecycle management, or vendor guardrail and observability platforms.

Pay

USD $122,000.00 - $200,000.00 annualized salary (base salary)

Tools

Python, OpenShift, Kubernetes, OpenAI APIs, LangChain, Grafana, Galileo, LangSmith

AI focus

AI Agent operations, including LLM monitoring and observability, guardrails and controls, model evaluation using LLM-as-a-Judge and specialized language models, and governance for agentic and multi-step AI systems.