Workday
Sr. Principal Architect - Observability
$222K–$393K
USA, CA, Pleasanton · Flex · Principal · Full-time · Posted Sep 24, 2026
- Experience
- At least 10+ years in software engineering/architecture; the posting also describes 10+ years writing high-quality code as ideal.
- Education
- BS/MS in Computer Science or a related technical field.
About the job
Provides hands-on architectural leadership for Workday’s observability platform, including platform standards, integrations, and AI-agent-powered incident response.
What you'll do
- Lead observability platform architecture, standards, and best practices across logs, metrics, traces, and alerts.
- Drive Observability MCP and agents for incident detection, triage, and recovery.
- Standardize observability tooling and processes, collaborate across teams, and help engineering teams adopt the platform.
What they ask for
- BS/MS in Computer Science or a related technical field.
- At least 10+ years of software engineering/architect background and a track record delivering high-quality products at massive scale.
- Hands-on expertise with observability platforms such as Prometheus, Mimir, Elasticsearch, Clickhouse, VictoriaMetrics, and Jaeger/Zipkin.
- Experience writing high-quality code in Java/Scala/Go/Python and designing systems to meet business needs.
Nice to have
- Expertise in LLM orchestration frameworks and observability for AI agents; experience building observability agents for triage and root-cause analysis.
- Low-latency/high-throughput data processing expertise and open-source contributions to observability frameworks or platforms.
Pay
USD 261,800 - USD 392,600 base salary; USD 221,600 - USD 392,600 base salary
Tools
Java, Scala, Go, Python, Prometheus, Mimir, Elasticsearch, OpenTelemetry
AI focus
Drive agentic incident detection, triage, and recovery using the Observability MCP and Observability Agents; build observability for AI agents, including latency, quality, evaluations, conversations, and tool/agent invocations.
Model-written summary · the original posting is authoritative