Senior Software Engineer, ML Infrastructure

What you'll do

• Design and operate compute infrastructure for production services, batch jobs, and ML workloads, including workflow orchestration, autoscaling, and quota management.
• Improve ML workflow performance and cost, and build observability and developer tools for monitoring, troubleshooting, and cost reporting.
• Strengthen production reliability and resiliency, including capacity management and multi-region routing; partner with ML teams to address their needs.

What they ask for

• Significant experience building and operating production infrastructure on a public cloud platform.
• Strong distributed systems and infrastructure skills, including Kubernetes, Terraform, cloud networking, IAM, and secrets management.
• Hands-on experience with workflow orchestration tools such as Flyte, Temporal, or Airflow.
• Proficiency in Python and Go, or experience with a similar systems language (C++, Java, Rust) and willingness to work in Python and Go.

Nice to have

• Experience running GPU or accelerator workloads on Kubernetes.
• Familiarity with observability tooling such as Grafana and Google Cloud Monitoring; experience building internal developer platforms or tooling is also noted.

Pay

$209,000 — $270,000 USD (base salary; posting labels this as pay range but does not specify pay type)

Tools

GCP, AWS, Azure, GKE, Kubernetes, Terraform, Flyte, Temporal

AI focus

Supports ML infrastructure, including continuous model training and deployment, ML workflow performance and cost improvements, and GPU or accelerator capacity management.