Mercor
AI Infrastructure Engineer
$200K–$400K
San Francisco · On-site · Full-time · Posted Oct 27, 2026
- Experience
- 5+ years building and operating production distributed systems or ML infrastructure.
About the job
Build and operate Mercor’s GPU infrastructure platform, including clusters, scheduling, distributed training, LLM serving, observability, and cost attribution.
What you'll do
- Design and operate GPU clusters across cloud, neocloud, and on-prem environments; build scheduling and autoscaling systems.
- Run distributed training and optimize high-throughput LLM serving; create benchmarking, health-check, and automated remediation tools.
- Instrument utilization and costs, support capacity planning, and build internal tools and dashboards.
What they ask for
- 5+ years building and operating production distributed systems or ML infrastructure.
- Production-quality Python plus Go or C++, and shell scripting.
- Deep hands-on Kubernetes and container experience, with working knowledge of Slurm or another HPC resource manager.
- GPU workloads at scale; hands-on experience with a distributed training or serving stack; deep experience with at least one major cloud provider.
Nice to have
- LLM inference optimization, high-performance networking, GPU profiling or kernel work, or non-NVIDIA accelerators.
- Capacity planning or cost attribution, automated fault diagnosis, configuration management, or secure/air-gapped deployments.
Pay
$200K – $400K USD base salary
Tools
AWS, GCP, Kubernetes, Slurm, PyTorch Distributed, Ray, vLLM, Megatron-LM
AI focus
Build and operate GPU infrastructure used for AI model training and serving, including LLM inference optimization.
Model-written summary · the original posting is authoritative