Goaly AI
Founding Senior AI Infrastructure Engineer
Palo Alto, CA, USA · On-site · Senior · Full-time · Posted Aug 28, 2026
- Experience
- Significant software-engineering, distributed-systems, high-performance-computing, or ML-infrastructure experience; no number of years stated.
About the job
Build and optimize infrastructure that improves the performance, efficiency, and reliability of large-scale AI training, inference, and reinforcement-learning workloads.
What you'll do
- Profile AI workloads to identify bottlenecks across GPUs, software, networking, storage, and cluster infrastructure, then deliver measurable improvements.
- Build and optimize distributed inference, training, scheduling, and fault-tolerant systems.
- Develop benchmarks, profiling tools, performance models, and observability in partnership with researchers.
What they ask for
- Exceptional programming and debugging ability in Python and at least one of C++, Rust, or Go.
- Strong systems fundamentals, including operating systems, concurrency, memory, networking, storage, scheduling, containerization, and failure handling.
- Track record of using measurement, benchmarks, profiles, and production validation to solve performance problems.
- Ability to reason across model and framework execution, accelerator behavior, distributed runtimes, and infrastructure.
Nice to have
- Experience with large-scale ML training, post-training, or inference systems.
- Experience with GPU programming or familiarity with ML frameworks and distributed stacks.
Tools
Python, C++, Rust, Go, CUDA, Triton, PyTorch, JAX
AI focus
Optimize infrastructure for AI model training, post-training, agentic reinforcement learning, rollout inference, and GPU cluster workloads.
Model-written summary · the original posting is authoritative