Sanas
Member of Technical Staff, ML Inference Engineering
Palo Alto, CA · On-site · Staff · Full-time · Posted Aug 10, 2026
- Experience
- 8+ years overall, including at least 5 years focused on machine learning systems.
About the job
Build and optimize low-latency, high-throughput ML inference and model-serving systems for real-time speech and language models deployed in on-premise data centers.
What you'll do
- Develop GPU kernels, compiler-level optimizations, and model execution improvements to increase inference performance.
- Build custom inference-server backends and scale multi-GPU, multi-node serving infrastructure.
- Profile bottlenecks and build fault-tolerant runtime services with performance benchmarking and monitoring.
What they ask for
- 8+ years writing high-quality, high-performance code, including at least 5 years focused on machine learning systems.
- Experience with one of C/C++/Rust and Python.
- Deep familiarity with modern NVIDIA GPU architectures, CUDA, and low-level system profiling.
- Experience with ML compilers and optimization frameworks, custom model-serving backends, and the AI serving stack.
Nice to have
- Experience with low-precision models, Multi-LoRA, multi-node GPU models, or large-scale on-premise AI clusters.
- Background in Speech or LLM inference, high-performance networking, distributed storage, hardware monitoring, or open-source ML/systems projects.
Tools
C/C++/Rust, Python, NVIDIA GPU architectures, CUDA, Apache TVM, TorchInductor/Dynamo, TensorRT, Triton
AI focus
Optimize and serve real-time speech and language AI models, including kernel optimization, quantization, compiler-level execution, and scalable inference.
Model-written summary · the original posting is authoritative