Gimlet Labs
Member of Technical Staff - ML Systems & Inference
$150K–$390K
San Francisco, CA · On-site · Staff · Full-time · Posted Mar 10, 2026
- Education
- Bachelor's degree in a relevant field, or an equivalent combination of education, training, and professional experience.
About the job
Build and optimize production ML inference systems that execute models across different hardware, balancing latency, throughput, and resource utilization.
What you'll do
- Develop inference execution strategies for batching, scheduling, concurrency, and resource utilization.
- Improve KV cache management, memory efficiency, and performance under production workloads.
- Bring up models on novel hardware and enable new model architectures and inference techniques in production.
What they ask for
- Strong software engineering fundamentals.
- Experience building or operating ML inference or model serving systems.
- Ability to reason about performance, memory usage, and system behavior under load.
- Bachelor's degree in a relevant field, or an equivalent combination of education, training, and professional experience.
Nice to have
- Experience with inference runtimes such as TensorRT-LLM, vLLM, or custom serving systems.
- Experience profiling and tuning latency- and throughput-critical systems.
Pay
$150K – $390K (base salary)
Tools
TensorRT-LLM, vLLM, Python, C++
AI focus
Build and optimize AI model inference systems, supporting production workloads, novel hardware, new model architectures, and inference techniques.
Model-written summary · the original posting is authoritative