Hyperbolic
Member of Technical Staff - Inference
San Francisco, CA · Staff · Full-time · Posted Sep 9, 2026
About the job
Build and operate end-to-end model inference capabilities on Hyperbolic’s Forge platform, serving models across globally distributed clusters and heterogeneous hardware.
What you'll do
- Deploy and serve models on Forge and Kubernetes; evaluate inference frameworks and establish production monitoring, gateways, and endpoints.
- Develop inference optimization, autoscaling, and KV-cache orchestration capabilities.
- Debug customer inference issues and build the inference product from initial deployment through real-world traffic.
What they ask for
- Deep, hands-on experience operating Kubernetes clusters in production.
- Broad inference knowledge, including TTFT, disaggregated inference, speculative decoding, and KV cache.
- Familiarity with modern inference frameworks and serving engines, plus working knowledge of NVIDIA Dynamo.
- Experience setting up production inference monitoring, gateways, and endpoints, and building a product from scratch to serving real traffic.
Nice to have
- Inference deployment and optimization experience, including model optimization such as quantization, batching strategies, or kernel-level tuning.
- Understanding of RDMA and high-performance networking, heterogeneous accelerators, or direct customer inference support.
Tools
Forge, Kubernetes, NVIDIA Dynamo, RDMA
AI focus
Build and optimize production AI model inference services, including distributed serving, autoscaling, and KV-cache orchestration.
Model-written summary · the original posting is authoritative