River AI
Software Engineer, Inference Systems
$200K–$420K
Palo Alto, CA +1 · Full-time · Posted Sep 11, 2026
- Location
- Palo Alto, CA
Palo Alto, California, United States - Education
- Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical experience.
- Visa sponsorship
- We sponsor visas and support the process for the right candidate.
About the job
Build and optimize the serving runtime for large models through the River API, delivering fast, reliable, and cost-efficient inference.
What you'll do
- Develop and improve inference serving, including scheduling, batching, caching, admission control, and recovery.
- Accelerate multi-GPU execution, model loading, and RL sampling while preserving model-version consistency and correctness.
- Profile performance and validate improvements with reproducible performance and correctness tests.
What they ask for
- Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical experience.
- Experience building inference engines or performance-sensitive distributed services.
- Strong understanding of transformer inference, GPU memory, concurrency, and networking.
- Proficiency in Python and C++ or Rust; strong debugging and profiling skills.
Nice to have
- Experience extending SGLang, vLLM, TensorRT-LLM, or similar frameworks.
- Experience with advanced serving techniques, expert parallelism, quantized inference, or production model-serving systems.
Pay
$200,000–$420,000 USD annual base pay
Tools
Python, C++, Rust, SGLang, vLLM, TensorRT-LLM, CUDA
AI focus
Serve and optimize large-model inference, including fine-tuned models and adapters, and improve sampling workloads that power reinforcement learning.
Model-written summary · the original posting is authoritative