Nebius
Senior Machine Learning Engineer, LLM Inference Optimization
$195K–$262K
Palo Alto, California, United States +1 · Senior · Full-time · Posted Jul 22, 2026
- Location
- Palo Alto, California, United States
San Francisco Bay Area, California, United States - Visa sponsorship
- Applicants must be authorized to work in the country in which they apply and provide proof of employment eligibility as a condition of hire.
About the job
Own optimization of LLM and VLM models and endpoints, from model artifacts through production deployment, improving inference performance, quality, reliability, and cost.
What you'll do
- Optimize model families, customer endpoints, and serving backends; compare inference engines and diagnose production regressions.
- Develop compression and inference-acceleration workflows, and improve routing, scheduling, caching, and multi-node serving.
- Build reproducible benchmarks, collaborate across engineering teams, and produce design documents, performance reports, and rollout plans.
What they ask for
- Strong Python and PyTorch engineering skills.
- Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference.
- Practical knowledge of a modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or equivalent.
- Strong understanding of transformer inference bottlenecks and experience designing or optimizing distributed inference architecture.
Nice to have
- Experience with quantization, distillation, speculative decoding, or related inference acceleration methods.
- Familiarity with CUDA or Triton; open-source contributions; or experience with cache-aware routing, PDD, WideEP, or GPU communication technologies.
Pay
$195,200 — $262,200 USD (base salary)
Tools
Python, PyTorch, vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve
AI focus
Optimize LLM and VLM inference through compression, quantization, speculative decoding, KV-cache and serving improvements, distributed inference, and benchmarking.
Model-written summary · the original posting is authoritative