Baseten
Software Engineer - GPU Networking & Distributed Systems
$165K–$330K
San Francisco +4 · Hybrid · Full-time · Posted Feb 23, 2026
- Location
- San Francisco
Toronto
New York
Montreal
Seattle
About the job
Build and optimize GPU networking and distributed systems for Baseten’s inference infrastructure, including RDMA, distributed inference, hardware performance, observability, and communication kernels.
What you'll do
- Integrate RDMA/RoCE/InfiniBand into the inference stack and optimize networking for distributed inference.
- Develop checkpointing and storage mechanisms for fast model startup; benchmark and validate networking performance on GPU clusters.
- Build networking observability tools and optimize communication libraries or custom kernels.
What they ask for
- Deep experience with high-performance networking protocols, including InfiniBand and RoCE v2.
- Fluency in C++ or Python and understanding of memory hierarchy in modern NVIDIA architectures.
- Ability to work with TensorRT-LLM source code, bindings, and NVLink topology issues.
- Ability to assess when to use off-the-shelf tools versus building custom solutions.
Nice to have
- Deep knowledge of NCCL, NVSHMEM, and UCX.
- Experience with Rust, GPUDirect Storage or high-performance filesystems, inference frameworks, or low-level hardware benchmarking.
Pay
$165K – $330K USD base salary
Tools
RDMA, RoCE v2, InfiniBand, C++, Python, NCCL, NVSHMEM, UCX
AI focus
Optimize networking and distributed inference for AI models, including MoE workloads, disaggregated KV cache offload, and fast startup for trillion-parameter models.
Model-written summary · the original posting is authoritative