Bloomberg
Senior Software Engineer - AI Inference
$160K–$240K
New York · Senior · Full-time · Posted Oct 28, 2026
- Experience
- 5+ years of professional software engineering experience; 4+ years of demonstrated experience working with an object-oriented programming language.
- Education
- A degree in Computer Science, Electrical Engineering, or equivalent practical experience.
About the job
Build and operate Bloomberg’s managed AI inference platform for deploying and serving predictive and generative machine learning models.
What you'll do
- Design scalable infrastructure for online and offline inference and integrate high-performance inference runtimes and serving frameworks.
- Guide platform architecture and technical decisions, balancing latency, throughput, reliability, and cost; improve deployment, observability, and production performance.
- Mentor junior engineers in system design, debugging, and performance optimization.
What they ask for
- Experience designing, building, and operating production distributed systems.
- Strong systems intuition and demonstrated ability to debug and optimize performance-critical services.
- Ability to own problems end-to-end and ramp up quickly in unfamiliar technical areas.
- At least 4 years of demonstrated experience with an object-oriented programming language.
Nice to have
- Experience deploying and operating machine learning systems at scale and optimizing inference.
- Familiarity with PyTorch, GPU software stacks, Kubernetes, cloud-native infrastructure, or distributed computing technologies.
Pay
160,000 - 240,000 USD Annual (base salary)
Tools
KServe, TensorRT, vLLM, ONNX, Triton, PyTorch, CUDA, NCCL
AI focus
Develop infrastructure for production inference of predictive and generative models, including model serving, deployment, autoscaling, and inference optimization.
Model-written summary · the original posting is authoritative