JPMorgan Chase
Principal Software Engineer - AI Foundations
$204K–$285K
Jersey City, NJ +3 · Principal · Full-time · Posted Sep 2, 2026
- Location
- Jersey City, NJ
Palo Alto, CA
Seattle, WA
New York, NY - Experience
- 7+ years applied experience.
- Education
- Formal training or certification on software engineering concepts.
About the job
Leads the design, development, and operation of an enterprise GenAI serving platform, optimizing LLM inference, model routing, GPU utilization, and AI-enabled engineering workflows.
What you'll do
- Design and operate high-throughput, low-latency LLM serving, gateway, routing, and distributed inference systems.
- Optimize GPU serving and model performance, and establish inference service reliability, observability, and operational practices.
- Govern agentic AI-enabled engineering workflows, including standards for validation, security, and reuse.
What they ask for
- Formal training or certification on software engineering concepts and 7+ years of applied experience delivering system design, application development, testing, and operational stability for large-scale platforms and services.
- Expert proficiency in one or more programming languages, such as Python, Java, Scala, or Go.
- Demonstrated experience designing and leading adoption of agentic AI-enabled development practices across teams.
- Experience designing and operating high-scale inference and distributed systems, with practical cloud-native production experience and clear SLOs.
Nice to have
- Deep experience with LLM inference optimization, quantization, or disaggregated serving architectures.
- Experience building model routing and governance layers or enterprise-scale multi-tenant platforms.
Pay
Jersey City,NJ: $204,250.00-$285,000.00 USD base salary; Palo Alto,CA: $204,250.00-$285,000.00 USD base salary; Seattle,WA: $204,250.00-$285,000.00 USD base salary; New York,NY: $204,250.00-$285,000.00 USD base salary
Tools
Python, Java, Scala, Go, TensorRT, INT8, INT4
AI focus
Builds and optimizes enterprise GenAI serving and LLM inference, including intelligent model routing and GPU efficiency; also governs agentic AI-enabled engineering workflows and enterprise-authorized AI-assisted development.
Model-written summary · the original posting is authoritative