Samsung
Technical Director, Large-Scale AI Model Inferencing
$219K–$351K
San Jose, California, United States · Director · Full-time · Posted Sep 9, 2026
- Experience
- 12+ years in systems engineering, with 4+ years hands-on in large-scale LLM inference or GPU systems performance. The posting also states a BS plus 10+ years of relevant experience; an MS with 8 years of relevant experience is preferred.
- Education
- BS in Computer/Electrical/Electronic Engineering or Computer Science. MS in those fields preferred.
About the job
Leads technical strategy and hands-on development of full-stack, tiered memory solutions for large-scale AI inference, connecting model behavior, inference software, and memory hardware.
What you'll do
- Define and validate tiered memory requirements, architectures, and performance models across GPU HBM, host DRAM, CXL memory, and NVMe/SSD.
- Optimize and benchmark production inference performance, including memory management, caching, offloading, and latency/throughput/cost tradeoffs.
- Set technical strategy, guide architecture reviews and open-source contributions, represent the company with customers, and mentor senior engineers.
What they ask for
- 12+ years in systems engineering, including 4+ years hands-on in large-scale LLM inference or GPU systems performance.
- First-principles understanding of transformer internals and working expertise with MoE behavior.
- Code-level experience with the memory-management internals of at least one major inference stack.
- Production systems-software experience at the memory, storage, or I/O layer; strong performance-engineering and modeling skills.
Nice to have
- Experience with State Space Models or hybrid SSM-attention serving.
- Experience with CXL memory pooling, SSD/NVMe cache tiers, or open-source inference and caching contributions.
Pay
$219,000 — $351,000 USD (base salary)
Tools
SGLang, vLLM, NVIDIA Dynamo, TensorRT-LLM, llama.cpp, LMCache, CXL, NVMe/SSD
AI focus
Develops and optimizes large-scale AI model inference through model-aware memory systems, including tiered caching, KV-cache management, MoE expert-weight offloading, benchmarking, and inference performance engineering.
Model-written summary · the original posting is authoritative