Parallel Wireless
Senior/Principal Local LLM & Generative AI Platform Engineer
US · Remote · Principal · Full-time · Posted Sep 9, 2026
- Experience
- Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems; exceptional candidates with equivalent depth are welcome.
- Education
- BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.
About the job
Build and operate a secure, production-ready local LLM platform for internal teams, including model serving, permission-aware retrieval, integrations, evaluation, security, and operations.
What you'll do
- Own the platform architecture and roadmap, and build reusable inference APIs, model routing, and integrations with internal workflows.
- Develop and operate enterprise search and RAG pipelines with source permissions, citations, freshness, and deletion controls.
- Establish evaluation, security, observability, release, and operational practices for reliable production use.
What they ask for
- Strong Python engineering skills and experience designing APIs, services, libraries, and data pipelines.
- Demonstrated experience building production RAG or enterprise-search systems and evaluating retrieval quality.
- Experience deploying and operating containerized Linux services using Docker and Kubernetes or an equivalent orchestration environment.
- Experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering.
Nice to have
- Experience operating LLMs in on-premises, private-cloud, restricted-network, or air-gapped environments.
- Hands-on experience with inference runtimes or serving systems such as vLLM, SGLang, TensorRT-LLM, llama.cpp, Ray Serve, KServe, or Triton.
Tools
Python, Go, Java, C/C++, Docker, Kubernetes, vLLM, SGLang
AI focus
Build a secure local generative-AI platform using open-weight models, RAG, enterprise search, evaluation, model serving, and safe tool workflows; assess model adaptation when measured gains justify it.
Model-written summary · the original posting is authoritative