FriendliAI
Software Engineer – AI Inference Engine
San Francisco · Hybrid · Full-time · Posted Mar 15, 2026
- Experience
- 5+ years in production or high-impact research environments.
- Education
- Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent.
About the job
Optimize FriendliAI’s inference engine by building GPU kernels and core infrastructure for fast, efficient generative AI workloads.
What you'll do
- Design and optimize GPU kernels; analyze bottlenecks across software and hardware.
- Develop core components such as the kernel compiler, memory planner, and runtime, and support new model architectures.
- Collaborate on end-to-end inference performance and maintain profiling, benchmarking, and validation tools.
What they ask for
- Production-level expertise in Python and C++.
- Experience developing machine learning frameworks or performance-critical runtime systems.
- Hands-on experience writing, optimizing, and profiling GPU kernels.
- Experience with generative AI models such as transformer and diffusion models.
Nice to have
- Experience developing machine learning compilers or code generation systems.
- Familiarity with dynamic shape compilation, memory planning, kernel fusion, or multi-GPU and distributed inference.
Tools
Python, C++, GPU kernels
AI focus
Optimize inference for generative and agentic AI workloads, including transformer and diffusion models.
Model-written summary · the original posting is authoritative