> Markdown version of [/jobs/ext/2229746-inference-performance-engineer](https://www.wearedevelopers.com/jobs/ext/2229746-inference-performance-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Inference Performance Engineer

- **Company:** ADAPTION LLC
- **Location:** United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** C++ (Programming Language), Nvidia CUDA, Python (Programming Language), Large Language Models, Machine Learning Operations, TensorRT, Decoding
- **Published:** August 25, 2026
- **Apply:** https://arc.dev/remote-jobs/j/redirect/pf1grtvdwz

## About the Role

* 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.
 * Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
 * Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
 * Strong Python skills and proficiency in C++, Rust, or another systems language.
 * Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Above all, we're looking for great teammates who make work feel lighter and aren't afraid to go out on a limb with bold ideas. You don't need to be perfect, but you do need to be adaptable. We encourage you to apply, even if you don't check every box.

## Description

You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change.

You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality.

Responsibilities

 * Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
 * Optimize long-context prefill and decode workloads based on real production traffic.
 * Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
 * Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
 * Build profiling and measurement systems that show where time, memory, and compute are being spent.


## Related Videos

- [Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated](https://www.wearedevelopers.com/videos/100099-tour-de-force-open-source-llm-inference-optimization-from-simple-to-sophisticated)
- [Efficient deployment and inference of GPU-accelerated LLMs​](https://www.wearedevelopers.com/videos/929-efficient-deployment-and-inference-of-gpu-accelerated-llms)
- [Coffee with Developers - Stephen Jones - NVIDIA](https://www.wearedevelopers.com/videos/1303-coffee-with-developers-stephen-jones-nvidia)
- [Trends, Challenges and Best Practices for AI at the Edge](https://www.wearedevelopers.com/videos/630-trends-challenges-and-best-practices-for-ai-at-the-edge)
- [The weekly developer show: Boosting Python with CUDA, CSS Updates & Navigating New Tech Stacks](https://www.wearedevelopers.com/videos/1293-the-weekly-developer-show-boosting-python-with-cuda-css-updates-navigating-new-tech-stacks)
- [Challenges and Solutions for Efficient, Large-Scale Video Analysis](https://www.wearedevelopers.com/videos/2022-challenges-and-solutions-for-efficient-large-scale-video-analysis)

## Related Articles

- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Dev Digest 157: CUDA in Python, Gemini Code Assist and Back-dooring LLMs](https://www.wearedevelopers.com/magazine/557-dev-digest-157-cuda-in-python-gemini-code-assist-and-back-dooring-llms)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)