> Markdown version of [/jobs/ext/1330374-senior-software-engineer-ai-inference](https://www.wearedevelopers.com/jobs/ext/1330374-senior-software-engineer-ai-inference). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Software Engineer - AI Inference - **Company:** NVIDIA Ltd. - **Location:** Santa Clara, CA, United States - **Experience:** Expert - **Salary:** $152,000.0 - $241,500.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Systems Engineering, C++ (Programming Language), Code Review, Nvidia CUDA, Computer Programming, Computer Engineering, Software Debugging, Distributed Systems, InfiniBand, Python (Programming Language), Open Source Technology, PCI Express, Regression Testing, Multithreading, Graphics Processing Unit (GPU), Pytorch, Large Language Models, Information Technology, Free and Open-Source Software - **Published:** July 18, 2026 - **Apply:** https://www.juju.com/job/00000000ghgm4s ## About the Role + 5+ years building production software with solid systems engineering fundamentals and a track record of delivering performance or reliability improvements. + Experience with LLM inference/serving stacks (e.g., vLLM, SGLang) and an understanding of the tradeoffs that drive real production performance. + Strong programming skills in Python plus C++ and/or CUDA; ability to debug and optimize performance-critical code. + Experience with profiling and performance investigation (microbenchmarks, flame graphs, GPU profiling) and a measurement-driven mindset. + Familiarity with distributed systems concepts and concurrency (queues/schedulers, multi-process/multi-threading, scaling across GPUs/nodes). + Strong communication skills and comfort working with open-source communities (issues, PR discussions, code review). + BS/MS in Computer Science, Computer Engineering, or related field (or equivalent experience). Ways to stand out from the crowd: + Open-source contributions to vLLM, SGLang, PyTorch, Triton, NCCL, Dynamo or adjacent serving/runtime projects. + Shipped performance work such as improved attention/KV cache efficiency, speculative decoding, scheduler improvements, quantization-aware serving, or streaming latency reductions. + Experience building reproducible benchmarking and performance regression infrastructure for latency/throughput. + Systems performance background spanning memory bandwidth, kernel fusion, PCIe/NVLink effects, and network fabrics (e.g., InfiniBand). We are widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and creative people in the world working for us. If you're creative and autonomous with a real passion for technology, we want to hear from you. ## Description NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Senior Software Engineer - AI Inference to advance open-source LLM serving by contributing directly to upstream inference engines like vLLM and SGLang-ensuring they run best-in-class on NVIDIA GPUs and systems-and by improving the underlying stack that enables high-throughput, low-latency inference at scale. This is a hands-on role for an engineer who enjoys digging into performance bottlenecks, designing pragmatic runtime improvements, and shipping high-quality changes that are broadly useful to the community and production deployments. What you'll be doing: + Contribute features, fixes, and optimizations upstream to vLLM/SGLang: author PRs, participate in reviews, write benchmarks/tests, and help drive designs to completion. + Implement and optimize inference-runtime capabilities: batching and scheduling policies, streaming, request lifecycle management, and KV-cache efficiency (paging/sharding) to improve throughput and tail latency. + Profile and improve hot paths across layers-from Python orchestration to C++/CUDA kernels-using data to guide optimization work. + Improve multi-GPU inference performance and reliability: parallelism strategies, communication patterns, and resource utilization across NVIDIA platforms. + Build and maintain performance and correctness regression tests to prevent slowdowns and ensure stable behavior across model and hardware configurations. + Collaborate with model, platform, and SRE teams to translate production requirements into upstreamable solutions with strong operability and maintainability. ## Related Videos - [Coffee with Developers - Stephen Jones - NVIDIA](https://www.wearedevelopers.com/videos/1303-coffee-with-developers-stephen-jones-nvidia) - [Are Code Reviews Worth It? Insights from 16 Years of Review Data](https://www.wearedevelopers.com/videos/1135-are-code-reviews-worth-it-insights-from-16-years-of-review-data) - [Photonic Computing: Programming a New Class of AI Accelerators (incl. Live Coding)](https://www.wearedevelopers.com/videos/100196-photonic-computing-programming-a-new-class-of-ai-accelerators-incl-live-coding) - [Your Next AI Needs 10,000 GPUs. Now What?](https://www.wearedevelopers.com/videos/1590-your-next-ai-needs-10-000-gpus-now-what) - [Serverless deployment of (large) NLP models ](https://www.wearedevelopers.com/videos/158-serverless-deployment-of-large-nlp-models) - [Build a CI/CD pipeline to automate code reviews and ensure code quality](https://www.wearedevelopers.com/videos/349-build-a-ci-cd-pipeline-to-automate-code-reviews-and-ensure-code-quality) ## Related Articles - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Best US AI Conferences for CTOs in 2026: Build vs. Buy, Vendor Evaluation, and Peer Intelligence](https://www.wearedevelopers.com/magazine/736-best-us-ai-conferences-for-ctos-in-2026-build-vs-buy-vendor-evaluation-and-peer-intelligence)