> Markdown version of [/jobs/ext/4323043-principal-software-engineer-large-scale-llm-memory-and-storage-systems](https://www.wearedevelopers.com/jobs/ext/4323043-principal-software-engineer-large-scale-llm-memory-and-storage-systems). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Principal Software Engineer - Large-Scale LLM Memory and Storage Systems

- **Company:** NVIDIA Corporation
- **Location:** Santa Clara, CA, United States
- **Salary:** $272,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Systems Engineering, C++ (Programming Language), Cloud Storage, Profiling, Distributed Systems, Dynamic Random-Access Memory, Memory Management, Python (Programming Language), Object-Oriented Software Development, Open Source Technology, Peer-To-Peer (P2P), Remote Direct Memory Access, Distributed Caching, Graphics Processing Unit (GPU), Large Language Models, Low Latency, SGLang, Machine Learning Operations, TensorRT, VLLM, Model Inference, Nvme
- **Published:** October 31, 2026
- **Apply:** https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Principal-Software-Engineer---Large-Scale-LLM-Memory-and-Storage-Systems_JR2010271

## About the Role

* Masters or PhD or equivalent experience
 * 15+ years of experience building large-scale distributed systems, high-performance storage, or ML systems infrastructure in C/C++ and Python, with a track record of delivering production services.
 * Deep understanding of memory hierarchies (GPU HBM, host DRAM, SSD, and remote/object storage) and experience designing systems that span multiple tiers for performance and cost efficiency.
 * Distributed caching or key-value systems, especially designs optimized for low latency and high concurrency.
 * Hands-on experience with networked I/O and RDMA/NVMe-oF/NVLink-style technologies, and familiarity with concepts like disaggregated and aggregated deployments for AI clusters.
 * Strong skills in profiling and optimizing systems across CPU, GPU, memory, and network, using metrics to drive architectural decisions and validate improvements in TTFT and throughput.
 * Excellent communication skills and prior experience leading cross-functional efforts with research, product, and customer teams.

Ways to stand out from the crowd:

 * Prior contributions to open-source LLM serving or systems projects focused on KV-cache optimization, compression, streaming, or reuse.
 * Experience designing unified memory or storage layers that expose a single logical KV or object model across GPU, host, SSD, and cloud tiers, especially in enterprise or hyperscale environments.
 * Publications or patents in areas such as LLM systems, memory-disaggregated architectures, RDMA/NVLink-based data planes, or KV-cache/CDN-like systems for ML.

## Description

We are seeking a Principal Systems Engineer to define the vision and roadmap for memory management of large-scale LLM and storage systems.

What you'll be doing:

 * Design and evolve a unified memory layer that spans GPU memory, pinned host memory, RDMA-accessible memory, SSD tiers, and remote file/object/cloud storage to support large-scale LLM inference.
 * Architect and implement deep integrations with leading LLM serving engines (such as vLLM, SGLang, TensorRT-LLM), with a focus on KV-cache offload, reuse, and remote sharing across heterogeneous and disaggregated clusters.
 * Co-design interfaces and protocols that enable disaggregated prefill, peer-to-peer KV-cache sharing, and multi-tier KV-cache storage (GPU, CPU, local disk, and remote memory) for high-throughput, low-latency inference.
 * Partner closely with GPU architecture, networking, and platform teams to exploit GPUDirect, RDMA, NVLink, and similar technologies for low-latency KV-cache access and sharing across heterogeneous accelerators and memory pools.
 * Mentor senior and junior engineers, set technical direction for memory and storage subsystems, and represent the team in internal reviews and external forums (open source, conferences, and customer-facing technical deep dives).