> Markdown version of [/jobs/ext/4346710-principal-engineer-inference-memory-and-storage-systems](https://www.wearedevelopers.com/jobs/ext/4346710-principal-engineer-inference-memory-and-storage-systems). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Principal Engineer, Inference Memory and Storage Systems

- **Company:** DigitalOcean, LLC
- **Location:** Seattle, WA, United States
- **Salary:** $249,600.0 - $312,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, C++ (Programming Language), Distributed Systems, Dynamic Random-Access Memory, Python (Programming Language), Object-Oriented Software Development, Open Source Technology, Peer-To-Peer (P2P), Remote Direct Memory Access, Digitalocean, Distributed Caching, System Programming, Network Routers, Large Language Models, Kubernetes, Low Latency, SGLang, Machine Learning Operations, TensorRT, Hardware Infrastructure, VLLM, Model Inference, Decoding, Nvme
- **Published:** October 7, 2026
- **Apply:** https://www.dice.com/job-detail/f9f786b4-2b5c-4b5c-bddd-d4188210546c

## About the Role

* 15+ years building large-scale distributed systems, high-performance storage, or ML systems infrastructure, with a track record of delivering and operating production services
 * Deep fluency in memory hierarchies (GPU HBM, host DRAM, NVMe, remote/object storage) and experience designing systems that span tiers for both performance and cost
 * Experience with distributed caching or key-value systems, particularly designs optimized for low latency under high concurrency
 * Hands-on work with networked I/O and RDMA / NVMe-oF / NVLink-class technologies, and familiarity with aggregated and disaggregated deployment topologies for AI clusters
 * Strong systems programming skills in C/C++, Go, Rust, or Python, and comfort reading and modifying serving-engine internals
 * Rigor in profiling and optimization across CPU, GPU, memory, and network, using measurement to drive architectural decisions and to validate improvements in TTFT, ITL, and throughput
 * Excellent written and verbal communication, and a history of leading cross-functional efforts with product, infrastructure, and customer-facing teams

Bonus:

 * Contributions to open source LLM serving or inference infrastructure projects-vLLM, SGLang, llm-d, NVIDIA Dynamo, LMCache, or similar-especially on KV cache offload, compression, or reuse
 * Experience designing a unified memory or storage layer that presents a single logical KV or object model across GPU, host, SSD, and cloud tiers in a hyperscale or public cloud environment
 * Familiarity with Kubernetes-based GPU orchestration, including DRA, MIG/MPS partitioning, and gateway/inference-extension routing patterns
 * Publications or patents in LLM systems, memory-disaggregated architectures, RDMA-based data planes, or CDN-like caching systems for ML workloads, * We innovate with purpose. You'll be a part of a cutting-edge technology company with an upward trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As a member of the team, you will be a Shark who thinks big, bold, and scrappy, like an owner with a bias for action and a powerful sense of responsibility for customers, products, employees, and decisions.

## Description

That layer is currently a collection of good decisions made independently. This role exists to make it one system: a unified, multi-tier memory and storage substrate spanning GPU HBM, host DRAM, local NVMe, and remote object storage, exposed through a coherent interface to every serving engine and router we run. The selected candidate will set direction across teams, contribute meaningfully to the open source projects we depend on, and translate hard systems work into unit economics our customers feel.
What You'll Be Doing:

 * Defining and evolving a unified memory layer spanning GPU memory, pinned host memory, RDMA-accessible memory, local NVMe tiers, and remote object storage for large-scale LLM inference
 * Architecting deep integrations with the serving engines we run in production (vLLM, SGLang, TensorRT-LLM), focused on KV cache offload, reuse, eviction policy, and cross-node sharing
 * Designing the interfaces and protocols behind disaggregated prefill/decode, peer-to-peer KV cache transfer, and cache-aware routing-including how the router, the engine, and the cache tier agree on what is resident and where
 * Owning the eviction and admission story across tiers (LRU and its successors, cost-aware policies, pluggable backends) and the metrics that prove those policies work under real traffic
 * Partnering with our GPU infrastructure, networking, and platform teams to exploit GPUDirect, RDMA, NVMe-oF, and NVLink for low-latency cache access across heterogeneous accelerator pools
 * Driving the unit economics: modeling and validating the effect of cache hit ratio, prefix caching, and tiering on $/token and $/GPU-hr, and making the tradeoffs legible to product and pricing partners
 * Setting technical direction and raising the bar through design review, mentorship of senior and staff engineers, and sponsorship of the initiatives that follow from this roadmap
 * Representing DigitalOcean externally-upstream contributions, conference talks, and customer-facing technical deep dives