> Markdown version of [/jobs/ext/4635487-software-engineer-gpu-networking-distributed-systems](https://www.wearedevelopers.com/jobs/ext/4635487-software-engineer-gpu-networking-distributed-systems). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Software Engineer - GPU Networking & Distributed Systems

- **Company:** Baseten, Inc
- **Location:** New York, NY, United States
- **Contract:** Permanent contract
- **Skills:** C++ (Programming Language), Data Transmissions, Extract Transform Load (ETL), Software Debugging, Distributed Systems, InfiniBand, Python (Programming Language), Network Protocols, Remote Direct Memory Access, TCP/IP, Weka, Large Language Models, Distributed Inference, Kubernetes, SGLang, TensorRT, Serverless Computing
- **Published:** November 8, 2026
- **Apply:** https://www.juju.com/job/28_25303_0a4b07bc7446cfdb70a0372f4e989756

## About the Role

* Deep experience with high-performance networking protocols (InfiniBand, RoCE v2) and understanding of the physics of data movement.
 * Fluency in C++ or Python, bridging high-level logic and hardware. Deep understanding of the memory hierarchy in modern NVIDIA architectures (H100/Blackwell) and know how to optimize for it.
 * Enjoy diving deep into code, writing custom C++/Python bindings, or debugging NVLink topology issues.
 * Know when to use an off-the-shelf solution and when we need to build a custom solution because the upstream tools (like standard Kubernetes networking) are too slow for our needs., * Deep knowledge of NCCL, NVSHMEM, and UCX.
 * Experience with GPUDirect Storage (GDS) or high-performance filesystems like Weka or 3FS.
 * Familiarity with TensorRT-LLM, vLLM, or Sglang.
 * Experience running low-level benchmarks to "qualify" new hardware clusters.

## Description

* Make RDMA First-Class: work on integrating RDMA/RoCE/InfiniBand capabilities directly into our inference stack, moving beyond TCP/IP to unlock order-of- magnitude improvements in bandwidth and latency.
 * Optimize Distributed Inference: implement and tune the networking layers necessary for efficient Disaggregated KV Cache Offload and WideEP, ensuring seamless communication across NVLink and InfiniBand for our MoE models.
 * Enable Serverless-Grade Startup Speeds for LLMs: work deeply with checkpointing and storage mechanisms to enable sub-10-second startup for trillion-parameter models.
 * Deep-Dive into Hardware: characterize and validate networking performance on bleeding-edge clusters (H100/H200, B200/B300, GB200/300 NVL72), writing acceptance tests that ensure our hardware delivers peak achievable throughput and minimal latency.
 * Build Observability: design the tools that let us visualize packet flow, congestion, and effective bandwidth across the GPU interconnects, helping us diagnose complex distributed system behaviors.
 * Optimize Kernels: work with communication libraries (NCCL, NVSHMEM) and potentially write custom communication kernels to overlap compute and data transfer., * Bleeding Edge Hardware: We are preparing to bring Blackwell (B200/B300) and then Rubin architectures online. You will be one of the first engineers in the industry optimizing networking for NVL72/GB300 racks.
 * We go deep: We operate at every depth-from tuning hardware interconnects to writing custom communication kernels and designing distributed inference strategies-across the entire stack to deliver performance that goes far and beyond.
 * High Impact: The networking optimizations you build will directly enable features that no one else in the industry has fully mastered yet, like seamless multi-node WideEP and instant model hydration.