> Markdown version of [/jobs/ext/3896903-senior-software-engineer-private-cloud-compute-inference-routing](https://www.wearedevelopers.com/jobs/ext/3896903-senior-software-engineer-private-cloud-compute-inference-routing). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Software Engineer, Private Cloud Compute Inference Routing

- **Company:** Apple Inc.
- **Location:** Cupertino, CA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Java (Programming Language), C Sharp (Programming Language), Unix, C++ (Programming Language), Client Server Models, Software Debugging, Distributed Systems, Cryptographic Protocols, Key Management, Routing, Network Protocols, Performance Tuning, X.509, Reliability Engineering, Private Cloud Environment, Load Balancing, Caching, Kubernetes, Low Latency, SGLang, Machine Learning Operations, VLLM, Model Inference, Golang
- **Published:** October 23, 2026
- **Apply:** https://www.dice.com/job-detail/36e5b341-1a94-402a-b3ab-7096e3b97305

## About the Role

3+ years of professional experience designing, building and operating large distributed systems in production

Server development in Java, C/C++, C#, Go or Swift

Knowledge of LLM inference serving, including inference request routing, KV-cache placement and reuse (e.g., prefix-aware routing), prefill/decode disaggregation, and the tradeoffs between latency, throughput and accelerator use

Experience with network protocols, application level networking and client-server architectures, including request routing, load balancing or scheduling at scale

Experience benchmarking, debugging and tuning the performance of distributed systems, plus knowledge of Unix system internals

Experience with cloud deployment and observability technologies such as Kubernetes

Bachelor's degree in an engineering related field or equivalent experience

Preferred Qualifications

Experience building LLM inference serving or routing systems such as llm-d, NVIDIA Dynamo, vLLM, SGLang or similar

Familiarity with KV cache aware routing, prefix caching, prefill/decode disaggregation and accelerator capacity placement

Experience with TLS, X.509 certificate evaluation or other secure protocols, and awareness of recent developments in standard protocols

Understanding of cryptographic protocols, encryption, key management and privacy-preserving system design (e.g., Oblivious HTTP, remote attestation)

Experience leading a substantial engineering project from design through production, resolving technical tradeoffs and coordinating across teams.

Ability to explain technical tradeoffs, collaborate across engineering teams, and make progress on problems with incomplete requirements.

## Description

Routing LLM inference at Apple's scale requires systems that adapt to changing demand, available capacity, and failures while maintaining low latency and high utilization. PCC adds a fundamental architectural requirement: request content remains encrypted outside trusted compute nodes. You'll develop routing and scheduling strategies within this privacy model, working with inference and security teams to evaluate techniques such as cache aware request placement and prefill/decode disaggregation.

You'll design system architectures, specify custom network protocols and write high performance code. You'll also build scaling strategies for services used by hundreds of millions of people. Our team is small and highly visible, so your chances to make an impact are wide open. If you take opportunities like that and run with them, you'll enjoy working here.

You'll own technical decisions from early design through production, using benchmarks and operational evidence to evaluate tradeoffs. You'll work closely with model serving, infrastructure, security, and SRE engineers to turn new inference requirements into reliable routing capabilities.