> Markdown version of [/jobs/ext/3968841-senior-inference-sw-engineer](https://www.wearedevelopers.com/jobs/ext/3968841-senior-inference-sw-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Inference SW Engineer

- **Company:** Hewlett-Packard Enterprise
- **Location:** The Woodlands, TX, United States (Remote available)
- **Experience:** Expert
- **Salary:** $144,000.0 - $273,000.0
- **Contract:** Permanent contract
- **Skills:** Multitier Architecture, Artificial Intelligence, C++ (Programming Language), Cloud Computing, Nvidia CUDA, Computer Programming, Continuous Integration, Software Debugging, InfiniBand, Python (Programming Language), Language Modeling, Remote Direct Memory Access, Software Engineering, Software Technical Review, Private Cloud Environment, Graphics Processing Unit (GPU), Enterprise Software Applications, Retrieval-Augmented Generation, Large Language Models, Discretization, Kubernetes, Information Technology, SGLang, TensorRT, Nim (Programming Language), VLLM, Model Inference, Decoding, KServe
- **Published:** October 27, 2026
- **Apply:** https://www.careerbuilder.com/job-details/senior-inference-sw-engineer-spring-tx--b1c02607-f52f-415f-843b-22d6aeaae149

## About the Role

Software Engineering, Software Engineering, Hewlett-Packard Product Family, Analysis Skills, Cloud Computing, Private Cloud, Artificial Intelligence (AI), GPU (Graphics Processing Unit), Modeling Languages, Continuous Integration, * Familiar with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI, or NVIDIA NIM, including modification of engine internals

 * Strong understanding of inference internals, including continuous batching, paged attention, KV cache reuse and prefix caching, chunked prefill, quantization, and speculative decoding
 * Working knowledge of tensor and pipeline parallelism, NCCL collective operations, and the GPU memory hierarchy and interconnect characteristics that govern them
 * Advanced proficiency in Kubernetes platform architectures, including operators, custom resources, controllers, and scheduling
 * Strong programming proficiency in Go and Python, with the ability to read, debug, and profile C++/CUDA using tools such as Nsight
 * Familiar with debugging/profiling multi-tier application workloads such as RAG, Agents
 * Excellent analytical, debugging, and problem-solving abilities

Preferred

 * Upstream contribution to vLLM, SGLang, TensorRT-LLM, llm-d, LMCache, or KServe
 * Disaggregated prefill/decode serving, or KV cache offload and reuse at scale
 * RDMA, GPUDirect Storage, InfiniBand, or RoCE
 * MIG, fractional GPU allocation, and multi-tenant GPU isolation
 * On-premises, air-gapped, or regulated enterprise software delivery, * Minimum of 8 years of experience in Software Engineering, including 1-2+ years working directly on LLM inference runtimes or production model serving
 * Degree in Computer Science or related field

## Description

This role has been designed as ''Hybrid'' with a requirement that you will work on average 2 days per week from an HPE office., * Design, implement, and own major components of the LLM serving deployment, including engine integration, continuous batching, KV cache management and reuse, and quantized execution

 * Partner with inference engineering teams and contribute to improving time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency
 * Build and operate distributed execution capabilities, including disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU memory, host memory, and RDMA-attached storage
 * Evaluate emerging runtimes, quantization schemes, speculative decoding, and mixture-of-experts serving, and make well-supported recommendations on adoption
 * Contribute to the orchestration layer supporting the runtime, including model admission, GPU scheduling and partitioning, cache-aware request routing, and autoscaling
 * Triage and resolve customer issues end-to-end, identifying root causes and improving systems and processes to prevent recurrence
 * Provide insightful code and design reviews, mentor team members, and lead by example on engineering practices within the team