> Markdown version of [/jobs/ext/4462639-inference-sw-engineer](https://www.wearedevelopers.com/jobs/ext/4462639-inference-sw-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Inference SW Engineer - **Company:** Hewlett-Packard Enterprise - **Location:** Fort Collins, CO, United States (Remote available) - **Experience:** Expert - **Salary:** $137,000.0 - $315,000.0 - **Contract:** Permanent contract - **Skills:** Multitier Architecture, Artificial Intelligence, C++ (Programming Language), Computational Fluid Dynamics, Cloud Computing, Nvidia CUDA, Computer Programming, Creo Elements/Pro (Computer-Aided Design Software), Software Debugging, InfiniBand, Python (Programming Language), Remote Direct Memory Access, Software Engineering, Software Technical Review, Private Cloud Environment, Enterprise Software Applications, Retrieval-Augmented Generation, Large Language Models, Discretization, Kubernetes, Information Technology, SGLang, TensorRT, Nim (Programming Language), VLLM, Model Inference, KServe - **Published:** October 7, 2026 - **Apply:** https://www.builtincolorado.com/job/senior-inference-sw-engineer/11537405?handler=ApplyRedirect ## About the Role · Strong understanding of inference internals, including continuous batching, paged attention, KV cache reuse and prefix caching, chunked prefill, quantization, and speculative decoding · Working knowledge of tensor and pipeline parallelism, NCCL collective operations, and the GPU memory hierarchy and interconnect characteristics that govern them · Advanced proficiency in Kubernetes platform architectures, including operators, custom resources, controllers, and scheduling · Strong programming proficiency in Go and Python, with the ability to read, debug, and profile C++/CUDA using tools such as Nsight · Familiar with debugging/profiling multi-tier application workloads such as RAG, Agents · Excellent analytical, debugging, and problem-solving abilities Preferred · Upstream contribution to vLLM, SGLang, TensorRT-LLM, llm-d, LMCache, or KServe · Disaggregated prefill/decode serving, or KV cache offload and reuse at scale · RDMA, GPUDirect Storage, InfiniBand, or RoCE · MIG, fractional GPU allocation, and multi-tenant GPU isolation · On-premises, air-gapped, or regulated enterprise software delivery Experience and Education · Minimum of 8 years of experience in Software Engineering, including 1-2+ years working directly on LLM inference runtimes or production model serving · Degree in Computer Science or related field ## Description This role has been designed as 'Hybrid' with a requirement that you will work on average 2 days per week from an HPE office., · Design, implement, and own major components of the LLM serving deployment, including engine integration, continuous batching, KV cache management and reuse, and quantized execution · Partner with inference engineering teams and contribute to improving time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency · Build and operate distributed execution capabilities, including disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU memory, host memory, and RDMA-attached storage · Evaluate emerging runtimes, quantization schemes, speculative decoding, and mixture-of-experts serving, and make well-supported recommendations on adoption · Contribute to the orchestration layer supporting the runtime, including model admission, GPU scheduling and partitioning, cache-aware request routing, and autoscaling · Triage and resolve customer issues end-to-end, identifying root causes and improving systems and processes to prevent recurrence · Provide insightful code and design reviews, mentor team members, and lead by example on engineering practices within the team Knowledge and Skills Required · Familiar with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI, or NVIDIA NIM, including modification of engine internals, All legitimate job opportunities will come through official company channels, and candidates are responsible for verifying the credentials of any third party claiming to represent the company. Any reliance on fraudulent communication is at the individual's own risk, and HPE disclaims legal liability for any resulting damages. If you suspect recruitment fraud, do not share personal information or make any payments and report the incident to your local authorities immediately., Leads long-term strategy and business planning for HPE Server Services. Develops strategic roadmaps, financial and business-driver models, forecasts, KPIs, competitive analyses, and performance reviews. Partners with Product Management, Sales, Finance, and Operations, advises senior executives, and oversees strategic initiatives from planning through execution. Drives resource allocation, operational efficiency, profitability, and alignment across global teams. Top Skills: Ai ToolsExcelPower BI Hewlett Packard Enterprise Thermal Engineer (Direct Liquid Cooling) - Houston, TX 5 Hours Ago In-Office 96K-221K Annually Mid level 96K-221K Annually Mid level Artificial Intelligence * Cloud * Information Technology * Consulting Leads mechanical and thermal design for liquid-cooled servers and rack architectures. Develops cold plates, manifolds, fluid connections, tubing, and serviceability features; performs CFD, thermal analysis, and laboratory testing. Creates qualification plans covering thermal, leak, structural, vibration, materials, corrosion, and manufacturing reliability. Drives root-cause analysis, supplier and manufacturing partner engagements, technical reviews, and design decisions. Mentors engineers while ensuring cooling solutions meet performance, reliability, acoustic, power, and customer requirements. Top Skills: 3D CadComputational Fluid Dynamics (Cfd)Coolant ManifoldsCpu/Gpu Cold PlatesCreo (Pro/Engineer)Direct Liquid CoolingQuick DisconnectsThermal Interface MaterialsThermal Modeling Hewlett Packard Enterprise