> Markdown version of [/jobs/ext/4492368-staff-software-engineer-inference](https://www.wearedevelopers.com/jobs/ext/4492368-staff-software-engineer-inference). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Staff Software Engineer, Inference - **Company:** Humanloop - **Location:** London, UK - **Salary:** £325,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Microsoft Azure, Cloud Computing, Distributed Systems, Routing, Software Engineering, Graphics Processing Unit (GPU), Load Balancing, Cloud Platform System, Autoscaling, Delivery Pipeline, Caching, Kubernetes, Machine Learning Operations, Model Inference - **Published:** November 2, 2026 - **Apply:** https://www.apply4u.co.uk/jobs/staff-software-engineer-inference/54832297 ## About the Role Software engineering experience building and operating distributed systems in production Working knowledge of containerized infrastructure (e.g., Kubernetes) and at least one major cloud platform (AWS, GCP, or Azure) Results-oriented, with a bias towards flexibility and impact Willingness to pick up slack, even if it goes outside your job description Desire to learn more about machine learning systems and infrastructure Thrive in environments where technical excellence directly drives both business results and research breakthroughs Care about the societal impacts of your work Preferred qualifications Significant experience with high-performance, large-scale distributed systems Experience implementing and deploying machine learning systems at scale Experience building load balancing, request routing, or traffic management systems Familiarity with LLM inference optimization, batching, and caching strategies Deep experience operating Kubernetes and cloud infrastructure at scale Experience with AI accelerator platforms (GPUs, TPUs, or emerging hardware), Logistics Minimum education: Bachelor's degree or an equivalent combination of education, training, and/or experience Required field of study: A field relevant to the role as demonstrated through coursework, training, or professional experience Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position ## Description Representative projects Designing intelligent routing algorithms that optimize request distribution across many accelerators in different environments Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads Building production-grade deployment pipelines for releasing new models to millions of users reliably Contributing to new inference features Supporting inference for new model architectures Analyzing observability data to tune performance based on real-world production workloads Managing multi-region deployments and geographic routing for global customers Deadline to apply: None. Applications will be reviewed on a rolling basis. The annual compensation range for this role is listed below. For sales roles, the range provided is the role's On Target Earnings (\"OTE\") range, meaning that the range includes both the sales commissions/sales bonuses target and annual base salary for the role.