> Markdown version of [/jobs/ext/2041592-senior-lead-software-engineer-llm-ops-platform-reliability](https://www.wearedevelopers.com/jobs/ext/2041592-senior-lead-software-engineer-llm-ops-platform-reliability). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Lead Software Engineer - LLM Ops Platform Reliability - **Company:** JPMorgan Chase & Co. - **Location:** Christchurch, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Automation of Tests, Cloud Computing, Computer Clusters, Software Quality, Code Review, Continuous Delivery, Memory Management, Python (Programming Language), Open Source Technology, Reliability Engineering, Software Tools, Runbook, Search Technologies, Secure Coding, Software Engineering, Systems Integration, Strategies of Testing, AI Infrastructure, Data Logging, Cloud Platform System, Autoscaling, Retrieval-Augmented Generation, System Availability, Large Language Models, Multi-Agent Systems, Parallel Computation, Backend, Kubernetes, Low Latency, Machine Learning Operations, Code Restructuring - **Published:** August 13, 2026 - **Apply:** https://jobs.theguardian.com/job/10172807/senior-lead-software-engineer-llm-ops-platform-reliability/ ## About the Role * Hands-on experience with system design, application development, testing, and operational stability in production environments * Advanced proficiency in Python for building production-grade services and tooling * Proficiency with automation and continuous delivery methods * Hands-on experience with cloud infrastructure platforms and infrastructure-as-code tooling for delivery and lifecycle management * Strong understanding of site reliability engineering practices, including incident management, root-cause analysis, runbooks, and reliability patterns * Practical knowledge of observability and instrumentation across metrics, logs, and traces * Hands-on experience with Kubernetes and container-based orchestration platforms, including managed cloud variants * Experience hosting and serving large language models on cloud-based infrastructure and local GPU environments * Knowledge of large language model reliability and risk considerations, including latency and throughput trade-offs, model versioning, prompt and response logging, and safe rollout patterns * Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, test creation, troubleshooting, or documentation) with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security * Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices, * Experience operating large language model inference servers such as vLLM and llm-d (or directly equivalent model serving stacks) in production * Experience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns * Experience building AI agents using orchestration frameworks such as LangChain, LangGraph, CrewAI, or similar platforms * Experience operating or integrating model serving platforms such as KServe, Ray Serve, or NVIDIA Triton Inference Server alongside other large language model serving stacks * Familiarity with Amazon SageMaker JumpStart, SageMaker Endpoints, and Amazon Bedrock for managed model hosting * Experience with online large language model quality monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventions * Contributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton) ## Description * Design, develop, troubleshoot, and deliver secure, high-quality production software and services for AI infrastructure * Build backend services and APIs that enable reliable operation of AI infrastructure in production environments * Operate and scale large language model serving infrastructure, including model hosting, request routing, continuous batching, and cache optimization * Deploy, host, and lifecycle-manage open-source and proprietary large language models on cloud-based container orchestration platforms and on-premises GPU clusters using reproducible infrastructure as code and continuous delivery pipelines * Implement observability across logs, metrics, and traces with dashboards and actionable alerting for large language model and GPU workloads * Tune GPU and accelerator capacity, autoscaling, and cost efficiency for large language model inference workloads using performance optimization techniques such as quantization, parallelism, and speculative decoding * Lead reliability engineering for large language model endpoints through capacity planning, load and soak testing, safe rollouts, failover, and incident response for outages and model-quality regressions * Participate in on-call rotations, lead incident triage and mitigation, and produce clear post-incident root-cause analyses and follow-up actions * Identify recurring operational issues and automate remediation to improve platform stability and developer experience * Build and maintain multi-agent systems with strong orchestration, including planning, coordination, tool-calling, state and memory management, and workflow control where appropriate * Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team., Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part of our company, ensuring that we're setting our businesses, clients, customers and employees up for success. You need to sign in or create an account to save a job. ## Related Videos - [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock) - [Technical Documentation - How Can I Write Them Better and Why Should I Care?](https://www.wearedevelopers.com/videos/681-technical-documentation-how-can-i-write-them-better-and-why-should-i-care) - [Understanding Kubernetes in a visual way](https://www.wearedevelopers.com/videos/100085-understanding-kubernetes-in-a-visual-way) - [DevOps for AI: running LLMs in production with Kubernetes and KubeFlow](https://www.wearedevelopers.com/videos/1222-devops-for-ai-running-llms-in-production-with-kubernetes-and-kubeflow) - [Instant KAI Sandboxes with vCluster: Multi-Tenant, Multi-Scheduler GPU Sharing](https://www.wearedevelopers.com/videos/100333-instant-kai-sandboxes-with-vcluster-multi-tenant-multi-scheduler-gpu-sharing) - [Bridging AI and Nomad: a Go-based MCP Server for Cluster Control](https://www.wearedevelopers.com/videos/2063-bridging-ai-and-nomad-a-go-based-mcp-server-for-cluster-control) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [The Best Large Language Models on The Market](https://www.wearedevelopers.com/magazine/319-the-best-large-language-models-on-the-market) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)