> Markdown version of [/jobs/ext/2220093-sr-principal-software-engineer-llm-engineering](https://www.wearedevelopers.com/jobs/ext/2220093-sr-principal-software-engineer-llm-engineering). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # SR Principal Software Engineer - LLM Engineering - **Company:** JPMorgan Chase & Co. - **Location:** Palo Alto, CA, United States - **Experience:** Expert - **Salary:** $232,750.0 - $325,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Cloud Engineering, Software Quality, Continuous Integration, Distributed Systems, Fraud Prevention and Detection, High-Level Architecture, Java Architectures, Python (Programming Language), Machine Learning, Open Source Technology, Systems Development Life Cycle, Reliability Engineering, Tensorflow, Azure Machine Learning, Software Engineering, Toolchain, Management of Software Versions, Graphics Processing Unit (GPU), Pytorch, Autoscaling, Large Language Models, Deep Learning, Caching, Cloudformation, Containerization, AI Platforms, Kubernetes, Information Technology, Low Latency, Optimization Algorithms, HuggingFace, Hardware Acceleration, Machine Learning Operations, Terraform, GPT, Docker - **Published:** August 25, 2026 - **Apply:** https://dejobs.org/x/x/B84123DCD0B44A3BBA0D5BAB8166E923/job/ ## About the Role * Required qualifications, capabilities, and skills Formal training or certification on software engineering concepts and 10+ years of applied experience. * 8+ years of AI/ML engineering experience with significant expertise in LLMs, GNNs and other model architectures (e.g., GPT, Llama, Falcon, Mistral). * Demonstrated success architecting and deploying LLM & GNN solutions on AWS (e.g., SageMaker, Bedrock, EKS) at enterprise scale; experience with Azure ML or GCP Vertex AI. * Experience building LLM and GNN serving platforms in large-scale environments typical of major tech firms. * Hands-on experience building LLM inference engines using Triton Inference Server and vLLM, including autoscaling, caching, and throughput optimization. * Advanced proficiency in Python and optimization techniques applied to deep learning frameworks (PyTorch, TensorFlow, Hugging Face Transformers). * Deep understanding of LLMOps/MLOps (e.g., MLflow, SageMaker Pipelines, Kubeflow) with a track record of implementing best practices at scale. * Demonstrated experience designing and scaling agentic AI-enabled development patterns (using enterprise-authorized tools within the work environment) across teams/functions, including establishing governance for human-in-the-loop validation, traceability/auditability, and secure handling of sensitive inputs/outputs. * Strong understanding of responsible AI use and control expectations at scale, including security/resiliency implications, data sensitivity, and risk-based governance; ability to advise senior leaders on safe adoption, reuse, and measurable outcomes. * Expertise in inference optimization and distributed systems for large models focused on high-throughput, low-latency applications, including system design, testing, and operational stability for enterprise AI platforms. * Excellent communication skills with proven collaboration with SRE to implement observability, incident response, and SLIs/SLOs for LLM services, and the ability to influence both technical and non-technical stakeholders to deliver value across functions at scale. . Preferred qualifications, capabilities, and skills * Master's or PhD in Computer Science, Engineering, or a related field (or equivalent experience). * Practical cloud-native experience, including containerization (Docker), orchestration (Kubernetes), and infrastructure-as-code (Terraform, CloudFormation). * Expertise in security, compliance, and governance for AI/ML deployments in regulated environments. * Experience in trust and safety or fraud prevention domains; familiarity with payments platforms is a plus. * Track record of contributions to open-source LLM projects or peer-reviewed research and/or experience presenting at industry conferences or leading technical communities. * Familiarity with hardware acceleration strategies across GPUs, TPUs, and specialized inference runtimes. * Experience in building java based applications ## Description * Advises and leads on the strategy, architecture, and development of model serving solutions for different model architectures (including LLMs & GNNs) across cloud and on-premises environments, aligning initiatives to business outcomes. * Defines and implements MLOps and LLMOps strategies for end-to-end model lifecycle management, including training, versioning, deployment, monitoring, and governance. * Drives optimization of model inferencing for high throughput and low latency using quantization, model parallelism, intelligent batching, and hardware acceleration for all model architectures. * Sets strategy and operating standards for agentic AI-enabled engineering across a portfolio (using enterprise-authorized tools within the work environment) to drive measurable improvements in delivery speed, reliability, and code quality (e.g., AI-orchestrated SDLC/TLM automation, release readiness gating, incident triage/root-cause acceleration, and large-scale refactoring/test modernization), while defining guardrails for validation, security, resiliency, and reuse across teams and functions. * Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation at scale. * Creates durable, reusable software and platform frameworks to standardize ML Engineering services, enabling scale across teams and functions. * Establishes best practices for automation, CI/CD, and infrastructure-as-code using containerization and orchestration technologies. * Partners closely with data science, platform engineering, and SRE teams to productionize models on AWS, ensuring observability, reliability, and cost efficiency. * Leads deployment and optimization using model inference servers such as Triton Inference Server and vLLM for high-throughput, low-latency serving at scale. * Oversees production operations for AI workloads, including monitoring, incident response, security, and compliance, with continuous improvement. * Translates complex technical concepts and emerging trends into actionable strategies, influencing senior stakeholders and cross-functional partners to prioritize and deliver AI/ML capabilities that drive measurable business impact while promoting a culture of diversity, opportunity, inclusion, and respect. ## Related Videos - [Efficient deployment and inference of GPU-accelerated LLMs​](https://www.wearedevelopers.com/videos/929-efficient-deployment-and-inference-of-gpu-accelerated-llms) - [ Evaluating AI models for code comprehension](https://www.wearedevelopers.com/videos/1462-evaluating-ai-models-for-code-comprehension) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [HTTP headers that make your website go faster](https://www.wearedevelopers.com/videos/1676-http-headers-that-make-your-website-go-faster) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge](https://www.wearedevelopers.com/videos/1328-how-to-avoid-llm-pitfalls-mete-atamel-and-guillaume-laforge) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)