> Markdown version of [/jobs/ext/2924159-principal-software-engineer-llm-engineering](https://www.wearedevelopers.com/jobs/ext/2924159-principal-software-engineer-llm-engineering). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Principal Software Engineer - LLM Engineering - **Company:** J.p. Morgan's Commercial & Investment Bank - **Location:** Jersey City, NJ, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Continuous Integration, Distributed Systems, Fraud Prevention and Detection, High-Level Architecture, Java Architectures, Python (Programming Language), Machine Learning, Reliability Engineering, Tensorflow, Azure Machine Learning, Software Engineering, Management of Software Versions, Graphics Processing Unit (GPU), Pytorch, Autoscaling, Large Language Models, Deep Learning, Caching, Cloudformation, Containerization, AI Platforms, Kubernetes, Information Technology, Low Latency, Optimization Algorithms, HuggingFace, Hardware Acceleration, Machine Learning Operations, Terraform, GPT, Docker - **Published:** September 15, 2026 - **Apply:** https://find.jobs/jobs-near-me/apply/ats-redirect/?id=2972678542-2 ## About the Role * Formal training or certification on software engineering concepts and 10+ years of applied experience. * 8+ years of AI/ML engineering experience with significant expertise in LLMs, GNNs and other model architectures (e.g., GPT, Llama, Falcon, Mistral). * Demonstrated success architecting and deploying LLM & GNN solutions on AWS (e.g., SageMaker, Bedrock, EKS) at enterprise scale; experience with Azure ML or GCP Vertex AI. * Experience building LLM, GNN serving platforms in largeâscale environments typical of major tech firms. * Handsâon experience building LLM inference engines using Triton Inference Server and vLLM, including autoscaling, caching, and throughput optimization. * Advanced proficiency in Python and optimization techniques applied to deep learning frameworks (PyTorch, TensorFlow, Hugging Face Transformers). * Deep understanding of LLMOps/MLOps (e.g., MLflow, SageMaker Pipelines, Kubeflow) with a track record of implementing best practices at scale. * Expertise in inference optimization and distributed systems for large models focused on highâthroughput, lowâlatency applications. * Practical experience delivering system design, application development, testing, and operational stability for enterprise AI platforms. * Proven collaboration with SRE to implement observability, incident response, and SLIs/SLOs for LLM services. * Excellent communication skills with the ability to influence both technical and nonâtechnical stakeholders and deliver value across functions at scale. Preferred qualifications, capabilities, and skills * Master's or PhD in Computer Science, Engineering, or a related field (or equivalent experience). * Practical cloudânative experience, including containerization (Docker), orchestration (Kubernetes), and infrastructureâasâcode (Terraform, CloudFormation). * Expertise in security, compliance, and governance for AI/ML deployments in regulated environments. * Experience in trust and safety or fraud prevention domains; familiarity with payments platforms is a plus. * Track record of contributions to openâsource LLM projects or peerâreviewed research and/or experience presenting at industry conferences or leading technical communities. * Familiarity with hardware acceleration strategies across GPUs, TPUs, and specialized inference runtimes. * Experience in building java based applications ## Description We're looking for a tech leader ready to take their career to new heights. Join the ranks of top talent at one of the world's most influential companies. As a Senior Principal Software Engineer at JPMorganChase within the Commercial & Investment Bank Trust & Safety Fraud Prevention team, you provide deep engineering expertise and work across agile teams to enhance, build, and deliver trusted marketâleading technology products in a secure, stable, and scalable way. Leverage your deep expertise to consistently challenge the status quo, innovate for business impact, lead the strategic development behind new and existing products and technology portfolios, and remain at the forefront of industry trends, best practices, and technological advances., * Advises and leads on the strategy, architecture, and development of Model serving solutions for different model architectures including LLMs & GNNs, across cloud and onâpremises environments, aligning initiatives to business outcomes. * Defines and implements MLOps and LLMOps strategies for endâtoâend model lifecycle management, including training, versioning, deployment, monitoring, and governance. * Drives optimization of Model inferencing for high throughput and low latency using quantization, model parallelism, intelligent batching, and hardware acceleration for all model architectures * Creates durable, reusable software and platform frameworks to standardize ML Engineering services, enabling scale across teams and functions. * Establishes best practices for automation, CI/CD, and infrastructureâasâcode using containerization and orchestration technologies. * Partners closely with data science, platform engineering, and SRE teams to productionize the models on AWS, ensuring observability, reliability, and cost efficiency. * Leads deployment and optimization using Model Inference servers such as Triton Inference Server and vLLM for highâthroughput, lowâlatency serving at scale. * Oversees production operations for AI workloads, including monitoring, incident response, security, and compliance, with continuous improvement. * Translates highly complex technical concepts and emerging trends into actionable strategies for executive and product leadership. * Influences senior stakeholders and crossâfunctional partners to prioritize and deliver AI/ML capabilities that drive measurable business impact. * Promotes the firm's culture of diversity, opportunity, inclusion, and respect across teams and communities., This position is subject to Section 19 of the Federal Deposit Insurance Act. As such, an employment offer for this position is contingent on JPMorgan Chase's review of criminal conviction history, including pretrial diversions or program entries. ## Related Videos - [ Evaluating AI models for code comprehension](https://www.wearedevelopers.com/videos/1462-evaluating-ai-models-for-code-comprehension) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [HTTP headers that make your website go faster](https://www.wearedevelopers.com/videos/1676-http-headers-that-make-your-website-go-faster) - [Efficient deployment and inference of GPU-accelerated LLMs​](https://www.wearedevelopers.com/videos/929-efficient-deployment-and-inference-of-gpu-accelerated-llms) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [Speak, Code, Deploy: Transforming Developer Experience with Voice Commands](https://www.wearedevelopers.com/videos/1159-speak-code-deploy-transforming-developer-experience-with-voice-commands) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)