Software Engineer III - Data Analytics Platform

• Build core backend services for LLM inference, including request routing, batching, scheduling, streaming responses, and quota and limits.
• Implement and maintain APIs and SDKs used by product and application teams across the firm.
• Profile and optimize performance end-to-end across CPU, memory, network, serialization, concurrency, GPU utilization, and caching.
• Improve reliability and operability through health checks, graceful degradation, autoscaling behaviors, incident follow-ups, and runbooks.
• Contribute to system design by breaking down ambiguous problems, proposing approaches, and making pragmatic tradeoffs.
• Add observability with metrics, tracing, logging, dashboards, and actionable alerts tied to SLOs.
• Support safe deployments through CI/CD improvements, canarying, feature flags, backward compatibility, and rollback plans.
• Learn LLM serving fundamentals, including tokenization costs, KV cache, quantization, context length tradeoffs, and throughput versus latency.
• Leverage enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity across complex deliverables, while validating outputs through peer review, automated testing, and secure coding standards.
• Contribute learnings and reusable patterns to improve broader team effectiveness.
• Apply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
• Contribute to a team culture of diversity, opportunity, inclusion, and respect.

Technologies:

• AI
• Backend
• CI/CD
• Docker
• Support
• Kubernetes
• LLM
• Network
• Python
• Redis
• Security
• TypeScript
• HTTP
• gRPC

• Formal training or certification on software engineering concepts and applied experience.
• Bachelors Degree in Computer Science or equivalent.
• Solid programming fundamentals, including data structures, concurrency basics, debugging, and testing.
• Comfort working in one or more of Go, Python, or TypeScript, with the ability to ramp up quickly on the others.
• Interest in distributed systems and system design, even if we have not built large systems yet.
• Curiosity about LLMs and AI model architecture, with willingness to learn quickly.
• A measurement-driven mindset, with interest in profiling, benchmarking, and proving improvements with data.
• Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment, with the ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security.
• Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs and outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices.
• Experience with performance profiling tools such as pprof, flamegraphs, or distributed tracing systems.
• Familiarity with containers and orchestration, including Docker, Kubernetes, and service-to-service networking.
• Understanding of inference concepts such as batching, streaming tokens, GPU memory constraints, and KV cache.
• Experience with high-throughput APIs, eventing or queues, or caching layers such as Redis.
• Exposure to reliability practices such as SLOs, SLIs, on-call rotations, and incident reviews.

hackajob is partnering directly with JPMorganChase to hire for this role. We are offering an opportunity to impact your career and push the limits of whats possible as part of our Firmwide LLM Serving Platform team. We are a global leader in financial services, and our Commercial & Investment Bank is a global leader across banking, markets, securities services, and payments, serving corporations, governments, and institutions in more than 100 countries. We value our people as our strength, take a first-class business in a first-class way approach to serving clients, and are committed to diversity, inclusion, and equal opportunity. We also provide reasonable accommodations for applicants and employees with religious practices and beliefs, as well as mental health or physical disability needs.