> Markdown version of [/jobs/ext/3037165-sr-software-engineer-ai-inference-serving](https://www.wearedevelopers.com/jobs/ext/3037165-sr-software-engineer-ai-inference-serving). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Sr. Software Engineer - AI Inference & Serving

- **Company:** Amazon.com, Inc.
- **Location:** Redmond, WA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** C (Programming Language), Java (Programming Language), Artificial Intelligence, User Authentication, C Sharp (Programming Language), C++ (Programming Language), Distributed Systems, Hypertext Transfer Protocols (HTTP), Network Architecture, Online Service Provider, Software Engineering, Transmission Control Protocol (TCP), System Availability, Large Language Models, Generative AI, Kubernetes, Information Technology, Low Latency, Docker, Golang, Programming Languages
- **Published:** September 23, 2026
- **Apply:** https://www.careerjet.com/job/us3179e0379dd485e66675a1ac4931ebdb/eaa

## About the Role

Design and implement core inference infrastructure for serving frontier AI models in production. Identify and drive improvements to end-to-end inference performance and efficiency of state-of-the-art LLMs and GenAI models from OpenAI, Anthropic and xAI hosted on AI Foundary. Design and implement efficient load scheduling and balancing strategies, by leveraging key insights and features of the model and workload. Scale the platform to support the growing inferencing demand and maintain high availability. Deliver critical capabilities required to serve the latest and greatest Gen AI models such as GPT5, Realtime audio, Sora, and enable fast time to market for them. Collaborate with our partners both internal and external. Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, or Java OR equivalent experience. Technical background and solid foundation in software
engineering principles, distributed computing and architecture. Experience working on high scale, reliable online systems and experience in real-time online services with low latency and high throughput. Experience working with L7 network proxies and gateways. Knowledge in Network architecture and concepts (HTTP and TCP Protocols, Authentication and Sessions etc). Knowledge and experience in OSS, Docker, Kubernetes, C++, Golang, or equivalent programming languages. Cross-team collaboration skills and the desire to collaborate in a team of researchers and developers



## Related Videos

- [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops)
- [Understanding Kubernetes in a visual way](https://www.wearedevelopers.com/videos/100085-understanding-kubernetes-in-a-visual-way)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [Go with the Flow: Stop the Leaks Before Your Memory's a Waterfall!](https://www.wearedevelopers.com/videos/100073-go-with-the-flow-stop-the-leaks-before-your-memory-s-a-waterfall)
- [Microservices: how to get started with Spring Boot and Kubernetes](https://www.wearedevelopers.com/videos/242-microservices-how-to-get-started-with-spring-boot-and-kubernetes)
- [You are not an AI developer](https://www.wearedevelopers.com/videos/1148-you-are-not-an-ai-developer)

## Related Articles

- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)