> Markdown version of [/jobs/ext/2953441-systems-engineer-aws-incident-response](https://www.wearedevelopers.com/jobs/ext/2953441-systems-engineer-aws-incident-response). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Systems Engineer, AWS Incident Response - **Company:** Amazon.com, Inc. - **Location:** Seattle, WA, United States - **Experience:** Experienced - **Salary:** $104,500.0 - $160,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Systems Engineering, Bash Shell, Cloud Computing, Linux, Distributed Systems, Perl (Programming Language), Python (Programming Language), Networking Basics, Operational Data Store, Ansible, Ruby, Runbook, Datadog, Scripting, System Availability, Grafana, Cloudformation, Cloudwatch, Terraform, Programming Languages - **Published:** September 17, 2026 - **Apply:** https://dejobs.org/x/x/54D5AB3F1BCF4905936B94ED66B6CC99/job/ ## About the Role * 3+ years of systems engineering, or 3+ years of technical support experience * Experience in written and verbal communication skills to communicate with technical and non-technical audiences, including senior leadership * Experience scripting in one or more language (e.g. Bash, Python, Perl, Ruby), or experience scripting in modern programming languages * Understanding of operating systems (Linux), networking fundamentals, and distributed systems * Experience with operational monitoring, alerting, and metrics (CloudWatch, Datadog, Grafana, or equivalent) * Demonstrated ability to troubleshoot complex technical problems spanning multiple systems or services, * Familiarity with incident management tooling and workflows in a large-scale production environment * Experience with AWS services and cloud infrastructure * Experience using generative AI or automation to solve operational problems or accelerate workflows * Track record of authoring post-incident analyses (post-mortems) and driving corrective actions to completion * Experience building operational dashboards, runbooks, or automation that improved team efficiency * Knowledge of infrastructure-as-code and deployment tooling, such as CDK, CloudFormation, Terraform, Ansible, or similar * Experience coordinating across globally distributed teams and time zones * Comfort operating with ambiguity and incomplete information, and a self-starting approach to identifying what needs doing ## Description As a Systems Engineer on AWS Incident Response, you will be on the front line of AWS incident response. You will lead high-severity calls, triage complex failures across distributed systems, coordinate resolver teams, and drive incidents to mitigation in real time while millions of customers depend on the outcome. Between incidents you will obsess over metrics and detection analysis, building dashboards and mechanisms that surface problems before customers notice, and you will drive operational improvements that make the incident management ecosystem faster and more accurate. You will make real-time decisions under pressure, deep-diving the largest and most complex technical environment in the world. You will develop expertise across AWS services, networking, and infrastructure, building a breadth of knowledge that few roles offer. Your scope spans all of AWS rather than a single service. You will own operational processes end to end and use data to find the next improvement in how we detect and mitigate faster. You will also have the opportunity to grow your development skills by taking on coding projects that accelerate incident response and reduce toil. This role includes participation in an on-call rotation covering weekdays, weekends, and holidays. On-call shifts fall within your local daytime hours., * Lead high-severity incident response calls end to end: assess impact, coordinate resolvers across AWS service teams, communicate clearly under pressure, manage escalations, and drive the incident to mitigation with documentation throughout. * Own and run operational health reviews, and build and maintain the dashboards, metrics, and monitoring that surface trends before they become incidents. * Improve detection accuracy and speed. Identify patterns across events and build proactive mechanisms that prevent recurrence. * Deep-dive operational data to find systemic issues, measure response effectiveness, and prioritize improvements against what the data shows is degrading. * Identify gaps in operational processes, documentation, and tooling, and build or improve mechanisms that reduce time-to-detection and time-to-mitigation. * Apply scripting, automation, and generative AI to accelerate incident response and reduce toil, including where AI can augment human judgment during an incident or surface insight from operational data at scale. * Work with service teams so that learnings from each incident drive corrective actions to completion, closing the loop between what broke and what gets fixed. * Mentor peers in your areas of technical and operational strength. A day in the life When you are on call, incidents take priority. You join the incident bridge, assess the scope of impact from real-time metrics and dashboards, engage the resolver teams that own the affected services, and drive the event to mitigation, escalating when progress stalls. Off-call, you work to reduce time-to-detection and time-to-mitigation: deep-diving recent events for detection that lagged the impact or alarms that trigger without customer impact, correcting the signal behind them, leading operational health reviews, driving other teams' corrective actions to completion, and owning projects that strengthen AIR's detection and incident management, including automation you can build yourself. About the team AWS Incident Response (AIR), part of the AWS Resilience organization, ensures high availability of AWS. When major incidents hit, AIR leads the response, coordinating resolver teams across AWS and driving mitigation. We move fast, but not carelessly, obsessing over observability of the cloud and continuously improving our detection and response speed and accuracy. Each incident feeds improvements to our detection, processes, and tooling, making the next one shorter or preventing it entirely. This is a high-visibility, high-impact role with a global view of AWS health that few teams get to see. ## Related Videos - [Applying Agile Principles to Incident Management ](https://www.wearedevelopers.com/videos/101-applying-agile-principles-to-incident-management) - [Coffee with Developers: David Heinemeier Hansson](https://www.wearedevelopers.com/videos/875-coffee-with-developers-david-heinemeier-hansson) - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [5 steps for running a Kubernetes environment at scale](https://www.wearedevelopers.com/videos/88-5-steps-for-running-a-kubernetes-environment-at-scale) - [Handling incidents collaboratively is like solving a rubix cube](https://www.wearedevelopers.com/videos/680-handling-incidents-collaboratively-is-like-solving-a-rubix-cube) - [Coroutine explained yet again 60 years later](https://www.wearedevelopers.com/videos/690-coroutine-explained-yet-again-60-years-later) ## Related Articles - [The Geometry of Incidents: Connecting User Impact to Architecture](https://www.wearedevelopers.com/magazine/764-the-geometry-of-incidents-connecting-user-impact-to-architecture) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Best Paying Jobs in Technology](https://www.wearedevelopers.com/magazine/256-best-paying-jobs-in-technology) - [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)