> Markdown version of [/jobs/ext/4702399-senior-software-engineer-resilience-engineering-dgx-cloud](https://www.wearedevelopers.com/jobs/ext/4702399-senior-software-engineer-resilience-engineering-dgx-cloud). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Software Engineer, Resilience Engineering - DGX Cloud

- **Company:** NVIDIA Corporation
- **Location:** Santa Clara, CA, United States
- **Experience:** Expert
- **Salary:** $184,000 - $287,500
- **Contract:** Permanent contract
- **Skills:** Cloud Computing, Python (Programming Language), Reliability Engineering, Prometheus, Software Engineering, Grafana, Production Code, Pagerduty
- **Published:** November 13, 2026
- **Apply:** https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Senior-Software-Engineer--Resilience-Engineering---DGX-Cloud_JR2029678

## About the Role

* Deep, hands-on experience running large-scale production systems with a proven track record.
 * A detailed understanding of failure modes in large systems, including cascading dependencies and retry storms.
 * Strong software engineering skills with current, hands-on experience in Go, Python, or similar languages.
 * Proven experience in establishing and maintaining an SLO program with operational rigor.
 * Practical experience in reliability fields such as chaos engineering and failure injection.
 * The ability to influence across team boundaries through credibility and expertise.
 * 10+ years of industry experience with a Bachelor's or Master's degree, or equivalent experience operating systems at scale.

Ways to stand out from the crowd:

 * Experience within a world-class reliability function like Google SRE or Meta production engineering.
 * Expertise in operating GPU, HPC, or AI training infrastructure with outstanding failure modes.
 * A track record of measurable reliability improvements within an organization.
 * Proficiency with modern observability and operational tools like Prometheus, OpenTelemetry, Grafana, PagerDuty, and Rootly.

## Description

* Build org-wide reliability strategy, guiding how NVIDIA matures its operational practices in a 24/7 environment.
 * Stand up a rigorous SLO program, defining and maintaining high standards across teams.
 * Lead incident response for high severity incidents, ensuring low drama and high signal resolution.
 * Build and improve production code daily, enhancing our data platform and related tooling.
 * Implement chaos engineering, failure injection, and resilience testing to elevate our team's standard practices.
 * Improve standards by setting an example with your hands-on experience and leadership.