> Markdown version of [/jobs/ext/2283717-principal-software-engineer-rack-scale-system-software-csp-engagements](https://www.wearedevelopers.com/jobs/ext/2283717-principal-software-engineer-rack-scale-system-software-csp-engagements). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Principal Software Engineer, Rack-Scale System Software - CSP Engagements

- **Company:** NVIDIA Corporation
- **Location:** Santa Clara, CA, United States
- **Salary:** $272,000.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), System Configuration, Software Design Patterns, Distributed Systems, Fault Tolerance, Firmware, System Software, Strategies of Testing, Kubernetes, Information Technology, Api Design, ArcSight Event Correlation
- **Published:** August 28, 2026
- **Apply:** https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Principal-Software-Engineer--Rack-Scale-System-Software---CSP-Engagements_JR2020316

## About the Role

* 15+ years of experience in system software, platform firmware, or large-scale distributed systems engineering. BS or MS in Computer Science, Electrical Engineering, or related field (or equivalent experience)
 * Deep understanding of rack-scale system software challenges: multi-component coordination, error propagation, health monitoring, and serviceability / reliability
 * Experience with fabric management software, cluster management, or system-level orchestration frameworks. Familiarity with firmware architectures and update lifecycle management (multi-component update sequencing, rollback, recovery)
 * Understanding of error handling and recovery design patterns in distributed systems - fault isolation, retry policies, graceful degradation
 * Experience with health monitoring and telemetry systems: health scoring, event correlation, API design for fleet-level observability
 * Understanding of GPU or accelerator system software (drivers, device management, power management) is a strong plus
 * Customer obsession - genuine passion for understanding how CSPs operate sophisticated systems at fleet scale and simplifying their experience
 * Proven success providing technical leadership across organizational boundaries and influencing system software design without direct authority. Strong communication - ability to translate complex system software architecture into actionable mentorship for customer engineering teams

Ways to stand out from the crowd:

 * Experience with NVIDIA NVSwitch, NVOS, or GPU fabric management software
 * Background in system software for large-scale clusters at a hyperscaler (cluster management, fleet orchestration, health platforms)
 * Experience crafting error handling and recovery frameworks for multi-component systems (hundreds or thousands of coordinating devices)
 * Familiarity with GPU or accelerator fleet operations - driver lifecycle, firmware rollout strategies, health-based scheduling
 * Understanding of how system software decisions impact serviceability, availability, and operational cost at fleet scale

## Description

We're looking for a Principal Software Engineer to join our CSP Engagements team as the technical focal point for rack-scale system SW/FW, working with CSP engineering teams to ensure they can deploy, monitor, and operate these systems reliably at fleet scale. In this role, you will collaborate with NVIDIA's cross-functional rack-scale system SW/FW engineering teams with dedicated CSP-facing technical leadership. Your focus is on the system-level software that manages, monitors, and recovers the rack as a whole - fabric management, GPU/NVSwitch error handling and recovery, health telemetry APIs, firmware update orchestration, and SW-driven serviceability. You will drive work streams with CSP engineering teams to build shared understanding of the architecture, incorporate their operational feedback, and ensure integration readiness.

What you'll be doing:

 * Drive rack-scale SW/FW architecture alignment across CSP engagements - including fabric management software, link health monitoring, GPU/NVSwitch error handling, SW/FW serviceability features (e.g., hot-plug support, component isolation, firmware-driven recovery), and multi-component firmware orchestration
 * Drive technical work streams with CSP engineering teams on rack-scale system software - ensuring they deeply understand fabric management, NVSwitch behavior, error handling and recovery policies, health telemetry APIs, and SW/FW-controlled recovery operation
 * Capture and synthesize CSP engineering feedback on rack-scale system software - health monitoring APIs, SW-driven serviceability workflows, firmware update orchestration, and error recovery behavior - champion that feedback into NVIDIA's architecture decisions
 * Collaborate with multi-functional teams to ensure customer operational requirements are reflected in system software and firmware development
 * Identify cross-CSP patterns in rack-scale SW/FW issues, error handling behavior, and system configuration practices - drive documentation, tooling, and test strategy improvements as a result
 * Collaborate with execution teams on left-shift strategy - ensuring customer-side SW/FW integration work is identified early and completed ahead of hardware availability
 * Make critical technical decisions on rack-scale system SW/FW tradeoffs and mitigate execution risks through early engagement with CSP engineering teams


## Related Videos

- [Coffee with Developers - Stephen Jones - NVIDIA](https://www.wearedevelopers.com/videos/1303-coffee-with-developers-stephen-jones-nvidia)
- [API Design - Getting Started](https://www.wearedevelopers.com/videos/33-api-design-getting-started)
- [Understanding Kubernetes in a visual way](https://www.wearedevelopers.com/videos/100085-understanding-kubernetes-in-a-visual-way)
- [Playing Pong on a shoulder press machine](https://www.wearedevelopers.com/videos/100140-playing-pong-on-a-shoulder-press-machine)
- [Accelerating Python on GPUs](https://www.wearedevelopers.com/videos/859-accelerating-python-on-gpus)
- [Rest API Antipatterns](https://www.wearedevelopers.com/videos/100208-rest-api-antipatterns)

## Related Articles

- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How software is steering vehicle technology](https://www.wearedevelopers.com/magazine/515-how-software-is-steering-vehicle-technology)
- [Best US AI Conferences for CTOs in 2026: Build vs. Buy, Vendor Evaluation, and Peer Intelligence](https://www.wearedevelopers.com/magazine/736-best-us-ai-conferences-for-ctos-in-2026-build-vs-buy-vendor-evaluation-and-peer-intelligence)
- [How to Answer the Interview Question: “Why Do You Want to Be a Software Engineer?”](https://www.wearedevelopers.com/magazine/392-how-to-answer-the-interview-question-why-do-you-want-to-be-a-software-engineer)
- [Why Attend a Developer Event in 2026?](https://www.wearedevelopers.com/magazine/688-why-attend-a-developer-event-in-2026)