Senior DevOps Engineer, Cloud Simulation...
Role details
Job location
Tech stack
Job description
We are seeking a Senior DevOps / Cloud Simulation Infrastructure Engineer to own the complete end-to-end cloud execution pipeline for SimReady assets! This role is critical to our product strategy, enabling us to transition from local, workstation-driven validation to high-scale, automated cloud validation on NVIDIA Cloud Functions (NVCF). You will be responsible for deploying a robust, multi-GPU pipeline that supports structural validation, AI-driven runtime behavioral testing, and automated asset remediation.
What you'll be doing:
-
Deployment: Deploy full Isaac Sim runtimes within GPU-aware NVCF containers. Manage container packaging, GPU initialization, and runtime utilities for physics, sensor, and rendering validation.
-
Deploy Structural Validation: Deploy services to validate USD structure and compliance without runtime overhead.
-
Deploy Runtime Validation: Architect scalable execution layers to conduct runtime behavior-based testing (e.g., drop/grasp tests). Deploy rule-based systems or AI based systems for automated pass/fail grading.
-
Deploy Automated Remediation: Develop an AI-based pipeline that intercepts failures, triggers automated asset fixes, and re-validates results to ensure quality standards.
-
Cloud Infrastructure Ownership: Scale execution from single-workstation validation to massive, multi-GPU cloud environments. Optimize for performance, addressing function-to-function networking, gRPC bottlenecks, and in-cluster proxy behavior.
-
Artifact & Evidence Pipeline: Automate the generation of verification videos, thumbnails, feature-level reports, and validation metadata. Ensure all assets are traceable and linked to quality gates.
-
Observability & CI/CD: Establish robust CI/CD, cluster verification, and monitoring pipelines. Implement logging, metrics, and tracing to ensure services are observable, debuggable, and production-ready.
-
Operational Reliability: Implement atomic update semantics and safe failure handling to ensure validation processes never corrupt the primary asset library.
Requirements
-
BS or MS degree in Computer Science, Computer Engineering, or related field (or equivalent experience).
-
8+ years of professional experience working on DevOps and/or cloud simulation.
-
Extensive experience in production-grade DevOps, SRE, or Infrastructure Engineering, with a focus on GPU-backed cloud services.
-
Proven expertise in container orchestration (Kubernetes/Docker) and CI/CD pipeline development.
-
Experience with automated testing frameworks, preferably involving AI/ML inference, computer vision, or rule-based validation.
-
Proficiency in Python and systems scripting for test orchestration and pipeline automation.
-
Strong ability to design and maintain distributed job lifecycle services (submit/poll/fetch/cancel) and handle asynchronous failure states.
-
Ability to diagnose and solve distributed network bottlenecks, including gRPC and function-to-function communication.
Ways to stand out from the crowd:
-
Direct experience deploying services on NVCF (NVIDIA Cloud Functions) or DGX Cloud.
-
Deep familiarity with Isaac Sim, Omniverse, USD, or Sensor RTX workflows.
-
Background in robotics simulation, physical AI, or large-scale content creation pipelines.
-
Experience building "self-healing" or automated remediation workflows.
-
Experience with cluster verification frameworks, stress testing, and deployment validation at scale.
We value different paths to technical excellence and welcome candidates who bring strong judgment, curiosity, and a collaborative approach. Come build the future of autonomous vehicle simulation with us!
Benefits & conditions
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD.