Software Engineer - Fleet Management

The platform will need to support thousands of concurrent workflows, with strong guarantees around reliability, auditability, idempotency, resumability and failure recovery. You will work closely with infrastructure, hardware, networking and operations teams to build highly scalable systems powering next-generation AI infrastructure. xrbllit

The ideal candidate will have extensive experience building distributed systems, infrastructure automation and production platforms, with strong Python skills and experience managing large-scale compute, datacenter or hardware infrastructure.

The key skills required for the Staff Software Engineer:

• Extensive experience building and operating distributed systems in production
• Strong Python software engineering skills
• Experience with workflow orchestration, state machines or event-driven architectures
• Experience building reliable automation for infrastructure provisioning and lifecycle management
• Strong understanding of idempotency, retries, checkpoints, recovery and failure handling
• Experience with bare-metal infrastructure, server provisioning or hardware lifecycle management
• Experience with BMC, IPMI, Redfish, PXE, MAAS, Ironic or NetBox is highly desirable
• Experience with GPU infrastructure, AI clusters, HPC or datacenter environments is advantageous
• Strong Kubernetes, Terraform, Ansible or cloud infrastructure experience
• Experience with networking and network automation, particularly at datacenter scale
• Experience building health monitoring, validation, observability and automated remediation systems
• Experience integrating infrastructure platforms, APIs, monitoring systems, credential stores and DCIM tools
• Strong understanding of production reliability, scalability and security
• Staff/Principal-level technical leadership with experience influencing architecture across teams
• Experience using modern AI-assisted development tools such as Claude, Cursor or similar is desirable

The role will involve building and owning a Fleet Management platform responsible for provisioning, testing, monitoring and remediating GPU servers and network infrastructure at scale.

You will design Python-based workflows covering the complete hardware lifecycle, from device enrolment and BMC configuration through provisioning, multi-day burn-in, health monitoring and automated remediation.