> Markdown version of [/jobs/ext/4639519-ai-infrastructure-engineer-senior-staff](https://www.wearedevelopers.com/jobs/ext/4639519-ai-infrastructure-engineer-senior-staff). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# AI Infrastructure Engineer (Senior/Staff)

- **Company:** Recruiting from Scratch
- **Location:** San Francisco, CA, United States
- **Experience:** Expert
- **Salary:** $130,000 - $300,000
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Data Analysis, Apple IOS, C++ (Programming Language), Computer Clusters, Information Systems, Distributed Computing Environment, Distributed Systems, Python (Programming Language), Prometheus, AI Infrastructure, Graphics Processing Unit (GPU), Pytorch, Large Language Models, Grafana, Kubernetes, Information Technology, Slurm, Megatron
- **Published:** November 8, 2026
- **Apply:** https://www.careerbuilder.com/job-details/ai-infrastructure-engineer-senior-staff-san-francisco-ca--4bfcf3f8-17cd-4d97-9d55-e2ef2f0b8c2f

## About the Role

Artificial Intelligence (AI), iOS, Artificial Intelligence (AI), Fortune 500 Customers, Operational Audit, Information Technology & Information Systems, Technical/Engineering Design, On Call, GPU (Graphics Processing Unit), Data Analysis

## Description

* GPU clusters across AWS, GCP, neoclouds and on-prem: custom Kubernetes and Slurm operators and the metadata services behind them.
 * Scheduling that keeps accelerators busy: gang scheduling, bin-packing, quotas and fair share for multi-tenant teams.
 * Distributed training infra (PyTorch Distributed, Ray, Megatron) with fast checkpointing; high-throughput LLM serving on vLLM (TP/EP, prefill/decode disaggregation, KV-cache offload, FP8/FP4).
 * Fleet health: hardware/network/software checks with automated remediation; DCGM, Prometheus, Grafana; utilization and billing ingestion for cost attribution and capacity planning.

Must-haves

 * 5+ years building and operating production distributed systems or ML infra (7+ for Staff), counted from full-time industry roles only.
 * Has already run GPU workloads at real scale in production. Resume names the systems and the numbers: cluster or node counts, GPU generations, QPS, GPU-hours recovered. Zero handholding; this is not a learn-on-the-job role.
 * Owned at least one of: a GPU cluster or scheduler (Kubernetes + Slurm), distributed training infrastructure, or an LLM serving stack. Hands-on, not "partnered with the scheduling team."
 * Production Python plus Go or C++. Deep Kubernetes. Working Slurm or other HPC resource manager. One major cloud in depth. Comfortable on-call.
 * Individual contributor who still writes code. Directors, architects and managers who stopped shipping code several years ago are not a fit (we have a separate Head of Infrastructure search).

Process

Recruiter screen initial technical screen with the VP of Engineering onsite: infra coding (Python), algorithms, AI-specific system design, and a 45-minute GPU-specific interview.