> Markdown version of [/jobs/ext/3000379-staff-site-reliability-engineer](https://www.wearedevelopers.com/jobs/ext/3000379-staff-site-reliability-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Staff Site Reliability Engineer - **Company:** Hippocratic AI - **Location:** Menlo Park, CA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Systems Engineering, Microsoft Azure, Computer Engineering, Continuous Integration, DevOps, Fault Tolerance, Python (Programming Language), Key Management, Reliability Engineering, Software Engineering, Datadog, Data Logging, Cloud Platform System, Delivery Pipeline, Grafana, Build Management, Containerization, Gitlab-ci, Kubernetes, Infrastructure Automation Frameworks, Information Technology, Hashicorp, Machine Learning Operations, Terraform, Docker - **Published:** September 19, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=b2f3c68dbf062ebc ## About the Role * 10+ years of professional experience across site reliability / DevOps engineering and software engineering * Computer Science Degree Required from a top CS program. * Strong software engineering fundamentals - you build orchestration and scheduling systems in Python and/or Go, not just configure off-the-shelf tools * Experience designing systems that make decisions from operational metrics - collecting signals, interpreting them, and driving control loops such as autoscaling, load shedding, or admission control * Deep experience with infrastructure automation and CI/CD (Terraform, GitLab CI/CD, or similar) * Hands-on production experience with at least one major cloud platform (AWS, GCP, or Azure) * Strong knowledge of containerization and orchestration (Docker, Kubernetes) * Experience with monitoring and logging stacks (ELK, Grafana, Datadog, or similar) * Familiarity with secrets management and security tooling (HashiCorp Vault, AWS KMS, Azure Key Vault) * Excellent problem-solving skills and the ability to work both independently and collaboratively * Strong communication and interpersonal skills Nice-to-Have * Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators * Familiarity with ML inference serving and model deployment (e.g. Triton, KServe, Ray Serve, or similar) * Experience with Kubernetes autoscaling internals (HPA/VPA, custom metrics, custom controllers) * Experience implementing HIPAA and SOC 2 compliance * Experience operating in an HPC environment * Bachelor's or Master's in Computer Science, Computer Engineering, or a related field ## Description We're looking for a Senior Site Reliability Engineer who is equally at home writing production software and running the infrastructure it lives on - and who wants to take ownership of one of the hardest, highest-leverage problems on our platform: intelligently managing a large fleet of GPU-backed models. We run nearly 30 models across heterogeneous hardware, and keeping that fleet fast, reliable, and cost-effective is a serious engineering challenge. You'll build the GPU management and scheduling platform that sits at the center of it - collecting utilization and load metrics, interpreting what they actually mean, and using them to make real-time decisions about admission control and scaling. The goal: route and schedule inference calls so we use our capacity efficiently without exceeding it, and scale model replicas up and down automatically as demand shifts. This is a senior role for someone with a decade in the field who can move fluidly between systems engineering and software development, and who is excited to own a complex, evolving system end to end. What You'll Do * Design and build our GPU management and scheduling platform - the system that decides when, where, and how inference calls run across a fleet of ~30 models on heterogeneous hardware * Build the metrics pipeline that collects GPU load and utilization data, and the logic that turns those signals into decisions * Implement admission control to protect capacity - deciding when to accept, queue, or shed inference requests so we operate within fleet limits * Build autoscaling that adjusts the number of model replicas in response to real-time demand and utilization * Develop cloud orchestration systems and operators in Python and Go to manage the model fleet * Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure * Design and build infrastructure automation and deployment pipelines (Terraform, CI/CD) as first-class software * Stand up and maintain monitoring, logging, and alerting that keep the platform reliable and performant * Develop and enforce security and compliance policies appropriate to a healthcare AI platform * Partner with engineers and research scientists to diagnose and resolve complex infrastructure, deployment, and operational issues * Mentor engineers and raise the technical bar across the team ## Related Videos - [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops) - [Debugging in the Dark](https://www.wearedevelopers.com/videos/1658-debugging-in-the-dark) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Designing UX for SRE Agents in High-Stakes Incidents](https://www.wearedevelopers.com/videos/100003-designing-ux-for-sre-agents-in-high-stakes-incidents) - [Software Engineering Social Connection: Yubo’s lean approach to scaling an 80M-user infrastructure](https://www.wearedevelopers.com/videos/1583-software-engineering-social-connection-yubo-s-lean-approach-to-scaling-an-80m-user-infrastructure) ## Related Articles - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Trustworthy AI Starts at Deployment: 5 Checks Before You Ship](https://www.wearedevelopers.com/magazine/753-trustworthy-ai-starts-at-deployment-5-checks-before-you-ship) - [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated) - [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this)