> Markdown version of [/jobs/ext/4693915-site-reliability-engineer-production](https://www.wearedevelopers.com/jobs/ext/4693915-site-reliability-engineer-production). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Site Reliability Engineer, Production

- **Company:** Thinking Machines Lab Inc.
- **Location:** San Francisco, CA, United States
- **Salary:** $300,000 - $350,000
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Cloud Computing, Continuous Delivery, Continuous Integration, Software Debugging, Distributed Computing Environment, Distributed Systems, Reliability Engineering, Cloud Services, Kubernetes, Information Technology
- **Published:** November 12, 2026
- **Apply:** https://www.careerbuilder.com/job-details/site-reliability-engineer-production-san-francisco-ca--5c2e503e-3104-4584-9f77-936edad9aefc

## About the Role

Reliability Engineering, Artificial Intelligence (AI), Reliability Engineering, Use Cases, Algorithms, Laboratory, Application Programming Interface (API), Research & Development (R&D), Incident Response, Continuous Deployment/Delivery, * Bachelor's degree or equivalent experience in computer science, engineering, or similar.

 * Experience in distributed systems, cloud infrastructure, or site reliability engineering.
 * Proficiency writing software to solve reliability problems, including building tooling and automation.
 * Experience with production incident response, postmortems, and systematic reliability improvement.
 * Strong communication skills and track record of coordination across engineering and research teams.

Preferred qualifications

 * Deep experience operating production cloud services at scale (e.g., public cloud platforms, internal cloud services)

 * Background in distributed training frameworks and how infrastructure failures surface in training behavior.
 * Track record building checkpoint and recovery systems for long-running distributed jobs.
 * Expertise in Kubernetes at scale: deploying, operating, debugging, and tuning clusters handling heterogeneous GPU workloads.

## Description

We're looking for a Site Reliability Engineer (SRE) to drive the reliability of Tinker end-to-end. You'll work alongside the engineers building the platform and research teams to make every layer of the system more robust and resilient., * Define and own end-to-end reliability, from CI/CD flows to production observability and incident response.

 * Develop appropriate Service Level Objectives for distributed training systems, balancing job completion reliability and scheduling latency with development velocity.
 * Design and implement monitoring and observability across the full training path.
 * Drive incident response for Tinker platform issues, ensuring rapid recovery, thorough incident reviews, and systematic improvements that prevent recurrence.
 * Harden multi-tenant isolation and resource scheduling so that LoRA-based workload co-scheduling maximizes utilization without compromising reliability or data separation
 * Collaborate with security teams to address production vulnerabilities