Founding Infrastructure Engineer

PANO AI, INC.
Palo Alto, CA, United States
2 days ago
Apply on www.indeed.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
3 years minimum
Working hours
Regular working hours
Job source

Tech stack

Automation of Tests Microsoft Azure Computer Clusters Computer Programming Continuous Integration Software Debugging Linux Distributed Computing Environment Distributed Systems Python (Programming Language) Linux System Administration Machine Learning
+14 more
Octopus Deploy Prometheus Data Logging Autoscaling Grafana Agentic-AI Kubernetes Infrastructure Automation Frameworks Deployment Automation Build Tools Slurm Hardware Infrastructure Terraform Docker

Job description

We’re building infrastructure that lets AI agents run experiments, train models and solve problems at scale. These workloads involve much more than serving requests: agents launch containers, execute code, interact with environments and share expensive compute resources. Making this work reliably takes people who understand the systems underneath it.

As a founding engineer on the team, you’ll build and operate the infrastructure behind our training, inference and agent workloads. A workload might involve launching thousands of containerized environments, scheduling jobs across GPU nodes and collecting results for the next training run. You’ll investigate why jobs fail, where resources sit idle and what slows down experiments, then build improvements that make the whole system more reliable and efficient.

You don’t need a machine learning background. We care most about strong infrastructure fundamentals and experience owning production systems. You’ll pick up the ML side working with our researchers and ML engineers.

Salary range: [range] + [bonus, equity, benefits]

Responsibilities

  • Build and operate Kubernetes clusters for GPU training, model inference and agent execution.
  • Create and maintain Docker images and reproducible environments for research and production workloads.
  • Build systems to launch, monitor, clean up and recover thousands of concurrent containerized tasks.
  • Improve scheduling, autoscaling and resource allocation to reduce GPU idle time and infrastructure costs.
  • Automate infrastructure provisioning, deployments and upgrades using infrastructure as code and CI/CD.
  • Build monitoring and debugging tools that make failures and performance bottlenecks easy to investigate.
  • Troubleshoot issues across Linux hosts, containers, networking, storage and GPU infrastructure.
  • Implement workload isolation, access controls and resource limits for shared clusters and agent environments.
  • Work with researchers and ML engineers to turn experimental workflows into reliable systems.
  • Own incident response and root-cause analysis, and make improvements that prevent recurring failures.

Requirements

  • Bachelor’s degree or equivalent practical experience.
  • 3 years of experience building or operating production infrastructure with Kubernetes and Docker.
  • 3 years of coding experience in Go, Python or a similar language.
  • Experience with Linux systems administration, networking, storage and troubleshooting distributed systems.
  • Experience deploying and managing infrastructure on AWS, GCP or Azure., * Experience operating GPU clusters for distributed training, inference or other compute-intensive workloads, especially on top of SLURM, Volcano, or other schedulers
  • Experience with Kubernetes scheduling, autoscaling, resource quotas and multi-tenant workload isolation.
  • Experience building containerized sandbox environments for running untrusted code, automated tests or AI agents.
  • Experience with infrastructure as code and deployment tools such as Terraform, Helm and Argo CD.
  • Experience building monitoring, logging and alerting with tools such as Prometheus, Grafana and OpenTelemetry.
  • A track record of improving infrastructure reliability, resource utilization or operating costs.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com
Prepare application

Good distractions

Loading talks and stories from around this role…