Head of Infrastructure

General Computing
San Francisco, CA, United States
6 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Working hours
Regular working hours

Tech stack

Artificial Intelligence Cloud Computing Data Centers Software Debugging Firmware Hardware Platform Interface InfiniBand Network Control Graphics Processing Unit (GPU) Load Balancing Computer Network Operations Application Specific Integrated Circuits
+5 more
Large Language Models Kubernetes SGLang Machine Learning Operations TensorRT

Job description

You’ll own the infrastructure layer of our inference cloud end-to-end. Today that means the control plane, the gateway in front of our ASIC fleet, and the observability stack that tells us where every millisecond goes. Over the next 6-8 months, it will grow into a heterogeneous fleet: ASICs for decode, GPUs for pre-fill, and the physical-layer ownership that comes with it., * Own the inference control plane. At the moment, it’s built on configuration provided by our ASIC partner; you’ll be the person who understands it deeply enough to modify, extend, and eventually replace pieces of it.

  • Own the gateway and load balancer that fronts the fleet. Model placement, request routing, and tail-latency engineering live here, driven by live utilization and per-model SLOs.
  • Own observability end-to-end. Per-request tracing from OpenRouter ingress through to the accelerator, with p50/p95/p99 dashboards, SLOs, and alerting that wakes the right person.
  • Run capacity planning against a real, distributed traffic mix across the open-weight models we serve.
  • Own the operational side of the ASIC partnership. Most weird production issues route through their engineering team until we build that expertise in-house, and you’ll be our technical face in those conversations.
  • Bring up the pre-fill side of our disaggregated architecture on a second hardware platform as it comes online. Different vendor, different fabric, different kernels.
  • Build the on-call and incident response practice from zero. Hire and grow the team underneath you.

Requirements

  • 7+ years in infrastructure, SRE, or platform engineering, with at least some of it at a serious inference, ML, or HPC shop.
  • Hands-on with Kubernetes at production scale - not just deploying, but debugging the weird stuff.
  • Strong instincts for tail latency. You think about p99 and utilization as the same problem, not different ones.
  • Comfortable owning a vendor relationship where the vendor’s bugs are now your production issues.
  • Track record of building observability practices that actually catch problems, not just generate dashboards.
  • Have been on-call through real incidents and can talk about what you learned.
  • Want to be the first infra hire at something early, not the tenth at something big.

NIce-to-Haves:

  • Experience operating non-NVIDIA accelerators in production - TPUs, ASICs, or alternative GPU vendors.
  • Background with model-serving stacks (vLLM, TGI, TensorRT-LLM, SGLang).
  • Network fabric experience at data-center scale (RoCE, InfiniBand).
  • Have hired and managed an infra team before.
  • Comfort at the hardware boundary - firmware, drivers, thermals - for when the roadmap takes us there.

About the company

General Compute is the neocloud for alternative chips.

Inference is fragmenting: purpose-built silicon from SambaNova, Cerebras, Positron, d-Matrix, and others already beats GPUs on decode, and we productionize that hardware - we buy the racks, find the data center space, and run it for our customers. Each piece of hardware runs the workload it’s actually built for: prefill stays on GPUs, decode moves to the chip built for it, and today that means generating tokens 5-7× faster than existing GPU-based competitors. Our customers are frontier labs, fast-growing AI application companies, and asset-light clouds.

We closed a $15M seed round in May 2026, and have since closed a $400M debt facility - $100M funded upfront by Upper90, with the balance available for drawdown - collateralized by our inference chips.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…