> Markdown version of [/jobs/ext/3253307-senior-ai-infrastructure-engineer](https://www.wearedevelopers.com/jobs/ext/3253307-senior-ai-infrastructure-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior AI Infrastructure Engineer

- **Company:** 104 STACKYARD LLC
- **Location:** San Francisco, CA, United States
- **Experience:** Expert
- **Salary:** $160,000.0 - $200,000.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Systems Engineering, Profiling, Nvidia CUDA, Data Centers, Linux, Memory Management, InfiniBand, Python (Programming Language), Performance Tuning, Remote Direct Memory Access, AI Infrastructure, Graphics Processing Unit (GPU), Large Language Models, Software Troubleshooting, Parallel Computation, TensorRT, Hardware Infrastructure, Docker
- **Published:** September 21, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=f5a7862d86af350e

## About the Role

We're particularly interested in engineers who have actually operated LLM inference infrastructure, not just consumed hosted model APIs.

You should have strong experience with several of:

 * vLLM, SGLang, TensorRT-LLM, TGI, Triton, or comparable inference runtimes
 * NVIDIA GPUs, CUDA, NCCL and the surrounding software stack
 * GPU profiling, benchmarking, memory management, and performance optimization
 * Quantization and model formats such as FP8, INT8, AWQ or GPTQ
 * Tensor parallelism, pipeline parallelism, and distributed inference
 * Docker and Linux systems engineering
 * Python and the modern ML ecosystem
 * Production observability and performance troubleshooting

## Description

You'll own the technical depth of our inference stack: how models are loaded, served, optimized, observed, and scaled across different GPU hardware.

You'll work hands-on with systems such as vLLM, SGLang, TensorRT-LLM, CUDA and ROCm, digging below the API layer into what actually determines inference performance.

You'll:

 * Deploy and operate production model-serving infrastructure using vLLM and similar runtimes
 * Optimize TTFT, inter-token latency, throughput, GPU utilization, memory usage, and cost
 * Tune batching, KV cache behavior, parallelism, quantization, model configuration, and runtime parameters
 * Benchmark models across different NVIDIA and AMD GPU architectures
 * Diagnose GPU memory, driver, runtime, networking, and performance bottlenecks
 * Build reliable patterns for multi-GPU and multi-node inference
 * Improve how StackYak provisions, configures, monitors, and operates inference workloads
 * Help determine the right hardware and serving configuration for different models and customer workloads
 * Build the telemetry and tooling needed to understand inference performance in production
 * Work closely with the rest of the engineering team as we turn this operational knowledge into the StackYak platform