ML Infrastructure Engineer

We serve inference at $/token margins that don’t tolerate sloppy stacks. You’ll own the serving layer - vLLM, TensorRT-LLM, Triton - and the benchmarking discipline that keeps it honest., * Serving-stack selection per workload (continuous batching vs. static, KV cache strategy, paged attention).

• Quantization (FP8, AWQ, GPTQ) and the eval harness that proves the trade-offs.
• The InferenceBench-style benchmarks that compare our serving against the field.

• Shipped at least one production inference stack on H100s or A100s.
• Read a recent vLLM PR for fun.
• Strong opinions about speculative decoding.