> Markdown version of [/jobs/ext/4618529-inference-infrastructure-engineer-serving](https://www.wearedevelopers.com/jobs/ext/4618529-inference-infrastructure-engineer-serving). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Inference Infrastructure Engineer, Serving

- **Company:** ELORIAN AI, INC.
- **Location:** San Francisco, CA, United States
- **Experience:** Experienced
- **Salary:** $200,000
- **Contract:** Permanent contract
- **Skills:** Open Source Technology, Inference Optimization, Azure Machine Learning, Load Balancing, Large Language Models, Discretization, Low Latency, SGLang, TensorRT, VLLM, Decoding
- **Published:** November 7, 2026
- **Apply:** https://www.juju.com/job/28_25304_f42649342d41a2ede1cfa0fa4d7668e8

## About the Role

* 3+ years of experience building low-latency, high-throughput inference serving systems for large models
 * Strong knowledge of inference optimization techniques (quantization, batching, speculative decoding, KV cache management)
 * Hands-on experience with serving frameworks such as vLLM, TensorRT-LLM, Triton, or SGLang
 * Experience with multi-GPU/multi-node model parallelism for serving (tensor or pipeline parallel)
 * Experience with autoscaling and load balancing for production ML services
 * A track record of GPU cost optimization at scale

Preferred qualifications (strong candidates may have some, not all):

 * Experience serving multimodal (vision + language) models
 * Contributions to open-source ML or systems infrastructure projects (e.g., vLLM, SGLang, TensorRT-LLM, Triton)
 * A bias for action and comfort working across stacks and teams in an early-stage environment

## Description

We're looking for an infrastructure engineer to design, optimize, and scale the systems that serve our large multimodal models. Your work will make inference faster, more cost-effective, and more reliable, so our teams can focus on advancing model capabilities rather than managing bottlenecks.

Our focus is on performant, efficient inference, both to power real-world applications and to accelerate research. This role owns the infrastructure that ensures every deployment and evaluation runs smoothly at scale for our visual foundation models.

What You Will Do

 * Build low-latency, high-throughput inference serving systems for our large multimodal models
 * Design and implement techniques that improve latency, throughput, and efficiency, including quantization, batching, speculative decoding, and KV cache management
 * Optimize our codebase and GPU fleet to fully utilize hardware FLOPs, bandwidth, and memory
 * Implement multi-GPU and multi-node model parallelism for serving (tensor or pipeline parallel)
 * Build autoscaling and load balancing for production ML services
 * Establish standards for reliability, observability, and reproducibility across the inference stack
 * Collaborate with researchers to enable high-performance inference for novel architectures