Senior ML Infrastructure Engineer

Design and operate production-grade inference infrastructure on RDU architecture to optimize performance and cost. Own the public API surface and build accuracy verification infrastructure to ensure reliable, high-throughput services.

Requirements: Requires a B.S. in Computer Science or a related field and 5+ years of experience building large-scale distributed systems. Must be proficient in Python and familiar with modern LLM inference techniques and serving stacks like vLLM.

Key Skills: Distributed Systems, ML Serving, Python, Systems Design, LLM Inference, vLLM, Algorithms, Data Structures, Concurrency, API Design, Speculative Decoding, Constrained Decoding, Prompt Caching, Long-context Inference, TensorRT-LLM, SGLang