On-Prem GenAI Platform Engineer

We are seeking an experienced On-Prem GenAI Platform Engineer to build, optimize, and manage enterprise AI/ML platforms supporting Large Language Models (LLMs) and Generative AI workloads. The ideal candidate will have expertise in Kubernetes/OpenShift AI, GPU infrastructure, distributed systems, and LLM inference optimization. < data-start="520" data-end="545">Key Responsibilities

• Build and operate on-prem Kubernetes/OpenShift AI platforms for GenAI and LLM workloads.
• Design and optimize inference solutions using vLLM, TensorRT-LLM, Triton Inference Server, and SGLang.
• Implement advanced optimization techniques including continuous batching, speculative decoding, KV caching, FP8, AWQ, and GPTQ.
• Manage GPU orchestration using Run:AI, CUDA, NCCL, MIG, and tensor parallelism.
• Deploy scalable ML serving frameworks using KServe, Helm, and Kubernetes Operators.
• Monitor platform performance using Prometheus, Grafana, and Arize AI.
• Collaborate with ML and research teams to productionize GenAI solutions.

• LLM Inference: vLLM, TensorRT-LLM, Triton, SGLang
• GPU & Distributed Systems: CUDA, NCCL, MIG, Tensor Parallelism
• Platforms: Kubernetes, OpenShift AI, KServe, Helm
• GPU Orchestration: Run:AI
• Observability: Prometheus, Grafana, Arize AI
• Performance Tuning: GuideLLM, Locust
• GenAI/LLMOps & Platform Engineering

< data-start="1579" data-end="1600">Preferred Skills

• Hybrid Cloud (Azure/Google Cloud Platform)
• Inferentia or alternative AI accelerators
• Service Mesh and GPU Cluster Networking
• Experience with enterprise AI/ML platform engineering

< data-start="1771" data-end="1786">Experience

• 8+ years in Platform Engineering, DevOps, SRE, ML Platform Engineering, or related fields.
• Strong experience supporting enterprise-scale AI/ML and Generative AI environments.