AI Engineer

5+ years of software engineering experience, including 2+ years on ML/LLM systems in production Strong Python, plus working knowledge of Go, C++, or Rust for performance-critical paths Hands-on experience with LLM serving frameworks: vLLM, TensorRT-LLM, TGI, SGLang, or Triton Inference Server Inference optimization techniques: Quantization (INT8/INT4, FP8, AWQ, GPTQ) KV-cache management, paged attention, prefix caching Continuodynamic batching and speculative decoding Tensor and pipeline parallelism Experience with PyTorch and the Hugging Face ecosystem Kubernetes, Docker, and GPU scheduling (NVIDIA GPU operator, node pools, autoscaling) Integration with hosted LLM APIs (OpenAI, Gemini, Anthropic), including streaming, retries, and rate limiting Observability tooling: PrometheGrafana, Splunk, Datadog, OpenTelemetry Solid grounding in distributed systems, API design (REST/gRPC), and performance profiling

Preferred: GenAI orchestration frameworks: Genkit, LangChain, LlamaIndex CUDA, Triton kernels, FlashAttention, or other low-level GPU optimization Model compilation and runtimes: ONNX Runtime, TorchScript/torch.compile, Core ML RAG pipelines and vector databases (FAISS, Milvus, pgvector) LLM evaluation frameworks and building automated quality and regression benchmarks Cost modeling and FinOps for GPU workloads Infrastructure as code (Terraform, Helm) and CI/CD (GitHub Actions, Jenkins) Contributions to open-source inference projects, or published work in ML systems Soft skills: Can turn ambiguous requirements into concrete technical plans and trade-off proposals Communicates clearly in writing (design docs, benchmark reports) Works well with cross-functional teams of engineers and PMs.