AI/ML Ops Engineer - Data & Reporting, Onsite

We have an immediate need for an experienced Senior AI/ML Ops Engineer with strong expertise in LLM evaluation, MLOps, AI agent reliability, Python, CI/CD, observability, and production AI systems to design, build, and operate a comprehensive evaluation and monitoring platform for LLM-powered agents.

This is an onsite position based in Austin, TX, with a preference for local candidates and candidates with prior Apple experience, for a candidate with deep experience in Python, TypeScript, LLMs, LLM Evaluation, AI Agents, MLOps, ML Engineering, CI/CD, GitHub Actions, GitLab, Jenkins, Buildkite, OpenTelemetry, Datadog, Grafana, Prometheus, LangSmith, Langfuse, Arize Phoenix, Braintrust, W&B Weave, RAG Pipelines, Multi-Agent Systems, Agent Orchestration, LLM-as-a-Judge, Golden Datasets, Regression Testing, Automated Scoring, Prompt Management, Model Routing, A/B Testing, Experimentation, Causal Inference, Knowledge Graphs, Monitoring, Alerting, Incident Response, RCA, and Production Observability.