AI Monitoring Engineer

We are seeking an AI Monitoring Engineer to design, implement, and manage monitoring solutions for AI, machine learning, and Generative AI systems. The ideal candidate will ensure the availability, reliability, performance, and quality of AI services by developing observability frameworks, monitoring model behavior, detecting anomalies, and automating alerts and incident response. This role collaborates closely with AI Engineers, ML Engineers, Data Scientists, DevOps Engineers, and Platform Engineers to maintain high-performing AI systems in production., * Design and implement monitoring frameworks for AI applications, machine learning models, and AI infrastructure.

• Monitor model performance, prediction quality, latency, throughput, and availability.
• Detect and analyze data drift, model drift, concept drift, and inference anomalies.
• Build dashboards and alerts for AI system health using observability tools.
• Develop Service Level Indicators (SLIs), Service Level Objectives (SLOs), and operational metrics for AI services.
• Implement logging, distributed tracing, and telemetry for AI applications.
• Configure automated alerting and incident response workflows.
• Monitor GPU, CPU, memory, storage, and network utilization for AI workloads.
• Analyze AI system performance trends and recommend optimization strategies.
• Collaborate with AI Engineers and Data Scientists to improve model reliability and operational performance.
• Support root cause analysis (RCA) for production incidents and implement preventive measures.
• Maintain monitoring documentation, operational runbooks, and reporting dashboards.
• Ensure compliance with monitoring, security, governance, and operational standards.

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
• 3 years of experience in Monitoring Engineering, Site Reliability Engineering (SRE), DevOps, Platform Engineering, MLOps, or Cloud Operations.
• Experience monitoring production AI or machine learning systems.
• Strong programming skills in Python and scripting with Bash.
• Experience with Linux administration.
• Hands-on experience with Docker and Kubernetes.
• Experience with AWS, Microsoft Azure, or Google Cloud Platform.
• Strong understanding of distributed systems, cloud infrastructure, and networking.
• Experience with observability tools such as Prometheus, Grafana, OpenTelemetry, ELK Stack, Datadog, Splunk, or New Relic.
• Experience with SQL and time-series databases for operational reporting.

Preferred Qualifications

• Experience with machine learning monitoring platforms such as Evidently AI, Arize AI, Fiddler AI, WhyLabs, or Azure AI monitoring capabilities.
• Experience monitoring Large Language Models (LLMs) and Generative AI applications.
• Knowledge of AI evaluation metrics, prompt monitoring, hallucination detection, and Retrieval-Augmented Generation (RAG) observability.
• Experience with MLOps platforms such as MLflow, Kubeflow, SageMaker, Vertex AI, or Azure Machine Learning.
• Familiarity with vector databases and model serving platforms.
• Experience monitoring GPU-enabled infrastructure and AI inference systems.
• Knowledge of Responsible AI, fairness monitoring, and AI governance.

Technical Skills

• Python
• Bash
• SQL
• Linux
• Docker
• Kubernetes
• AWS / Azure / Google Cloud Platform
• Prometheus
• Grafana
• OpenTelemetry
• ELK Stack
• Datadog
• Splunk
• New Relic
• MLflow
• Kubeflow
• SageMaker
• Vertex AI
• Azure Machine Learning
• Evidently AI
• Arize AI
• Fiddler AI
• WhyLabs
• PostgreSQL
• Redis

Soft Skills

• Strong analytical and troubleshooting skills
• Excellent communication and collaboration
• Attention to detail
• Problem-solving mindset
• Ownership and accountability
• Continuous learning and improvement

Nice to Have

• Experience monitoring LLMs, AI agents, and Retrieval-Augmented Generation (RAG) systems
• Knowledge of AI evaluation frameworks and model quality assessment
• Experience implementing automated anomaly detection and predictive alerting
• Familiarity with FinOps and AI infrastructure cost monitoring
• Cloud, observability, or Kubernetes certifications

Key Performance Indicators (KPIs)

• AI service availability and uptime
• Model inference latency and throughput
• Alert accuracy and mean time to detect (MTTD)
• Mean time to resolve (MTTR) production incidents
• Model drift detection and response time
• Dashboard coverage for AI services
• Reduction in production incidents and false alerts
• Infrastructure resource utilization and optimization