AIOps ML Engineer

We are seeking a highly skilled Senior AIOps ML Engineer to design, build, and scale an enterprise-grade AIOps platform that powers real-time observability, anomaly detection, incident prediction, and intelligent operations. You will work across large-scale streaming data pipelines, Lakehouse architecture, machine learning, and MLOps to deliver intelligent insights from telemetry data including logs, metrics, traces, infrastructure, applications, security, and business KPIs., Lakehouse Architecture & Data Engineering Design and evolve scalable Lakehouse architectures using Delta Lake or Apache Iceberg for multi-domain observability data. Develop and maintain high-throughput streaming data pipelines from OpenTelemetry (OTel) Collectors through Apache Kafka into the Lakehouse with schema enforcement and exactly-once processing. Build robust data transformation models using dbt to create optimized fact and dimension tables for analytics and machine learning. Define and enforce data quality standards, schema contracts, freshness SLAs, and governance policies. Optimize query performance using partitioning, Z-ordering, bloom filters, clustering, and materialized views for large-scale time-series workloads. Machine Learning & AIOps Design, develop, and deploy machine learning models for: Multivariate anomaly detection Root cause analysis (RCA) Incident prediction and forecasting Behavioral analytics KPI correlation Build real-time inference pipelines using Apache Flink or Spark Structured Streaming. Develop advanced log intelligence capabilities including log clustering, NLP-based classification, and error deduplication using techniques such as DRAIN3 and LogBERT. Design and manage enterprise ML Feature Stores including feature engineering, versioning, point-in-time joins, and backfill pipelines. Implement comprehensive MLOps practices including model monitoring, drift detection, automated retraining, and performance tracking. AIOps Platform Engineering Build scalable end-to-end AIOps workflows covering telemetry ingestion, feature computation, model inference, intelligent alerting, and automated remediation. Develop high-performance model serving infrastructure supporting REST/gRPC APIs and batch inference with low-latency SLAs. Integrate AI-driven insights with incident management platforms such as PagerDuty and Opsgenie. Develop business impact analytics to quantify incident severity, affected users, service degradation, and revenue impact. Security & Compliance Collaborate with security teams to build observability pipelines supporting threat detection, UEBA, CVE correlation, and security analytics. Develop anomaly detection models for suspicious access patterns and lateral movement detection. Ensure compliance with enterprise security standards including data residency, PII masking, governance, and audit logging. Engineering Excellence Define telemetry schema contracts with instrumentation teams to improve upstream data quality. Drive engineering best practices, architecture standards, and technical design reviews. Mentor junior engineers and contribute to the evolution of the organization''s AIOps and observability platform.

Programming & Data Engineering Strong proficiency in Python and advanced SQL Extensive experience with Apache Kafka Hands-on experience with Apache Flink or Spark Structured Streaming Expertise in Apache Spark Strong understanding of distributed data processing and streaming architectures Lakehouse & Data Platforms Hands-on experience with Delta Lake or Apache Iceberg Data Lakehouse architecture Data modeling and schema design dbt Data quality frameworks and governance Performance optimization for large-scale analytics workloads Machine Learning & MLOps Time-series analytics Multivariate anomaly detection Predictive analytics Root cause analysis models Feature engineering Feature Store implementation Model deployment and monitoring Drift detection and automated retraining Production MLOps best practices Observability OpenTelemetry (OTel) Application Performance Monitoring (APM) Distributed tracing Metrics, logs, and traces Observability platforms Log analytics AIOps Intelligent alerting Event correlation Auto-remediation Incident prediction Noise reduction Streaming ML inference Cloud & Platform AWS, Azure, or Google Cloud Platform Kubernetes Docker CI/CD pipelines Linux