AI Observability Engineer

You will help ensure AI workloads, LLM applications, and cloud infrastructure remain measurable, reliable, and production-ready. Working at the intersection of AI engineering, cloud platforms, and DevOps, you will design systems that improve visibility, performance, and operational excellence. You will contribute to monitoring strategies, telemetry pipelines, dashboards, and automation frameworks supporting large-scale AI environments. The role requires a strong engineering mindset, hands-on technical expertise, and the ability to transform complex operational data into actionable insights. This is an opportunity to shape the reliability of next-generation AI infrastructure within an innovative and globally distributed environment. Accountabilities:

As an AI Observability Engineer, you will own the tools, processes, and systems that provide visibility into AI applications, platform services, and infrastructure performance. You will collaborate with engineering teams to improve reliability, detect issues proactively, and establish scalable observability practices.

• Design, implement, and operate observability solutions for AI workloads, including LLM and agent monitoring.
• Configure and maintain AI tracing systems to capture latency, token usage, cost, quality metrics, prompt analytics, model versions, and safety signals.
• Develop internal tooling and automation solutions using Python for instrumentation and data collection.
• Build and maintain dashboards, metrics, and monitoring solutions using Grafana, Prometheus, and cloud observability platforms.
• Instrument AI platforms and workloads to provide visibility into health, usage, performance, cost, and service-level objectives.
• Create actionable telemetry pipelines and operational insights to support platform engineering improvements.
• Manage infrastructure-as-code workflows using Terraform and maintain CI/CD pipelines for observability tooling.
• Support incident investigations, troubleshooting activities, and root-cause analysis.
• Define and improve monitoring strategies around logs, metrics, traces, alerting, SLIs, and SLOs.
• Collaborate with engineering teams to enhance reliability, scalability, and operational maturity across AI systems.

The ideal candidate is an experienced observability, SRE, DevOps, or cloud engineering professional with strong expertise in monitoring AI-driven systems and building reliable production environments.

• 5-8 years of experience in observability, Site Reliability Engineering, platform engineering, DevOps, or cloud engineering roles.
• Strong hands-on experience with Azure Monitor, Application Insights, Log Analytics, and Managed Grafana.
• Experience working with Langfuse, Grafana, and Prometheus for AI and application monitoring.
• Solid knowledge of Terraform and CI/CD practices.
• Strong Python skills for automation, instrumentation, exporters, and internal tooling development.
• Familiarity with machine learning workloads and AI-specific observability requirements.
• Understanding of logs, metrics, traces, dashboards, alerting strategies, SLIs, and SLOs.
• Ability to work effectively in distributed, fast-paced, and collaborative environments.
• Intermediate or higher English communication skills.

Preferred qualifications include:

• Experience with PromQL and Kusto Query Language.
• Knowledge of OpenTelemetry, including Generative AI semantic conventions.
• Familiarity with LLM evaluation frameworks and AI quality measurement approaches.
• Experience building AI cost monitoring dashboards and FinOps solutions.
• Background with alerting systems, on-call processes, and incident management tooling.
• Experience with Kubernetes and AKS observability.

• Competitive compensation package.
• Career development and continuous learning opportunities.
• Flexible working environment with strong ownership and autonomy.
• Opportunity to work on impactful AI infrastructure and technology projects.
• Collaborative culture with highly skilled international teams.
• Chance to contribute to the evolution of next-generation AI platforms.
• Fast-paced environment focused on innovation, growth, and meaningful impact.