> Markdown version of [/jobs/ext/1999187-ai-observability-engineer](https://www.wearedevelopers.com/jobs/ext/1999187-ai-observability-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Observability Engineer - **Company:** Jobgether - **Location:** Germany - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Application Performance Management, Cloud Computing, Cloud Engineering, Continuous Integration, DevOps, HP Systems Insight Manager, Python (Programming Language), Log Analysis, Machine Learning, Operational Data Store, Reliability Engineering, Prometheus, Kusto Query Language, AI Infrastructure, Datadog, Cloud Monitoring, Large Language Models, Grafana, Generative AI, AI Platforms, Kubernetes, Terraform - **Published:** August 9, 2026 - **Apply:** https://www.adzuna.de/details/5833709793 ## About the Role The ideal candidate is an experienced observability, SRE, DevOps, or cloud engineering professional with strong expertise in monitoring AI-driven systems and building reliable production environments. * 5-8 years of experience in observability, Site Reliability Engineering, platform engineering, DevOps, or cloud engineering roles. * Strong hands-on experience with Azure Monitor, Application Insights, Log Analytics, and Managed Grafana. * Experience working with Langfuse, Grafana, and Prometheus for AI and application monitoring. * Solid knowledge of Terraform and CI/CD practices. * Strong Python skills for automation, instrumentation, exporters, and internal tooling development. * Familiarity with machine learning workloads and AI-specific observability requirements. * Understanding of logs, metrics, traces, dashboards, alerting strategies, SLIs, and SLOs. * Ability to work effectively in distributed, fast-paced, and collaborative environments. * Intermediate or higher English communication skills. Preferred qualifications include: * Experience with PromQL and Kusto Query Language. * Knowledge of OpenTelemetry, including Generative AI semantic conventions. * Familiarity with LLM evaluation frameworks and AI quality measurement approaches. * Experience building AI cost monitoring dashboards and FinOps solutions. * Background with alerting systems, on-call processes, and incident management tooling. * Experience with Kubernetes and AKS observability. ## Description You will help ensure AI workloads, LLM applications, and cloud infrastructure remain measurable, reliable, and production-ready. Working at the intersection of AI engineering, cloud platforms, and DevOps, you will design systems that improve visibility, performance, and operational excellence. You will contribute to monitoring strategies, telemetry pipelines, dashboards, and automation frameworks supporting large-scale AI environments. The role requires a strong engineering mindset, hands-on technical expertise, and the ability to transform complex operational data into actionable insights. This is an opportunity to shape the reliability of next-generation AI infrastructure within an innovative and globally distributed environment. Accountabilities: As an AI Observability Engineer, you will own the tools, processes, and systems that provide visibility into AI applications, platform services, and infrastructure performance. You will collaborate with engineering teams to improve reliability, detect issues proactively, and establish scalable observability practices. * Design, implement, and operate observability solutions for AI workloads, including LLM and agent monitoring. * Configure and maintain AI tracing systems to capture latency, token usage, cost, quality metrics, prompt analytics, model versions, and safety signals. * Develop internal tooling and automation solutions using Python for instrumentation and data collection. * Build and maintain dashboards, metrics, and monitoring solutions using Grafana, Prometheus, and cloud observability platforms. * Instrument AI platforms and workloads to provide visibility into health, usage, performance, cost, and service-level objectives. * Create actionable telemetry pipelines and operational insights to support platform engineering improvements. * Manage infrastructure-as-code workflows using Terraform and maintain CI/CD pipelines for observability tooling. * Support incident investigations, troubleshooting activities, and root-cause analysis. * Define and improve monitoring strategies around logs, metrics, traces, alerting, SLIs, and SLOs. * Collaborate with engineering teams to enhance reliability, scalability, and operational maturity across AI systems. ## Related Videos - [Unlocking the AI Black Box: Building Trust in the Era of Agentic Production](https://www.wearedevelopers.com/videos/100086-unlocking-the-ai-black-box-building-trust-in-the-era-of-agentic-production) - [5 steps for running a Kubernetes environment at scale](https://www.wearedevelopers.com/videos/88-5-steps-for-running-a-kubernetes-environment-at-scale) - [Debugging in the Dark](https://www.wearedevelopers.com/videos/1658-debugging-in-the-dark) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Software Engineering Social Connection: Yubo’s lean approach to scaling an 80M-user infrastructure](https://www.wearedevelopers.com/videos/1583-software-engineering-social-connection-yubo-s-lean-approach-to-scaling-an-80m-user-infrastructure) - [#90DaysOfDevOps - The DevOps Learning Journey](https://www.wearedevelopers.com/videos/548-90daysofdevops-the-devops-learning-journey) ## Related Articles - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts) - [Best US AI Conferences for CTOs in 2026: Build vs. Buy, Vendor Evaluation, and Peer Intelligence](https://www.wearedevelopers.com/magazine/736-best-us-ai-conferences-for-ctos-in-2026-build-vs-buy-vendor-evaluation-and-peer-intelligence)