> Markdown version of [/videos/1232-observability-with-opentelemetry-elastic?t=1183](https://www.wearedevelopers.com/videos/1232-observability-with-opentelemetry-elastic?t=1183). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Observability with OpenTelemetry & Elastic

Are you still debugging complex Python microservices with native print statements? Discover how to achieve vendor-agnostic observability by combining OpenTelemetry's standardized signals with Elastic's powerful dashboards.

- **Speakers:** [Iulia Feroli](https://www.wearedevelopers.com/@iulia-feroli)
- **Event:** WeAreDevelopers LIVE
- **Published:** October 30, 2024
- **Duration:** 30:29
- **URL:** https://www.wearedevelopers.com/videos/1232-observability-with-opentelemetry-elastic

## Summary

As Python applications grow in complexity—particularly with the rise of black-box large language models (LLMs) and "GenAI Ops"—traditional debugging methods like print statements and native logging quickly become unmanageable. These naive approaches fail to provide the holistic context needed when applications span multiple languages, microservices, and web interfaces. True observability shifts the paradigm from manually searching for errors after they occur to proactively instrumenting systems to emit continuous, standardized signals.

Addressing this gap, OpenTelemetry has emerged as a rapidly growing open-source standard—described as "the language that everyone is talking." It enables developers to instrument their code to emit three core pillars of telemetry: logs (records of discrete events), metrics (numerical data tracked over time, like CPU utilization), and traces (the sequential path of a request through various application spans). By leveraging the Python SDK for OpenTelemetry, developers can implement both manual and automatic instrumentation, seamlessly wrapping popular libraries like Jinja or urllib. This decoupling of telemetry collection from specific backend vendors prevents lock-in while ensuring consistent monitoring across the entire technological stack.

To visualize and analyze these emitted signals, teams can ingest OpenTelemetry data into scalable observability platforms like Elastic. Because Elastic natively adopted the OpenTelemetry schema in 2023, data flows frictionlessly into centralized dashboards for anomaly detection, throughput analysis, and error tracking without translation overhead. Ultimately, moving toward structured, standardized observability allows engineering teams to understand a system's internal state solely from its external outputs—a vital transition for optimizing model performance, debugging distributed transactions, and ensuring the reliability of modern Python architectures.

**Keywords:** observability, opentelemetry, python debugging, code instrumentation, distributed tracing, application metrics, system logging, telemetry signals, elastic observability, genai ops, llm monitoring, vendor lock-in avoidance, open-source monitoring, anomaly detection, latency tracking

## Chapters

1. **The increasing complexity of modern application observability** (00:02) — As applications and generative AI solutions become more complex, standardized monitoring processes are required to diagnose unexpected failures.
1. **Limitations of native debugging and logging in Python** (01:58) — Relying on print statements or the built-in logging module creates manual overhead and lacks compatibility across robust software systems.
1. **Defining OpenTelemetry as a standardized observability framework** (07:10) — OpenTelemetry provides a rapidly growing open-source toolkit to standardize data collection across all vendor ecosystems and programming languages.
1. **Core concepts of application instrumentation and telemetry** (09:05) — Instrumenting application code enables the continuous emission of telemetry signals that detail system behavior without inspecting internal mechanisms.
1. **Differentiating between metrics, logs, and distributed traces** (13:00) — Telemetry data breaks down into numeric metrics over time, discrete event logs, and sequence-based distributed traces for mapping request paths.
1. **Preventing vendor lock-in with an independent observability layer** (17:13) — Implementing an open standard separates data collection from specific cloud providers or analysis tools to ensure architectural flexibility.
1. **Implementing manual and automatic instrumentation in Python** (19:43) — Python developers can embed specific library calls manually or use an automated toolkit to seamlessly track common frameworks without altering core code.
1. **Integrating Elastic dashboards with OpenTelemetry data schemas** (22:22) — Routing standardized telemetry into Elastic's unified schema provides powerful search capabilities and comprehensive dashboards for advanced debugging.
1. **Live dashboard demonstration of tracing errors and system metrics** (25:35) — Running a containerized Python demo highlights how observability platforms surface latency bottlenecks, crash logs, and complex request transactions automatically.
1. **Future trends for monitoring complex software and language models** (28:44) — Embedding open-source observability practices simplifies long-term maintenance and model optimization for emerging machine learning applications.

## Related Moments

- [Distinguishing between telemetry, monitoring, and observability](https://www.wearedevelopers.com/videos/100158-the-opentelemetry-mistakes-i-keep-seeing-and-how-to-stop-making-them) (from "The OpenTelemetry mistakes I keep seeing (and how to stop making them)")
- [Exploring advanced observability stacks and distributed infrastructure challenges](https://www.wearedevelopers.com/videos/544-plan-ci-cd-on-the-enterprise-level) (from "Plan CI/CD on the Enterprise level!")
- [Integrating OpenTelemetry pipeline platforms with embedded third-party observability providers](https://www.wearedevelopers.com/videos/841-hands-on-with-opentelemetry) (from "Hands on with OpenTelemetry")
- [Maintaining code quality with open standard observability and pipelines](https://www.wearedevelopers.com/videos/1706-the-ai-ready-stack-rethinking-the-engineering-org-of-the-future) (from "The AI-Ready Stack: Rethinking the Engineering Org of the Future")
- [Introduction to metrics and observability challenges in monitoring](https://www.wearedevelopers.com/videos/57-all-your-telemetry-data-from-any-source-in-one-place) (from "All your telemetry data from any source in one place")
- [Discovering incidents using logs, metrics, and traces](https://www.wearedevelopers.com/videos/680-handling-incidents-collaboratively-is-like-solving-a-rubix-cube) (from "Handling incidents collaboratively is like solving a rubix cube")

## Related Articles

- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)

## Related Jobs

- [Lead Software Engineer - Data Engineering](https://www.wearedevelopers.com/jobs/ext/2000968-lead-software-engineer-data-engineering) at **Dynatrace**
- [Software Engineer, Platform Engineering (L2)](https://www.wearedevelopers.com/jobs/ext/1956829-software-engineer-platform-engineering-l2) at **Twilio**
- [Software Engineer (L2) Segment Team](https://www.wearedevelopers.com/jobs/ext/209570-software-engineer-l2-segment-team) at **Twilio**
- [Software Engineer (L2) Segment Team](https://www.wearedevelopers.com/jobs/ext/238794-software-engineer-l2-segment-team) at **Twilio**
- [Senior Engineer, Infrastructure Platform](https://www.wearedevelopers.com/jobs/ext/328836-senior-engineer-infrastructure-platform) at **Intercom, Inc.**
- [Senior AI Agent Software Engineer (Go, Python) (m/f/x)](https://www.wearedevelopers.com/jobs/48277-senior-ai-agent-software-engineer-go-python-m-f-x) at **Dynatrace**