WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Aug 20, 2025 • Session details

Logs in observability - Correlation

Michal Bojko

Stop treating logs as an afterthought. By adopting log-driven development and K-means clustering, you can transform petabytes of noisy data into actionable business intelligence within seconds.

Pause
Mute Enter Fullscreen
#1 about 4 min

Understanding the daily challenges of massive log volumes

Analyzing gigabytes of chaotic infrastructure logs presents a significant scaling challenge without proper data storytelling.

#2 about 5 min

Creating contextual data stories from messy log files

Adding identifiers and descriptive values during logging enables teams to trace user flows across disparate systems.

#3 about 3 min

Automating log correlation through preprocessing and clustering

Parsing varying timestamp formats and establishing time constraints allows algorithms to cluster discrete system events efficiently.

#4 about 4 min

Applying K-means clustering to group massive datasets

The K-means algorithm translates system metrics into numeric values to efficiently group parameters like high CPU usage across millions of logs.

#5 about 3 min

Choosing between rule-based queries and machine learning

While exact filtering works for known parameters, non-deterministic log structures require dynamic algorithms to reveal hidden infrastructure relationships.

#6 about 4 min

Transitioning from monitoring to proactive system observability

Rather than just spotting immediate failures, true observability analyzes historical data patterns to predict issues and measure overarching business efficiency.

#7 about 3 min

Implementing log-driven development for better software reporting

Designing clear log messages and categorization hierarchies before writing code ensures the application generates intuitive, actionable system insights.

Matching moments

2:41 min

Defining observability beyond basic metrics, logs, and traces

Mathias Palmersheim Mathias Palmersheim · Europe 2026 Virtual

1:23 min

Understanding logs, metrics, and traces for observability

Carly Richmond Carly Richmond · Europe 2026 Virtual

2:29 min

Moving beyond logging to comprehensive API observability

Rustam Mehmandarov Rustam Mehmandarov · World Congress 2026 Europe

17:03 min

Introduction to metrics and observability challenges in monitoring

Liam Hurrell · LIVE

1:09 min

The evolution of machine learning in observability

Costa Costa · World Congress 2025

2:12 min

Elevating observability with anomaly detection and root cause analysis

Raz Cohen · LIVE

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33
12 months ago

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28
about 1 year ago

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel, Laura Moritz, Madalina Florean

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12
about 1 year ago

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe, Konstantin Vinogradov, Nils Eiteneyer, Stephan Wirris

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19
about 1 year ago

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler, Jasmin Kaiser, Jeff Hausmann, Sonja Alvarez

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09
about 1 year ago

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher, Jan Oberhauser, Keno Dreßel, Mirko Novakovic

AI Sovereignty: What Does It Take? thumbnail
29:13
about 1 year ago

AI Sovereignty: What Does It Take?

Boris Hecker, Clemens Wasner, Robin Hermann, Sonja Alvarez

Upcoming sessions on this topic

Open session

We Deleted Our Observability Stack and Rebuilt It with OTEL: 12 Engineers to 4 at 20K+ clusters

  • Yash Sharma

    DigitalOcean

    Developer Advocate

Open session

How We Cut Our API's p99 Latency from Minutes to Under a Second

  • Deepak Agrawal

    Atlassian

    Principal Software Engineer

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

Beyond File Dumps: Context Engineering for Coding Agents

  • Animesh Dutta

    Arm

    Senior Software Engineer

Open session

From Profiler to Production: Measuring What Actually Matters in React & React Native

  • Andrei Tazetdinov

    Dynatrace

    React Native Developer

Open session

Shipping MCP to Real Users: Scaling, Auth, and the PII Leak You Didn't See Coming

  • Hadar Geva

    Myop

    CTO & Co-founder

Related videos

Strategies for Efficient Log Management in Large-Scale Kubernetes Clusters thumbnail
30:01
World Congress 2026 Europe • 3 months ago

Strategies for Efficient Log Management in Large-Scale Kubernetes Clusters

Aliaksandr Valialkin

Mastering AI-Driven Problem Solving in Engineering with Observability thumbnail
27:39
World Congress 2024 • about 2 years ago

Mastering AI-Driven Problem Solving in Engineering with Observability

Jemiah Sius

Data binning and understanding histograms thumbnail
30:35
Europe 2026 Virtual • 3 months ago

Data binning and understanding histograms

Michal Bojko

Observability with OpenTelemetry & Elastic thumbnail
30:29
LIVE • almost 2 years ago

Observability with OpenTelemetry & Elastic

Iulia Feroli

Why LLMs Need Observability and How to Do It thumbnail
32:41
Europe 2026 Virtual • 3 months ago

Why LLMs Need Observability and How to Do It

Rahul Gupta

Keycloak case study: Making users happy with service level indicators and observability thumbnail
27:15
World Congress 2025 • about 1 year ago

Keycloak case study: Making users happy with service level indicators and observability

Alexander Schwartz

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.