WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Aug 20, 2025 • Session details

Logs in observability - Correlation

Michal Bojko

Stop treating logs as an afterthought. By adopting log-driven development and K-means clustering, you can transform petabytes of noisy data into actionable business intelligence within seconds.

Pause
Mute Enter Fullscreen
#1 about 4 min

Understanding the daily challenges of massive log volumes

Analyzing gigabytes of chaotic infrastructure logs presents a significant scaling challenge without proper data storytelling.

#2 about 5 min

Creating contextual data stories from messy log files

Adding identifiers and descriptive values during logging enables teams to trace user flows across disparate systems.

#3 about 3 min

Automating log correlation through preprocessing and clustering

Parsing varying timestamp formats and establishing time constraints allows algorithms to cluster discrete system events efficiently.

#4 about 4 min

Applying K-means clustering to group massive datasets

The K-means algorithm translates system metrics into numeric values to efficiently group parameters like high CPU usage across millions of logs.

#5 about 3 min

Choosing between rule-based queries and machine learning

While exact filtering works for known parameters, non-deterministic log structures require dynamic algorithms to reveal hidden infrastructure relationships.

#6 about 4 min

Transitioning from monitoring to proactive system observability

Rather than just spotting immediate failures, true observability analyzes historical data patterns to predict issues and measure overarching business efficiency.

#7 about 3 min

Implementing log-driven development for better software reporting

Designing clear log messages and categorization hierarchies before writing code ensures the application generates intuitive, actionable system insights.

Matching moments

2:41 min

Defining observability beyond basic metrics, logs, and traces

Mathias Palmersheim Mathias Palmersheim · Europe 2026 Virtual

1:23 min

Understanding logs, metrics, and traces for observability

Carly Richmond Carly Richmond · Europe 2026 Virtual

2:29 min

Moving beyond logging to comprehensive API observability

Rustam Mehmandarov Rustam Mehmandarov · World Congress 2026 Europe

17:03 min

Introduction to metrics and observability challenges in monitoring

Liam Hurrell · LIVE

2:12 min

Elevating observability with anomaly detection and root cause analysis

Raz Cohen · LIVE

1:30 min

Processing and structuring noisy application logs for LLMs

Alisa Hrustic Alisa Hrustic · Europe 2026 Virtual

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33
11 months ago

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28
about 1 year ago

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel, Laura Moritz, Madalina Florean

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12
about 1 year ago

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe, Konstantin Vinogradov, Nils Eiteneyer, Stephan Wirris

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19
about 1 year ago

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler, Jasmin Kaiser, Jeff Hausmann, Sonja Alvarez

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09
about 1 year ago

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher, Jan Oberhauser, Keno Dreßel, Mirko Novakovic

AI Sovereignty: What Does It Take? thumbnail
29:13
about 1 year ago

AI Sovereignty: What Does It Take?

Boris Hecker, Clemens Wasner, Robin Hermann, Sonja Alvarez

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 24, 2026 · 14:50–15:20

Stage 4

When Logging Becomes The Outage: Escaping the ECS Logging Trap

Rahul Tanniru

Senior Vice President Of Software Engineering, Jp Morgan Chase

Rahul Tanniru
Open session

World Congress 2026 North America

September 25, 2026 · 09:00–09:30

Stage 1

Test Before Release, Enforce at Runtime: Governance for Tool-Using AI Agents

Sachin Gupta

Member of Technical Staff 2 at eBay

Sachin Gupta
Open session

World Congress 2026 North America

September 24, 2026 · 11:40–12:10

Stage 4

Taming Rogue Agents: Observability-Driven Evaluation for Production Reliability

Anagha Rumade, Anjana Umapathy, Apoorva Jaiswal

Anagha Rumade
Anjana Umapathy
Apoorva Jaiswal
Open session

World Congress 2026 North America

September 24, 2026 · 10:20–10:50

Stage 4

AI Decision Observability: Enabling Transparency and Trust in Intelligent Systems

Amjad Shaikh, Soumil Mandal

Amjad Shaikh
Soumil Mandal
Open session

World Congress 2026 North America

September 24, 2026 · 15:30–16:00

Stage 9

Run your agents in Kubernetes: Build once, deploy anywhere. But really?

Michal Salanci

Senior Systems Engineer at ESET Cybersecurity

Michal Salanci
Open session

World Congress 2026 North America

September 25, 2026 · 11:00–11:30

Outdoor Stage

Loop Engineering: Designing and Observing the Loops That Run Your Agents With Opik

Abigail Morgan

AI Developer Advocate at Comet

Abigail Morgan

Related videos

Strategies for Efficient Log Management in Large-Scale Kubernetes Clusters thumbnail
30:01
World Congress 2026 Europe • about 2 months ago

Strategies for Efficient Log Management in Large-Scale Kubernetes Clusters

Aliaksandr Valialkin

Mastering AI-Driven Problem Solving in Engineering with Observability thumbnail
27:39
World Congress 2024 • about 2 years ago

Mastering AI-Driven Problem Solving in Engineering with Observability

Jemiah Sius

Data binning and understanding histograms thumbnail
30:35
Europe 2026 Virtual • 2 months ago

Data binning and understanding histograms

Michal Bojko

Observability with OpenTelemetry & Elastic thumbnail
30:29
LIVE • almost 2 years ago

Observability with OpenTelemetry & Elastic

Iulia Feroli

Why LLMs Need Observability and How to Do It thumbnail
32:41
Europe 2026 Virtual • 2 months ago

Why LLMs Need Observability and How to Do It

Rahul Gupta

Keycloak case study: Making users happy with service level indicators and observability thumbnail
27:15
World Congress 2025 • about 1 year ago

Keycloak case study: Making users happy with service level indicators and observability

Alexander Schwartz

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.