Nele Uhlemann

Handling incidents collaboratively is like solving a rubix cube

What if developers could instrument their code and define SLOs with a simple decorator? Learn a new approach to making observability a shared responsibility.

Handling incidents collaboratively is like solving a rubix cube
#1about 4 minutes

The Rubik's Cube metaphor for engineering teams

Different engineering teams like backend and SREs operate on different sides of the system, requiring collaboration during incidents.

#2about 3 minutes

The first phase of resolving incidents collaboratively

The initial step in incident response is to establish a common understanding and transparency across teams before applying quick fixes.

#3about 2 minutes

Preventing future incidents with best practices

After resolving an incident, teams must collaborate on prevention by documenting best practices for patterns like service retries.

#4about 2 minutes

Discovering incidents through system observability

The discovery phase relies on making systems observable by collecting telemetry data like logs, metrics, and traces.

#5about 2 minutes

Standardizing telemetry collection with OpenTelemetry

OpenTelemetry provides a vendor-neutral standard for instrumenting applications, preventing vendor lock-in for observability backends.

#6about 2 minutes

Simplifying metrics with the Autometrics library

The open-source Autometrics library uses decorators to automatically generate key metrics like latency, errors, and request rate from functions.

#7about 5 minutes

Demo of generating metrics and SLOs from code

A live demo shows how Autometrics provides live metrics in the IDE and helps define SLOs that can be visualized in Grafana.

#8about 1 minute

Summary of collaborative incident management phases

A recap of the three key phases for collaborative incident handling: resolving, preventing, and discovering issues together.

#9about 2 minutes

Q&A on tooling and open source contribution

The speaker answers audience questions about managing tool complexity and the role of community contributions in open-source projects.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Applying agile and SRE principles to incident response
06:30 MIN

Applying agile and SRE principles to incident response

Applying Agile Principles to Incident Management

Actionable takeaways for SREs on incident management
27:09 MIN

Actionable takeaways for SREs on incident management

Serverless Observability: where SLOs meet transforms

Using an incident console to manage response and resolvers
20:29 MIN

Using an incident console to manage response and resolvers

Applying Agile Principles to Incident Management

Fostering cross-team collaboration with SLOs
24:30 MIN

Fostering cross-team collaboration with SLOs

Serverless Observability: where SLOs meet transforms

Understanding observability and the need for a process
02:16 MIN

Understanding observability and the need for a process

Mastering AI-Driven Problem Solving in Engineering with Observability

Handling operational challenges and infrastructure failures at scale
22:38 MIN

Handling operational challenges and infrastructure failures at scale

How building an industry DBMS differs from building a research one

How engineers handle production errors and monitoring
29:58 MIN

How engineers handle production errors and monitoring

DevOps at Netflix

Overcoming observability challenges with a unified platform
18:09 MIN

Overcoming observability challenges with a unified platform

All your telemetry data from any source in one place

Featured Partners

Related Videos

See all videos
Applying Agile Principles to Incident Management 37:33

Applying Agile Principles to Incident Management

Tobias Dunn-Krahn

Mastering AI-Driven Problem Solving in Engineering with Observability27:39

Mastering AI-Driven Problem Solving in Engineering with Observability

Jemiah Sius

Empathy: The secret sauce of Resilience29:55

Empathy: The secret sauce of Resilience

Malin Litwinski

The AI-Ready Stack: Rethinking the Engineering Org of the Future28:09

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Jan Oberhauser, Mirko Novakovic, Alex Laubscher & Keno Dreßel

SRE Methods In an Agency Environment44:31

SRE Methods In an Agency Environment

Martin Beránek

Metrics Handle with Care: The Paradox of Measuring Team Performance17:56

Metrics Handle with Care: The Paradox of Measuring Team Performance

Stefan Stelzer & Volker Zöpfel

Breaking Silos: Successful Collaboration Between Tech & Business Teams in Complex Enterprise Systems26:20

Breaking Silos: Successful Collaboration Between Tech & Business Teams in Complex Enterprise Systems

Stefan Menschner & Alexander Weißhaupt

Unveiling the Dark Side: Navigating the Pitfalls of Digital Ambitions21:39

Unveiling the Dark Side: Navigating the Pitfalls of Digital Ambitions

Johannes Hansen

Related Articles

View all articles
DC
Daniel Cranney
Dev Digest 188: CfP time, the risks of NPM and IKEA algorithms
Inside last week’s Dev Digest 188 . 🤖 GitHub Copilot CLI is now in public review 💻 Microsoft is bringing ‘vibe working’ to office apps 🎣 Attackers abuse AI tools to generate captchas in fishing attacks ⚠️ When LLMs autonomously attack 🧠 Common cause...
Dev Digest 188: CfP time, the risks of NPM and IKEA algorithms
DC
Daniel Cranney
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security
Inside last week’s Dev Digest 196 . ⚖️ Political bias in LLMs 🫣 AI written code causes 1 in 5 security breaches 🖼️ Is there a limit to alternative text on images? 📝 CodeWiki - understand code better 🟨 Long tasks in JavaScript 👻 Scare yourself into n...
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security

From learning to earning

Jobs that call for the skills explored in this talk.

Rust and GoLang

Rust and GoLang

NHe4a GmbH
Karlsruhe, Germany

Remote
€55-65K
Intermediate
Senior
Go
Rust
Backend Engineer (m/w/d)

Backend Engineer (m/w/d)

fulfillmenttools
Köln, Germany

€35-65K
Intermediate
TypeScript
Agile Methodologies
Google Cloud Platform
Site Reliability Engineer

Site Reliability Engineer

Synsel Techniek
Rotterdam, Netherlands

€9K
Intermediate
Azure
Ansible
Terraform
Amazon Web Services (AWS)