Nele Uhlemann

Handling incidents collaboratively is like solving a rubix cube

What if developers could instrument their code and define SLOs with a simple decorator? Learn a new approach to making observability a shared responsibility.

Handling incidents collaboratively is like solving a rubix cube
#1about 4 minutes

The Rubik's Cube metaphor for engineering teams

Different engineering teams like backend and SREs operate on different sides of the system, requiring collaboration during incidents.

#2about 3 minutes

The first phase of resolving incidents collaboratively

The initial step in incident response is to establish a common understanding and transparency across teams before applying quick fixes.

#3about 2 minutes

Preventing future incidents with best practices

After resolving an incident, teams must collaborate on prevention by documenting best practices for patterns like service retries.

#4about 2 minutes

Discovering incidents through system observability

The discovery phase relies on making systems observable by collecting telemetry data like logs, metrics, and traces.

#5about 2 minutes

Standardizing telemetry collection with OpenTelemetry

OpenTelemetry provides a vendor-neutral standard for instrumenting applications, preventing vendor lock-in for observability backends.

#6about 2 minutes

Simplifying metrics with the Autometrics library

The open-source Autometrics library uses decorators to automatically generate key metrics like latency, errors, and request rate from functions.

#7about 5 minutes

Demo of generating metrics and SLOs from code

A live demo shows how Autometrics provides live metrics in the IDE and helps define SLOs that can be visualized in Grafana.

#8about 1 minute

Summary of collaborative incident management phases

A recap of the three key phases for collaborative incident handling: resolving, preventing, and discovering issues together.

#9about 2 minutes

Q&A on tooling and open source contribution

The speaker answers audience questions about managing tool complexity and the role of community contributions in open-source projects.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Applying agile and SRE principles to incident response
06:30 MIN

Applying agile and SRE principles to incident response

Applying Agile Principles to Incident Management

Actionable takeaways for SREs on incident management
27:09 MIN

Actionable takeaways for SREs on incident management

Serverless Observability: where SLOs meet transforms

Using an incident console to manage response and resolvers
20:29 MIN

Using an incident console to manage response and resolvers

Applying Agile Principles to Incident Management

Fostering cross-team collaboration with SLOs
24:30 MIN

Fostering cross-team collaboration with SLOs

Serverless Observability: where SLOs meet transforms

How engineers handle production errors and monitoring
29:58 MIN

How engineers handle production errors and monitoring

DevOps at Netflix

Overcoming observability challenges with a unified platform
18:09 MIN

Overcoming observability challenges with a unified platform

All your telemetry data from any source in one place

Improving incident response to make on-call less painful
28:08 MIN

Improving incident response to make on-call less painful

What Developers Get Wrong About Application Quality

Building resilience across all software stack layers
13:39 MIN

Building resilience across all software stack layers

System Resilience: Surviving the Software Storm

Featured Partners

Related Videos

See all videos
Applying Agile Principles to Incident Management 37:33

Applying Agile Principles to Incident Management

Tobias Dunn-Krahn

Mastering AI-Driven Problem Solving in Engineering with Observability27:39

Mastering AI-Driven Problem Solving in Engineering with Observability

Jemiah Sius

Empathy: The secret sauce of Resilience29:55

Empathy: The secret sauce of Resilience

Malin Litwinski

The AI-Ready Stack: Rethinking the Engineering Org of the Future28:09

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Jan Oberhauser, Mirko Novakovic, Alex Laubscher & Keno Dreßel

SRE Methods In an Agency Environment44:31

SRE Methods In an Agency Environment

Martin Beránek

Metrics Handle with Care: The Paradox of Measuring Team Performance17:56

Metrics Handle with Care: The Paradox of Measuring Team Performance

Stefan Stelzer & Volker Zöpfel

Breaking Silos: Successful Collaboration Between Tech & Business Teams in Complex Enterprise Systems26:20

Breaking Silos: Successful Collaboration Between Tech & Business Teams in Complex Enterprise Systems

Stefan Menschner & Alexander Weißhaupt

Unveiling the Dark Side: Navigating the Pitfalls of Digital Ambitions21:39

Unveiling the Dark Side: Navigating the Pitfalls of Digital Ambitions

Johannes Hansen

From learning to earning

Jobs that call for the skills explored in this talk.

Rust and GoLang

Rust and GoLang

NHe4a GmbH
Karlsruhe, Germany

Remote
€55-65K
Intermediate
Senior
Go
Rust
Cloud Engineer (m/w/d)

Cloud Engineer (m/w/d)

fulfillmenttools
Köln, Germany

€50-65K
Intermediate
TypeScript
Google Cloud Platform
Continuous Integration
Backend Engineer (m/w/d)

Backend Engineer (m/w/d)

fulfillmenttools
Köln, Germany

€35-65K
Intermediate
TypeScript
Agile Methodologies
Google Cloud Platform
Lead Full-Stack Engineer

Lead Full-Stack Engineer

Hubert Burda Media
München, Germany

€80-95K
Intermediate
Senior
React
Vue.js
Node.js
MongoDB
+1