Mario Valderrama

Operating etcd for Managed Kubernetes

To achieve zero-downtime migrations, we modified etcd snapshots to artificially inflate the revision number. Discover the surprising challenges of operating etcd at massive scale.

Operating etcd for Managed Kubernetes
#1about 3 minutes

The journey to managed Kubernetes at IONOS

From its first release in 2019 to managing over 20,000 clusters, IONOS scaled its Kubernetes service by building on a massive etcd foundation.

#2about 4 minutes

Evolving etcd deployment strategies over time

The team progressed from the CoreOS operator and Bitnami Helm charts to a simplified custom Helm chart for better control and stability.

#3about 3 minutes

Understanding multi-tenancy and its performance impact

Using a shared etcd with client-side prefixes reduces cost but creates noisy neighbor problems, requiring careful tuning like compaction and defragmentation.

#4about 3 minutes

Iterating on etcd cluster layouts for reliability

Initial cross-location clusters suffered from latency and revision drift, leading to a more stable single data center layout using availability zones.

#5about 3 minutes

A zero-downtime control plane migration strategy

A live migration process using `etcdctl mirror` allows moving a Kubernetes control plane to a new etcd cluster without global downtime or data loss.

#6about 3 minutes

Manipulating etcd revisions for seamless migration

By modifying an etcd snapshot to insert a high revision number, clients like kubelet continue watching for changes without needing a restart after migration.

#7about 2 minutes

Future plans for etcd management and automation

The team is working on automating the migration process, offering dedicated etcd clusters, and contributing their migration learnings to the Kaji project.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Organizing a developer conference for 15,000 attendees
01:32 MIN

Organizing a developer conference for 15,000 attendees

Cat Herding with Lions and Tigers - Christian Heilmann

Selecting strategic partners and essential event tools
03:17 MIN

Selecting strategic partners and essential event tools

Cat Herding with Lions and Tigers - Christian Heilmann

The business case for sustainable high performance
03:34 MIN

The business case for sustainable high performance

Sustainable High Performance: Build It or Pay the Price

Breaking down silos between HR, tech, and business
03:39 MIN

Breaking down silos between HR, tech, and business

What 2025 Taught Us: A Year-End Special with Hung Lee

Balancing business, technology, and people for holistic success
06:51 MIN

Balancing business, technology, and people for holistic success

The Future of HR Lies in AND – Not in OR

Using AI agents to modernize legacy COBOL systems
06:28 MIN

Using AI agents to modernize legacy COBOL systems

Devs vs. Marketers, COBOL and Copilot, Make Live Coding Easy and more - The Best of LIVE 2025 - Part 3

Balancing the trade-off between efficiency and resilience
03:38 MIN

Balancing the trade-off between efficiency and resilience

What 2025 Taught Us: A Year-End Special with Hung Lee

Establishing a single source of truth for all data
02:39 MIN

Establishing a single source of truth for all data

Cat Herding with Lions and Tigers - Christian Heilmann

Featured Partners

Related Videos

See all videos
Chaos in Containers - Unleashing Resilience27:52

Chaos in Containers - Unleashing Resilience

Maish Saidel-Keesing

Fifty Shades of Kubernetes Autoscaling30:26

Fifty Shades of Kubernetes Autoscaling

Mario-Leander Reimer

Kubernetes Maestro: Dive Deep into Custom Resources to Unleash Next-Level Orchestration Power!29:21

Kubernetes Maestro: Dive Deep into Custom Resources to Unleash Next-Level Orchestration Power!

Um e Habiba

From Factory Floor to Kubernetes Core: Building an Edge Platform One Step at a Time17:34

From Factory Floor to Kubernetes Core: Building an Edge Platform One Step at a Time

Dean Oren & Stefan Belsch

Kubernetes Security Best Practices23:08

Kubernetes Security Best Practices

Rico Komenda

Mastering Kubernetes – Beginner Edition57:24

Mastering Kubernetes – Beginner Edition

Hannes Norbert Göring

Winning the Hybrid Cloud26:19

Winning the Hybrid Cloud

Alex Soto

Kubernetes Security - Challenge and Opportunity42:45

Kubernetes Security - Challenge and Opportunity

Marc Nimmerrichter

Related Articles

View all articles
Learning Kubernetes made easy with KubeCampus
Learning to use Kubernetes? KubeCampus by Kasten offers free educational content for all skill levels to get you started!Kubernetes is an open-source system for deploying, scaling and managing containerized applications. It allows you to deploy your ...
Learning Kubernetes made easy with KubeCampus
BB
Benedikt Bischof
MLOps – What’s the deal behind it?
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed ...
MLOps – What’s the deal behind it?

From learning to earning

Jobs that call for the skills explored in this talk.