Hauke Brammer

MLOps on Kubernetes: Exploring Argo Workflows

Stop writing brittle glue code for your MLOps pipelines. See how Argo Workflows provides a Kubernetes-native way to define and execute complex processes declaratively.

MLOps on Kubernetes: Exploring Argo Workflows
#1about 3 minutes

Understanding the core principles and lifecycle of MLOps

MLOps applies DevOps principles to machine learning to automate and streamline the entire model lifecycle from data collection to deployment and monitoring.

#2about 4 minutes

Why Argo Workflows is a powerful Kubernetes-native engine

Argo Workflows is a Kubernetes-native engine that orchestrates complex, multi-stage processes as custom resources, eliminating the need for extensive glue code.

#3about 4 minutes

Building and running a basic workflow with YAML

A simple workflow is defined in YAML using templates for each step, which are then executed inside containers on a Kubernetes cluster.

#4about 4 minutes

Managing data files in pipelines using artifacts

Argo artifacts simplify data handling by automatically downloading input files from cloud storage into a container and uploading outputs upon completion.

#5about 4 minutes

Orchestrating complex training jobs with DAGs

Directed acyclic graph (DAG) templates in Argo allow you to define complex workflows with multiple dependencies, enabling parallel and sequential task execution for model training.

#6about 4 minutes

Building resilient batch inference pipelines with retries

For reliable batch inference, Argo's retry strategies with configurable limits and backoff policies can automatically recover from transient failures in individual steps.

#7about 3 minutes

Evaluating if Argo Workflows is right for your team

Argo is ideal for teams already using Kubernetes to manage complex, multi-stage ML pipelines, but may be overkill for small projects or teams without Kubernetes expertise.

#8about 1 minute

Integrating Argo with tools like Argo CD and MLflow

Argo Workflows can be used alongside Argo CD for deployment and MLflow for experiment tracking, with Argo providing more flexible, language-agnostic container orchestration.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Videos

See all videos
DevOps for Machine Learning44:40

DevOps for Machine Learning

Hauke Brammer

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

GitOps for the people29:49

GitOps for the people

Lian Li

Effective Machine Learning - Managing Complexity with MLOps45:45

Effective Machine Learning - Managing Complexity with MLOps

Simon Stiebellehner

The state of MLOps - machine learning in production at enterprise scale47:19

The state of MLOps - machine learning in production at enterprise scale

Bas Geerdink 

Get ready for operations by pull requests50:02

Get ready for operations by pull requests

Liviu Costea

How E.On productionizes its AI model & Implementation of Secure Generative AI.19:27

How E.On productionizes its AI model & Implementation of Secure Generative AI.

Kapil Gupta

MLOps - What’s the deal behind it?29:24

MLOps - What’s the deal behind it?

Nico Axtmann

From learning to earning

Jobs that call for the skills explored in this talk.