Implementing continuous delivery in a data processing pipeline
Álvaro Martín Lozano - 3 years ago
Continuous delivery is not a new concept anymore. It has been already in many technical talks for quite a while. Usually, these talks focus on standard software development: where the deliverable is a piece of software. This deliverable can be presented as an installable or deployed to be consumed as a service. When we speak about Data Pipelines data is considered the deliverable, which implies some changes in the continuous delivery process.
In this talk, we will present the technical decisions and design that we followed in a recent project to be able to apply continuous delivery into a Data Processing pipeline.