> Markdown version of [/videos/145-database-devops-with-containers?t=1289](https://www.wearedevelopers.com/videos/145-database-devops-with-containers?t=1289). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Database DevOps with Containers

Stop treating your database like application code. Discover how to use Docker containers to securely distribute sanitized, production-like data into dev environments for risk-free daily deployments.

- **Speakers:** Rob Richardson
- **Event:** WeAreDevelopers LIVE
- **Published:** May 12, 2021
- **Duration:** 39:15
- **URL:** https://www.wearedevelopers.com/videos/145-database-devops-with-containers

## Summary

In the realm of modern software development, application DevOps traditionally treats source control as the ultimate master copy. However, database DevOps requires a paradigm shift because the production user data itself is the true master copy. This presentation explores how to build a robust database DevOps pipeline using containers to safely distribute production-like data into dev, test, and pre-production environments. By fully automating database restores and schema migrations, engineering teams can practice production deployments daily, significantly reducing the risks of rollout failures and shifting quality feedback earlier into the development cycle.

To safely distribute this data without violating compliance or security, organizations must adopt an 'anonymize, sanitize, and shrink' methodology. Anonymization alters PII to ensure HIPAA or PCI compliance and prevents accidental engagement, such as emailing real users from a staging environment. Sanitization cleans out live API keys and production secrets to prevent internal leaks. Crucially, shrinking the database focuses on reducing bulk—such as dropping mid-tier accounts—while intentionally leaving the largest, most complex datasets and anomalous records intact. Preserving unexpected nulls and dangling foreign keys ensures developers can properly test and code against real-world edge cases.

Implementing this methodology securely relies on Docker and SQL Server running inside Linux containers. Utilizing multi-stage Docker builds acts as a critical security boundary; operations can restore the full production backup and execute redaction scripts in an ephemeral build stage, subsequently copying only the cleaned data files into the final developer image. This guarantees that unredacted layers are never accidentally distributed or exposed via caching. Wrapping these customized, data-safe SQL containers into automated CI/CD pipelines eliminates manual configuration steps, thereby empowering developers to independently reset their environments in seconds while preserving absolute data integrity.

**Keywords:** database devops pipelines, docker multi-stage builds, SQL server containers, production data sanitization, data anonymization techniques, database shrinking strategies, container virtualization, test environment provisioning, ephemeral developer databases, database migration automation, shift-left testing, HIPAA compliance workflows, PCI compliance checks, CI/CD database automation

## Chapters

1. **Defining the goals of database DevOps** (01:45) — Automating processes ensures consistency, reduces operational costs, and minimizes the risk of human error during deployments.
1. **Building an application DevOps pipeline with user feedback** (03:00) — Starting from user suggestions and ending with user feedback creates a continuous loop where source control serves as the ultimate system of record.
1. **Treating production data as the master database copy** (05:55) — Unlike application code where source control is authoritative, production data is the true source of truth for safe database architectures.
1. **Adapting production databases for safe non-production use** (08:14) — Anonymizing sensitive user details, removing production secrets, and safely shrinking data bulk protects privacy while maintaining realistic testing scenarios.
1. **Automating database pipelines using container virtualization** (13:16) — Leveraging containers rather than full virtual machines provides a lightweight, automated method to manage environment dependencies by virtualizing the operating system kernel.
1. **Defining infrastructure consistently using Docker configuration files** (18:26) — Transforming simple configuration text files into portable image binaries allows teams to spin up complete application environments uniformly.
1. **Managing ephemeral system layers in Docker containers** (21:29) — Docker constructs executable images using stacked read-only layers and provisions an ephemeral read-write layer during execution to cleanly isolate runtime modifications.
1. **Provisioning ephemeral SQL Server environments using Docker** (25:21) — Containerized databases provide a fast and disposable mechanism for teams to execute workloads without the overhead of manual server installations.
1. **Building sanitized multi-stage Docker database images** (28:50) — Automating the restore, sanitization, and packaging process via multi-stage builds prevents intermediate production secrets from leaking into the final developer image.
1. **Validating sanitized container databases in local development** (34:39) — Connecting locally to a freshly built database container empowers engineers to safely query realistic data and instantly reset their environment.

## Related Moments

- [Shifting mindsets on running databases in containers](https://www.wearedevelopers.com/videos/74-databases-on-kubernetes-why-you-should-care) (from "Databases on Kubernetes: Why you should care")
- [Deploying mission-critical databases within container infrastructures](https://www.wearedevelopers.com/videos/382-kubernetes-and-microservices-with-multi-model-databases) (from "Kubernetes and Microservices with Multi-Model Databases")
- [Bringing DevOps practices to data transformation with DBT](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) (from "Modern Data Architectures need Software Engineering")
- [Modern improvements driving database adoption in Kubernetes](https://www.wearedevelopers.com/videos/255-databases-on-kubernetes) (from "Databases on Kubernetes")
- [Running isolated database integration tests using containers](https://www.wearedevelopers.com/videos/190-continuous-testing-run-automated-tests-for-every-change) (from "Continuous testing - run automated tests for every change!")
- [Eliminating operational errors with containerized application deployments](https://www.wearedevelopers.com/videos/36-devsecops-security-in-devops) (from "DevSecOps: Security in DevOps")

## Related Articles

- [Building AI Solutions with Rust and Docker](https://www.wearedevelopers.com/magazine/494-building-ai-solutions-with-rust-and-docker)
- [Now is the time for industrialized software development](https://www.wearedevelopers.com/magazine/601-now-is-the-time-for-industrialized-software-development)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)

## Related Jobs

- [Lead Software Engineer - Data Engineering](https://www.wearedevelopers.com/jobs/ext/2000968-lead-software-engineer-data-engineering) at **Dynatrace**
- [Senior Data Engineer](https://www.wearedevelopers.com/jobs/ext/1589390-senior-data-engineer) at **Douglas GmbH**
- [Principal Software Engineer, Database Infrastructure](https://www.wearedevelopers.com/jobs/ext/1465908-principal-software-engineer-database-infrastructure) at **GitHub**
- [Devops Engineer](https://www.wearedevelopers.com/jobs/ext/1940926-devops-engineer) at **Bitpanda**
- [Staff Software Engineer, Database Infrastructure](https://www.wearedevelopers.com/jobs/ext/1470125-staff-software-engineer-database-infrastructure) at **GitHub**
- [DevSecOps Architect - Healthcare](https://www.wearedevelopers.com/jobs/ext/1905317-devsecops-architect-healthcare) at **BWI GmbH**