> Markdown version of [/videos/201-advanced-caching-patterns-used-by-2000-microservices](https://www.wearedevelopers.com/videos/201-advanced-caching-patterns-used-by-2000-microservices). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Advanced Caching Patterns used by 2000 microservices

How do you handle 500 billion daily requests across 2,000 microservices without cascading failures? Discover advanced caching patterns using Kafka, DynamoDB, and Varnish.

- **Speakers:** [Natan Silnitsky](https://www.wearedevelopers.com/@natan-silnitsky)
- **Event:** World Congress 2021
- **Published:** June 30, 2021
- **Duration:** 44:37
- **URL:** https://www.wearedevelopers.com/videos/201-advanced-caching-patterns-used-by-2000-microservices

## Summary

Managing a system with over 500 billion daily HTTP requests across 2,000 microservices highlights the vital, yet delicate, role of data caching. To improve scalability and prevent cascading failures without adding unnecessary failure points, the golden rule is to never put a cache prematurely if an application is young and traffic is manageable. When latency reductions become mandatory, architecture decisions branch across three specific use cases based on traffic scale and volatility.

For high-risk network constraints, such as a startup microservice dependent on a rigid third-party configuration service, implementing an S3-backed static read-through cache prevents critical downtime. When mitigating database latency is the goal, engineering shifts to an in-memory LRU cache backed by DynamoDB, a solution that seamlessly handles cross-DC automatic data replication. To synchronize these updates across distributed node clusters, DynamoDB change streams are bridged into a Kafka Change Data Capture (CDC) topic, enabling rapid instances warmup and ensuring that data is consistently mirrored. For particularly robust small datasets, leaning on Kafka compacted topics provides an elegant way to trim stale states while housing records strictly in memory.

Alternatively, server-side rendering setups facing heavy external load demand HTTP reverse proxy caching, typically managed by Varnish Cache. Operating exactly ahead of CPU-expensive renderers demands bulletproof invalidation logic, relying on HTTP ETags matched to Kafka purge requests to strip out-of-date content seamlessly. Finally, for situations managing the hoarding effect—where an influx of concurrent requests triggers a cache stampede natively on an un-cached item—employing a functional tool like ZIO's Software Transactional Memory (STM) guarantees fast, atomic execution without duplicate un-cached hits.

**Keywords:** microservices caching patterns, s3 backed static cache, dynamodb cross-dc replication, kafka change data capture topics, kafka compacted topics, lru cache strategy, varnish cache reverse proxy, cache invalidation logic, server-side rendering caching, cascading network failures, http etag invalidation, zio software transactional memory, distributed cache syncing, cache stampede mitigation

## Chapters

1. **The necessity of caching for microservice reliability** (00:02) — High-scale environments use caching to decouple services and mitigate cascading failures.
1. **Caching strategies overview and architectural considerations** (04:48) — Premature caching introduces unnecessary architecture complexity before latency or scale demands it.
1. **Designing a read-through static configuration cache** (08:13) — A read-through pattern backed by object storage shields static data configuration from third-party service outages.
1. **Distributing cache updates using DynamoDB and Kafka** (12:04) — Replicating key-value updates across data centers via a change data capture topic ensures strong consistency and fast cache warmups.
1. **Reducing rendering latency with HTTP reverse proxies** (22:32) — Putting an open-source HTTP cache in front of heavy rendering services significantly decreases CPU load and response times.
1. **Selecting the appropriate cache type for workload requirements** (28:15) — A decision tree logic helps identify whether object storage, distributed databases, or proxy solutions match specific workload requirements.
1. **Enhancing event flows with high-level Kafka consumers** (30:19) — Custom SDK abstractions simplify parallel consumption and error handling in distributed event-driven architectures.
1. **Addressing HTTP headers and custom invalidation strategies** (32:34) — Handling HTTP responses and implementing accurate targeted purging rely on consistent identifier mapping across rendering pipelines.
1. **Mitigating cache stampedes during concurrent database misses** (36:05) — Strategies like increased memory limits and cluster sharding help buffer the impact of simultaneous access requests for uncached items.
1. **Using sidecar deployments to share JVM infrastructure** (38:53) — Encapsulating functionality into sidecar containers allows different language stacks to share core infrastructure without duplicating code.
1. **Safely handling concurrent writes with software transactional memory** (41:30) — Synchronizing state safely across threads using software transactional memory isolates writes and stops duplicate cache population attempts.

## Related Moments

- [Trade-offs between storage and network resources in caching architectures](https://www.wearedevelopers.com/videos/792-introducing-green-it-practices-to-a-large-software-company) (from "Introducing Green IT practices to a large Software Company")
- [Balancing delivery latency with stream reliability and scale](https://www.wearedevelopers.com/videos/32-the-fundamentals-of-online-video) (from "The Fundamentals of Online Video")
- [Navigating caching and downstream throttling in event-driven setups](https://www.wearedevelopers.com/videos/793-from-0-to-1-000-000-how-to-build-a-serverless-raffle-service-for-hyperscale) (from "From 0 to 1.000.000: How to build a serverless raffle service for hyperscale")
- [Reducing reliance on external services with caching](https://www.wearedevelopers.com/videos/511-our-journey-with-spring-boot-in-a-microservice-architecture) (from "Our journey with Spring Boot in a microservice architecture")
- [Introduction to web acceleration and caching paradigms](https://www.wearedevelopers.com/videos/1676-http-headers-that-make-your-website-go-faster) (from "HTTP headers that make your website go faster")
- [Examining strategic data caching patterns via serverless capabilities](https://www.wearedevelopers.com/videos/591-protector-of-the-realm) (from "Protector Of The Realm")

## Related Articles

- [Why Event-Driven Architecture Isn’t About Speed (and When You Actually Need It)](https://www.wearedevelopers.com/magazine/745-why-event-driven-architecture-isn-t-about-speed-and-when-you-actually-need-it)
- [Dev Digest 139 - Soft and hard queries](https://www.wearedevelopers.com/magazine/487-dev-digest-139-soft-and-hard-queries)
- [Why You Shouldn’t Build a Microservice Architecture](https://www.wearedevelopers.com/magazine/118-why-you-shouldn-t-build-a-microservice-architecture)
- [Introducing Redis Agent Memory Server](https://www.wearedevelopers.com/magazine/699-introducing-redis-agent-memory-server)

## Related Jobs

- [Principal Engineer - AI Search & Vector Infrastructure](https://www.wearedevelopers.com/jobs/ext/353953-principal-engineer-ai-search-vector-infrastructure) at **Redis**
- [Devops Engineer](https://www.wearedevelopers.com/jobs/ext/1940926-devops-engineer) at **Bitpanda**
- [Staff Software Engineer](https://www.wearedevelopers.com/jobs/ext/1425755-staff-software-engineer) at **GitHub**
- [Staff Frontend Engineer](https://www.wearedevelopers.com/jobs/48313-staff-frontend-engineer) at **SMG Swiss Marketplace Group**
- [Remote Senior Full-Stack Engineer](https://www.wearedevelopers.com/jobs/ext/682327-remote-senior-full-stack-engineer) at **Edge Impulse**
- [Remote Senior Full-Stack Engineer](https://www.wearedevelopers.com/jobs/ext/643147-remote-senior-full-stack-engineer) at **Edge Impulse**