⚡ Design a Distributed Cache

1M+ Read QPS • Consistent Hashing • XFetch Stampede Guard • CDC Invalidation

High-throughput in-memory distributed caching architecture: L1 in-process microsecond cache (Caffeine), Consistent Hashing ring with 256 virtual nodes, clustered Redis L2 sharding, probabilistic early refresh (XFetch) to defeat the thundering herd, and Debezium CDC-driven database invalidation.

Strategy:

Cluster Read QPS

1,048,500req/s

64 Active Shards

P99 Read Latency

0.78msRAM

L1: 30μs • L2: 0.8ms

Cache Hit Ratio

96.4%global

Shields DB from 96.4% traffic

Memory Utilization

7.82 TB/ 10 TB

Sampled LRU Active

Stampede Guard

Active (0 Spikes)XFetch

0 Thundering Herds

🌐 Distributed Cache Architecture Topology

Multi-tier memory topology: Ingress → L1 Process RAM → Consistent Hash Router → L2 Shards → Stampede Guard → CDC Bus → Primary DB.

System Ready • Click any node to inspect

Idle

Click any component card in the SVG to inspect architectural trade-offs.

In-Memory Cache (L1/L2)

Consistent Hash Router

Stampede Guard

CDC Invalidation Bus

👆 Click any component in the architecture diagram to open its floating interview details.

Core Requirements (The Hello Interview Breakdown)

Functional Requirements:

1. get(key): Retrieve cached value with sub-millisecond response time.
2. set(key, value, ttl): Store key-value pair with automatic expiration.
3. Eviction: Evict least recently / frequently used keys when memory reaches capacity.

Non-Functional Requirements:

1. Ultra-Low Latency: P99 read latency < 1ms across all geographically distributed nodes.
2. High Availability: 99.999% uptime with automated master-replica failover (Raft/Sentinel).
3. Scalability: Horizontally partition keys across 100+ nodes with minimal rebalancing overhead.
4. Cache Consistency: Prevent stale reads through automated invalidation and atomic update pipelines.

Scale: 1M+ Read QPS, 50k Write QPSP99 Latency: < 1ms (L1 < 50μs)

What is Expected at Each Level?

Mid-Level: Design the Cache-Aside pattern, explain how TTL expiration works, articulate basic LRU eviction concepts, and show how Redis caches database query results.

Senior: Design a Consistent Hashing ring with virtual nodes, explain how to eliminate cache stampedes using Singleflight / XFetch probabilistic refresh, contrast Write-Through vs Write-Back, and build an L1 in-process cache to protect L2 from hot keys.

Staff+: Proactively solve dual-write race conditions using CDC (Debezium/Kafka) log tailing, analyze memory allocation and fragmentation overhead (jemalloc sizing), design cross-region active-active cache synchronization, and formulate automated circuit breaking and degraded read modes under catastrophic partition failure.

L2 Distributed Cache Cluster (Redis)

Role & Overview

Clustered in-memory data store partitioned across dozens of master-replica pairs. Delivers sub-millisecond lookups, pipelined batch operations, and configurable persistence (RDB snapshots + AOF rewrite).

Architecture Implementation

1. Master-replica topology: Each shard has 1 Master (writes/reads) and 2 Replicas (read scaling + failover). 2. Memory optimization: Hashes, zsets, and string values stored using compact ziplist / jemalloc allocations. 3. Sentinel / Raft consensus initiates automated failover in < 3 seconds if master becomes unresponsive.

Caching Write Patterns & Trade-offs

❌ Bad Solution: Write-Back (Write-Behind) without WAL: Writes to cache first and delays DB write by 30 seconds. A Redis crash destroys all uncommitted customer updates permanently.

⚠️ Good Solution: Write-Through: App writes to cache and DB synchronously. Guarantees consistency, but write latency is doubled because it waits for slow disk DB.

⭐ Great Solution: Cache-Aside with Atomic Eviction (or CDC invalidation): Writes go straight to the primary DB; upon commit, cache key is evicted. Next read repopulates fresh data. High throughput and zero permanent drift.

Specification & Snippet

// Redis Cache-Aside Pattern
func GetUserProfile(ctx context.Context, userID string) (*User, error) {
    key := "user:" + userID
    val, err := redisClient.Get(ctx, key).Result()
    if err == nil {
        return deserialize(val), nil // Cache HIT (< 1ms)
    }
    // Cache MISS: Query DB and re-hydrate
    user, err := db.QueryUser(ctx, userID)
    if err != nil { return nil, err }
    redisClient.Set(ctx, key, serialize(user), 10*time.Minute)
    return user, nil
}