V
System Design #13 Hello Interview Architecture Peak Load: 100,000+ QPS

Design Auto-Complete / Type-Ahead Backend

Ultra-low-latency distributed search suggestions: Client debouncing, Edge CDN prefix micro-cache, Redis Top-5 cache, In-Memory Trie with precomputed Top-K vectors (O(p) lookup), and asynchronous Kafka/Batch aggregation with zero-downtime atomic pointer swaps.

Target: Sub-10ms P99 Server Latency
Query Throughput100k+ QPS5B+ searches per day
P99 Server Latency< 8ms P99O(p) Precomputed Top-K
Cache Hit Ratio86.4%Edge CDN + Redis Zipf Law
Total Trie Memory~26.5 GBSharded across cluster
Trie Hot-Swap Time< 1msZero-lock atomic pointer
Search Box:
150ms Client Debounce Active
Simulate Flows:
PATH 1: REAL-TIME QUERY SERVING PATH (SYNCHRONOUS • READ-HEAVY • SUB-10MS SLA) PATH 2: DATA COLLECTION & OFFLINE AGGREGATION PATH (ASYNCHRONOUS • WRITE-HEAVY • DECOUPLED) Debounced GETForward Miss1. Check Redis Top-52. Fallback O(p)User Submits Search (Enter / Click)Stream BatchRolling AggPost-Order Top-KSave ArtifactPull Green ImageAtomic Pointer Flip: current_trie.store(new_trie) < 1msBrowser Search Box• 150ms Debounce• Local Memoization Cache• AbortController Cancel• Cuts 45% Backend TrafficAPI Gateway & Edge CDN• Anycast BGP Routing• 1-2 Char Hot Prefix Cache• Token Bucket Rate Limit• Absorbs 65% Hot SpikesTypeahead Query Service• Stateless Go/Rust Fleet• Lowercase & Normalize• Truncate at 30 Chars• Async Clickstream DispatchPrefix Cache (Redis)• Precomputed Top-5 Lists• Power Law (Zipf's Law)• 80%+ Queries Hit RAM• Sub-2ms P99 LatencyIn-Memory Trie Fleet• Precomputed Top-K / Node• O(p) Lookup (Zero DFS)• Sharded via Consistent Hash• Instant Microsecond ReadSearch Log Collector• Async Background Worker• Ring Buffer In-Memory• Zero User Query Delay• Snappy Binary BatchesKafka Event Bus• Partition: hash(query)• 100k+ Msgs/Sec Ingestion• Decouples Read from Write• 3x Replication FactorOffline Batch Aggregator• Rolling 7-30 Day Window• Time-Decay Score Weights• Profanity & PII Filter• Prunes Count < 5Offline Trie Builder• Bottom-Up Post-Order Pass• Precomputes Node Top-K• Compiles Binary Image• Off-Peak Nightly RunZero-Downtime Swapper• Double Buffering RAM• Atomic Pointer Flip < 1msTrie Snapshot Store (S3)• Versioned Binary Blobs• Instant Cold Boot Recovery
Synchronous Query Path Redis Precomputed Top-5 In-Memory Trie Fleet (O(p)) Asynchronous Offline Aggregation
Click any component box to view Hello Interview deep dive

Core Requirements (Hello Interview)

Functional 1: Return Top 5-10 suggestions in real time as the user types each character.
Functional 2: Non-personalized ranking based on historical search popularity and rolling query frequency.
Functional 3: Fast prefix matching with graceful behavior on typos / empty prefix results.
Non-Functional: Ultra-low latency (< 10-20ms server response), high availability (99.99%), 100k+ QPS scale, and zero-lock atomic memory swapping.

What is Expected at Each Level?

Mid-Level: Design basic Trie, separate read query path from write collection path, and implement basic Redis caching for hot terms.
Senior: Proactively store precomputed Top-K vectors on Trie nodes to achieve strict O(p) time. Implement client debounce (150ms), multi-tier caching, time-decay scoring, and consistent hash sharding.
Staff+: Address hot-key stampedes on single-character prefixes, design zero-downtime atomic memory pointer swaps in RAM, analyze memory sizing (~30 GB), and formulate graceful degradation under extreme traffic.

Browser & Mobile Search Box

Role & Overview

Captures user keystrokes in the search bar. Debounces outbound requests by 100-150ms so fast typists don't spam the backend on intermediate characters, and checks local in-memory cache for recently typed prefixes.

Architecture Implementation

1. Debounce timer (150ms) fires network request only after typing pauses. 2. In-memory memoization (LRU map) caches recent queries for the active session. 3. Request cancellation: Superseded in-flight requests are aborted using AbortController.

Client-Side Traffic Reduction
⚠️ Good Solution: Unthrottled Keystrokes: Every character fires an HTTP request. At 350 CPM, a 10-character query sends 10 requests, swamping edge infrastructure.
⭐ Great Solution: 150ms Keystroke Debounce + AbortController + Local Memoization: Eliminates 40-50% of backend queries with zero perceived latency penalty to the user.
Specification & Snippet
// Client Debounce Example
let debounceTimer = null;
let abortCtrl = null;

function onKeystroke(prefix) {
  clearTimeout(debounceTimer);
  if (abortCtrl) abortCtrl.abort(); // Cancel previous in-flight request
  
  if (localCache.has(prefix)) {
    renderSuggestions(localCache.get(prefix));
    return;
  }
  
  debounceTimer = setTimeout(async () => {
    abortCtrl = new AbortController();
    const res = await fetch(`/api/v1/suggest?q=${encodeURIComponent(prefix)}`, { signal: abortCtrl.signal });
    const data = await res.json();
    localCache.set(prefix, data.suggestions);
    renderSuggestions(data.suggestions);
  }, 150);
}