A multi-tenant SaaS application serves 200 enterprise customers, each with unique Claude configurations (system prompts, tool sets, temperature settings). The platform processes 800 requests/minute at peak. The architecture team is evaluating three approaches to per-customer configuration management: (A) reconstruct configuration from database on each request, (B) maintain in-memory configuration cache per application instance with TTL, (C) encode all configuration into a signed JWT passed by the client. During a load test, approach A caused P99 latency of 450ms on configuration retrieval alone. Which architecture should be adopted, and what is the primary risk to mitigate?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a distributed cache (Redis) is architecturally superior to per-instance memory cache for a multi-tenant multi-instance deployment. It solves the core problem identified in Option A's risk: a single cache invalidation signal propagates to all instances simultaneously because they share one cache.
Full explanation below image
Full Explanation
A distributed cache (Redis) is architecturally superior to per-instance memory cache for a multi-tenant multi-instance deployment. It solves the core problem identified in Option A's risk: a single cache invalidation signal propagates to all instances simultaneously because they share one cache. Per-instance caches (Option B) have a split-brain risk — if instance 1's cache is invalidated on customer config update, instances 2-5 still serve stale config until TTL. The primary risk with Redis is cache stampede: when a popular cache key expires under high load, many requests simultaneously miss and pile onto the database. This is mitigated with probabilistic early expiration or request coalescing. Option B's single-instance cache is inadequate for 800 req/min across multiple instances. Option C's JWT approach is wrong for system prompts — large system prompts (thousands of tokens) encoded in JWTs would far exceed HTTP header size limits and expose configuration data to clients. Option A's read replica fix doesn't solve the fundamental O(n) database lookup cost at 800 req/min.