A multi-tenant SaaS application serves 200 enterprise customers, each with unique Claude configurations (system prompts, tool sets, temperature settings). The platform processes 800 requests/minute at peak. The architecture team is evaluating three approaches to per-customer configuration management: (A) reconstruct configuration from database on each request, (B) maintain in-memory configuration cache per application instance with TTL, (C) encode all configuration into a signed JWT passed by the client. During a load test, approach A caused P99 latency of 450ms on configuration retrieval alone. Which architecture should be adopted, and what is the primary risk to mitigate?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you want a shared distributed cache, like Redis, so every app instance sees the same tenant configuration and one invalidation signal can reach them all. Don't ignore cache stampede: a hot key expiring under load can flood the database. Use early expiration or request coalescing to keep it safe.
Full Explanation
The shared distributed cache, such as Redis used across all application instances, is architecturally superior to a per-instance memory cache for a multi-tenant, multi-instance deployment. It solves the core problem identified with per-instance caching: a single cache invalidation signal propagates to all instances simultaneously because they share one cache. Per-instance caches have a split-brain risk — if one instance's cache is invalidated on a customer configuration update, the other instances still serve stale configuration until TTL expiry. The primary risk with Redis is cache stampede: when a popular cache key expires under high load, or during cold start, many requests simultaneously miss and pile onto the database. This is mitigated with probabilistic early expiration or request coalescing. The per-instance in-memory cache option is inadequate for 800 requests per minute across multiple instances, and its stated risk is backwards: each process's cache is not globally shared, so one stale tenant entry does not automatically affect every request across the fleet; the real issue is stale entries surviving on individual instances. The client JWT option is wrong for system prompts — large system prompts, potentially thousands of tokens, encoded in JWTs would far exceed HTTP header size limits and expose configuration data to clients. The database reconstruction option with read replicas and connection pooling does not solve the fundamental O(n) database lookup cost at 800 requests per minute, and read replica lag can still cause configuration drift during updates.