A proprietary trading firm needs to propagate trade execution events — occurring at sub-millisecond frequency during market hours — to downstream systems including a real-time P&L calculator, a risk aggregator, a regulatory reporting engine, and an ML-based market impact estimator. The current batch-based ETL pipeline introduces 15-minute latency that is incompatible with intraday risk limits. Which architectural pattern most effectively decouples event producers from consumers while supporting the required throughput and latency profile?
Select an answer to reveal the explanation.
Short Explanation
You need an event stream like a financial newswire — it publishes every trade once, and any subscriber can tune in at its own pace, even replay missed events. A durable, partitioned log decouples the execution system from downstream consumers and supports the required sub-millisecond propagation. The REST polling, RabbitMQ-style queue, and database-trigger approaches can't give you the same independent consumption and replay.
Full Explanation
Event-driven architecture (EDA) using a durable log-based streaming platform is the standard infrastructure pattern for high-frequency financial event propagation. Apache Kafka and Redpanda implement the log-structured broker model where events are written once to a partitioned, replicated log and can be consumed independently by any number of consumer groups. This provides three critical properties for the use case described: producer-consumer decoupling — the execution management system publishes events without knowing or caring about downstream systems; independent consumption — P&L, risk, regulatory reporting, and ML systems consume the same events at their own rates and processing guarantees without interfering with each other; and replay capability — the immutable event log allows downstream systems to reprocess historical events for backtesting, audit, or disaster recovery.
The REST API polling approach trades latency for simplicity but creates a polling anti-pattern. With 30-second polling intervals, the firm still has substantial latency. More critically, multiple consumers each polling a production execution system multiplies load on that system and creates thundering-herd problems at market open. REST polling cannot match the push-based latency of streaming.
The RabbitMQ-style message queue approach is architecturally distinct from a log-structured streaming platform. Traditional message queues deliver each message to exactly one consumer and delete it after acknowledgment, and the described round-robin delivery to a single consumer is task distribution, not broadcast event propagation. For a use case where four downstream systems all need every trade event, a queue requires fan-out exchange configurations that become complex to manage and lack the replay semantics of a durable log. Queues excel at task distribution; logs excel at event propagation to multiple independent consumers.
The database-trigger approach couples the execution management system's database directly to all downstream systems. This creates a synchronous dependency chain where a slow regulatory reporting engine can back-pressure and destabilize the trading system's core database. It violates the fundamental principle that operational systems should not be blocked by analytical consumers — a principle especially critical in latency-sensitive trading environments.