Wide World Importers needs a multi-agent order-exception handler that spikes during holiday peaks, must recover from agent failures, and must control inference cost. Architects are selecting compute for orchestration workers and tool hosts. Which approach best balances scalability, reliability, operational efficiency, security, and cost?
Select an answer to reveal the explanation.
Short Explanation
You want elastic containerized or serverless compute for orchestration and tool endpoints, autoscaling on queue depth, health probes and retries, private networking, and separate scaling for model-heavy and lightweight tool work. That balances bursty holiday load, failure recovery, security, and cost. Don't pin expensive model capacity as if it were a static web server, and don't trust browser-only or debug-mode setups for production.
Full Explanation
The correct approach is elastic containerized or serverless compute for orchestration and tool endpoints, autoscaling on queue depth, health probes and retries for agent workers, private networking, and separate scale policies for high-token model calls versus lightweight tools. This meets scalability, reliability, operational efficiency, security, and cost needs, and differentiated scaling matches spiky order-exception traffic. Always-on dedicated GPU VMs sized for Black Friday peaks year-round, with agents running as unmonitored console apps under one admin login, use static peak-sized GPUs that waste cost and violate security and reliability practices. The single developer laptop running the orchestrator in debug mode as the production control plane is not a resilient control plane. Running all agents inside the user's browser with no backend so Azure compute cost is zero cannot safely host privileged tools, durable workflow state, or enterprise networking controls. Prefer managed identities, private endpoints, and autoscaling signals tied to workflow queues.