Meridian Line wants to add an LLM step to the customs-compliance agent that only needs to be available occasionally, at low and unpredictable request volume, and the team does not want to manage or pay for reserved capacity. Which Model Serving option best fits?
Select an answer to reveal the explanation.
Short Explanation
Occasional, unpredictable traffic is exactly what pay-per-token is for: you pay for tokens you actually send, no cluster to size, no idle capacity sitting around waiting.
Full Explanation
Foundation Model APIs offers two modes: pay-per-token, billed per token with no capacity to provision, well suited to low or spiky traffic; and provisioned throughput, which reserves dedicated capacity for consistent, high-volume, latency-sensitive workloads and is billed whether or not it's fully used. Meridian's occasional, unpredictable customs-agent traffic matches pay-per-token's profile exactly — no idle reserved capacity being paid for between bursts. Provisioned throughput sized for peak load is the wrong tool here: it optimizes for sustained high volume, which is the opposite of what's described, and it's the more expensive choice for infrequent use. Standing up a self-hosted open-weights model on an always-on GPU cluster reintroduces the exact operational burden (cluster management, idle cost) the team explicitly wants to avoid. External Models is Databricks' way of routing to a third-party provider (OpenAI, Anthropic, etc.) through a governed endpoint — it doesn't have a 'fixed monthly capacity reservation' billing mode, so that option is fabricated. Exam framing: matching workload shape (steady/high-volume vs. bursty/low-volume) to pay-per-token vs. provisioned throughput is a recurring cost/latency trade-off objective under Application Development and Deployment alike.