Meridian Line's evaluation team wants to build a held-out test set from real production traffic on the safety-review summarizer endpoint, capturing the actual prompts sent and responses returned so they can later score them for groundedness. Which AI Gateway feature should be enabled on the endpoint to make this possible?
Select an answer to reveal the explanation.
Short Explanation
You can’t build an evaluation set from requests you never captured. Payload logging is the AI Gateway feature that records prompts and responses so you can retrieve and sample them later.
Full Explanation
Payload logging, an AI Gateway control enabled on the endpoint, records the content of requests and responses—not just aggregate counts—providing the raw material to sample real production traffic into an evaluation set for later groundedness scoring. Rate limiting governs how many requests are allowed through; setting a high threshold may avoid throttling, but it captures nothing and provides no logged content for a dataset. Model version aliasing associates a name with a model version for routing purposes; it does not capture request or response payloads and cannot build an evaluation set on its own. Traffic splitting distributes a percentage of live requests across served model versions on the same endpoint; it does not route traffic to a separate evaluation endpoint, and splitting alone does not record payload content, so logging is still needed. Enable payload logging with awareness of what the payloads contain: crew and incident narratives can carry PII, so logging must be considered alongside applicable governance and masking requirements. Exam caveat: distinguish capturing live prompts and responses from controlling request volume, naming model versions, or splitting traffic. Practical check: verify that the endpoint’s logs contain both the actual prompts sent and responses returned, and that access and masking controls are appropriate before sampling them into a dataset.