A developer is building a chat interface on top of a Foundry-hosted GPT model. User testing shows that although the model typically finishes a full response in about six seconds, people perceive the assistant as slow and sometimes leave before the answer appears, because the message stays blank until the entire response is ready. Which change would most directly address this perception without changing the model or the total generation time?
Select an answer to reveal the explanation.
Short Explanation
The real problem is that users stare at a blank screen for six seconds and assume it's broken. Switch to streaming so tokens appear as they're generated, making the same six-second response feel responsive without changing the model or total generation time.
Full Explanation
Streaming changes how the response is delivered, sending tokens to the client as soon as the model generates them, so the user sees the answer building word by word within a fraction of a second rather than staring at a blank screen for six seconds; this directly fixes the perceived-latency problem without shortening the response or altering how fast the model itself works. Reducing max_tokens is incorrect because it shortens the content of the answer, potentially cutting off useful information, and it still leaves the user waiting for a full block of text to appear at once, just a smaller one. Adding a caching layer is incorrect because it only helps when a question has been asked before and the exact response can be reused; it does nothing for the large share of unique customer questions that make up normal chat traffic. Upgrading to a provisioned throughput tier is incorrect because it is about reserving consistent capacity to handle a predictable request volume reliably, an infrastructure and cost decision, not a mechanism for displaying partial output sooner; it would not change when the first visible text appears to the user.