A Meridian Line summarizer returns safety-incident records as JSON. For long narratives the response is valid at the start but ends abruptly mid-string, and json.loads raises an error. The prompt and schema are correct and temperature is 0. What is the most likely cause?
Select an answer to reveal the explanation.
Short Explanation
Cut off mid-string with a clean start is the signature of running out of output budget, not of a bad prompt. max_tokens is a hard stop — the model does not get to finish the sentence. Raise it, or ask for less per call.
Full Explanation
The described symptom is diagnostic: well-formed output that terminates abruptly rather than degrading in quality points at a length cap, and max_tokens is a hard ceiling on generated tokens that halts decoding wherever it lands, including in the middle of a JSON string. The fixes are to raise max_tokens to cover the worst-case response, to reduce what each call must emit by chunking the narrative or requesting fewer fields per call, and to check the response's finish reason in code so a truncated generation is detected rather than parsed. The context-window distractor is the closest trap because both are token limits and both produce broken results, but an input that exceeds the context window generally raises an error or drops input rather than producing a clean truncated tail on the output side. Temperature zero has no early-termination behaviour; it changes token selection, not stopping. Schema size can degrade adherence at extremes, but the failure mode there is wrong or missing keys spread through the response, not a clean cut in the final string. For the exam, remember that max_tokens is subtracted from the shared context budget, so an oversized value on a long-input call can itself cause a context error — size it to the expected response, not to the maximum the model allows.