A community makerspace desk allowed 2,000-token answers and now prints tool novels nobody asked for, which also inflates the bill. Which inference control should the team apply?
Select an answer to reveal the explanation.
Short Explanation
Think of tool questions that turned into 2,000-token novels. Cap maximum generation length. Temperature is not a page limit, and unbounded replies are how the bill blew up.
Full Explanation
Maximum output length is both a user-experience and a spend control. Capping generation to the desired reply size stops tool novels and the extra tokens. Temperature is not a length cap, and unbounded generation is what created the problem.