311 leadership wants sub-two-second answers, but lowering max tokens truncates complete fee explanations. What trade-off should the tuning decision reflect?
Select an answer to reveal the explanation.
Short Explanation
It’s like packing a lunch—you want enough food, but the bag still has to zip. Tune max tokens so fee explanations stay complete without blowing the sub-two-second promise. Endless tokens or one-word stubs both fail residents.
Full Explanation
Parameter tuning for civic agents must negotiate latency SLOs against answer completeness for fee schedules and similar authoritative explanations. When 311 leadership wants sub-two-second answers but lowering max tokens truncates complete fee explanations, the correct trade-off is balancing generation length so fee answers stay complete enough without routinely breaking the latency target. Municipal callers need both timely replies and enough detail to pay the right amount; tuning should reflect that dual constraint with measured token budgets, caching, or retrieval of structured fee snippets rather than unbounded generation.
Always maximizing tokens regardless of the city’s latency SLO systematically violates the published performance goal and drives abandonment. Dropping fee explanations entirely so every reply fits in one word sacrifices the authoritative content residents need. Ignoring latency complaints because “accuracy never conflicts with speed” denies a real production trade-off that operators must manage explicitly.
Exam caveat: decoding and length parameters are service-level decisions—completeness and latency both matter for civic fee answers; the exam favors balanced limits over absolute maximization of either axis. Operational check: set candidate max-token values, run a fee-question eval set under load, confirm p95 latency stays under the SLO while fee fields (amount, due date, method) remain present in a sampled output review, then lock the chosen budget in config with a regression test.