A water utility is deploying a GenAI work-order rewrite assistant and mistakenly sizes capacity as if it were their existing classical fraud-detection endpoint. Which statement best captures the deployment difference they must plan for?
Select an answer to reveal the explanation.
Short Explanation
Fraud models chew tidy rows of numbers; GenAI chews long prompts and giant weight files. If the utility sizes the rewrite assistant like the old fraud box, it will run out of memory and tokens long before the work orders finish rewriting.
Full Explanation
Foundation-model inference for a water-utility work-order rewrite assistant is driven by token throughput, context length, and model-weight memory in ways classic tabular fraud-detection endpoints usually are not. Classical fraud models often serve fixed-width feature vectors with relatively predictable CPU and memory profiles; GenAI rewrite assistants ingest long prompts such as work-order text, codes, and history and depend on large resident model weights plus token-rate capacity. Sizing the GenAI assistant like the old fraud box therefore underestimates memory, latency variance, and concurrency limits the utility will hit during storm or main-break ticket floods. Claiming GenAI endpoints never consume GPU resources so classic CPU autoscaling transfers unchanged fails because many FM deployments use accelerators or still need memory- and token-aware scaling rules that CPU-only classical heuristics miss. Claiming classical ML and GenAI share identical capacity metrics so only IAM needs redesign fails—the capacity model itself differs even when IAM is also redesigned. Claiming GenAI eliminates timeouts because foundation models always complete instantly fails operationally: generation can be slow under load, and utilities still need timeouts, queues, and UX handling for long rewrites. Exam caveat: not every classical endpoint is tiny and not every GenAI path needs a giant GPU, but the planning axes differ—tokens, context, and weights versus fixed-feature inference. Also separate online rewrite assistants from batch summarization when setting SLOs. Operational check: Profile prompt token distributions from real work orders; size context limits and concurrency from those distributions; set autoscaling on GenAI-relevant signals such as queue depth, token throughput, and GPU or memory—not only on the fraud endpoint’s old CPU rule; and load-test storm-week volumes before cutover.