A city's 311 program piloted a generative AI chatbot to answer resident questions, then compared it against a simple rule-based FAQ bot handling the same top request types. The rule-based bot matched the chatbot's resolution rate at a fraction of the licensing and compute cost. What is the most defensible next step for the 311 program's leadership?
Select an answer to reveal the explanation.
Short Explanation
Picture choosing between a bike and a car for a commute the bike already handles just fine — if the cheap option gets you there just as reliably, the car's extra horsepower isn't buying you anything. That's exactly what this pilot found: the rule-based bot ties the chatbot on results for a lot less money. When the evidence says terminate, reinvesting elsewhere beats tuning a system that was never the differentiator.
Full Explanation
A pilot exists to produce comparative evidence, and here that evidence is conclusive: a much cheaper, simpler tool matches the AI system on the metric that matters, resident resolution rate. That is the textbook signal to terminate an AI initiative rather than scale or extend it, because value comes from the outcome delivered, not from how sophisticated the underlying technology is. Running both tools in parallel confuses keeping options open with creating value; it just preserves the losing cost structure without a plan to retire it. Extending the evaluation window assumes the comparison was inconclusive, but a matched resolution rate across the tested period is itself a clear result, not a reason to stall. Continued prompt tuning assumes the performance gap is both closeable and worth closing, yet nothing in the pilot showed that shrinking a gap that does not currently exist would justify the AI system's added licensing and compute cost. A useful scope caveat: this call holds only while resident questions stay narrow enough for deterministic rules to cover well; if inquiries grow more varied, the comparison should be rerun. Before finalizing, leadership should confirm the cost comparison used fully loaded figures, including support and maintenance, not just sticker license price.