A city compares open-source versus commercial LLM hosting for a RAG assistant on municipal policies. What evaluation focus should guide the decision?

Evaluating hosting and model options against municipal requirements for control, cost, and support is the correct focus when comparing open-source versus commercial LLM hosting for a RAG assistant on municipal policies. Neither open source nor commercial SaaS is automatically correct for city hall; the decision depends on who operates the stack, where data resides, how patches and model updates are handled, total cost of ownership, vendor or community support expectations, and how tightly the city must control prompts, corpora, and logs. A structured evaluation against those requirements prevents ideology-driven picks that fail operations or budget later. The approach works because hosting and model choice are governance and operations decisions, not brand loyalty tests.

Always picking open source because it never needs security review fails by concept: open-source stacks still require security review, hardening, patching, and operational ownership. Always picking commercial SaaS because open source cannot serve government text fails by concept—open-source models can process government text when hosted and governed appropriately; the real issue is fit and control, not a categorical ban. Skipping hosting analysis and downloading the largest model onto a clerk laptop fails by concept and risk: unmanaged local hosting creates sprawl, weak backup, uneven security, and unsupported shadow AI.

Exam caveat: the exam tests requirement-driven evaluation of hosting options, not a mandate that cities must self-host or must buy SaaS. Keep municipal RAG / policy-assistant framing. Operational check: score open-source and commercial options on control (data path, admin rights), cost (licenses, GPUs, staff time), support (SLAs vs internal ops), and security review outcomes; pick the option that meets municipal requirements with an explicit owner for ongoing operations.