Open-data educators show that LLMs predict likely next words—they do not “know” city policy like a clerk does. What conceptual model of LLMs is taught?

Open-data educators teach a core conceptual model: LLMs generate text by predicting likely next tokens from patterns in training data and prompts. That statistical generation differs from a municipal clerk’s accountable policy knowledge, where a person can cite the controlling rule, apply discretion under authority, and accept responsibility for the answer. Fluency from next-token prediction is not the same as certified knowledge of city policy, and treating it as such creates governance failures.

LLMs do not mimic a clerk’s policy knowledge via vector retrieval; vector similarity scores help predict likely next tokens from patterns, not replicate a clerk’s ability to cite controlling authority, exercise discretion, and accept responsibility. They do not map a clerk’s accountable logic to attention layers; attention layers are numerical weighting operations that guide next-token prediction, with no representation of legal authority, discretion, or responsibility for the answer. They do not extract accountable policy knowledge from semantic weights; weights encode statistical word-pattern associations from training, not certified city policy a clerk can cite and stand behind. Those distractors anthropomorphize or over-credential the model.

Exam caveat: retrieval grounding and tools can supply current policy text, but the generator can still mis-read or overgeneralize retrieved material. Operational check: require citations to the controlling policy for public answers, route high-stakes questions to clerks, and train staff to explain LLMs as probabilistic generators rather than unofficial deputy clerks.