A permit answer fails in production across orchestrator, NIM, and vector DB. What diagnostic approach helps most?
Select an answer to reveal the explanation.
Short Explanation
Three systems, one failed permit answer—you need a flight recorder, not a vibe check. Distributed traces across orchestrator, NIM, and vector DB show where it broke.
Full Explanation
A permit answer that fails across orchestrator, NIM, and vector DB needs a diagnostic approach that spans hops. Distributed traces that link orchestrator → NIM → vector DB for the failed permit answer provide that flight recorder: each span shows where latency exploded, where an error originated, and whether retrieval or generation failed. Multi-component civic stacks are otherwise opaque if you inspect only one layer.
Inspecting only the chat UI CSS for clues about backend failures fails because presentation styling does not explain inference or retrieval faults. Guessing which component failed based on the resident’s tone of voice fails as methodology—tone is not telemetry. Disabling tracing to reduce overhead and never inspecting cross-component paths fails the diagnostic mission; modest trace overhead is the price of debuggability in production municipal AI.
Exam caveat: traces show where and when a failure occurred; they may not show why a retrieved ordinance was wrong. Pair tracing with content-level eval (wrong chunk vs wrong generation) when the failure is semantic rather than an exception.
Operational check: reproduce the failed permit question in staging with tracing enabled, confirm spans stitch across orchestrator/NIM/vector DB, verify error attributes identify the failing service, and keep trace sampling high enough on the permit path to catch rare failures without drowning storage.