Latency must be broken down across embedding, retrieval, and generation for the housing RAG path. How should instrumentation look?

Housing RAG latency must be broken down across embedding, retrieval, and generation, which means creating OpenTelemetry custom spans that mark those stages separately under the parent request span. Distinct spans (and related metrics) let municipal operators see whether slow housing answers come from embedding calls, vector/search retrieval, or model generation—each with different remediation (scale embedding, tune indexes/RU, or adjust generation parameters). Black-box total time alone cannot drive that decision.

Emitting only a single black-box span covering the entire request with no stage detail fails diagnosis: you know the housing path is slow but not which station. Skipping instrumentation because RAG stages are ‘too short to measure’ fails empirically—those stages often dominate tail latency under load and still need spans or timing metrics. Recording metrics only for disk I/O and ignoring model and retrieval timings fails relevance: RAG bottlenecks are typically remote model and data-plane calls, not local disk on the app host.

Exam caveat: custom spans should follow clear naming and attributes (stage name, model deployment, index name) without stuffing PII from housing applications into span attributes. Parent-child relationships matter so generation appears under the same resident request as embedding and retrieval. Traces complement—not replace—dependency telemetry from Azure SDKs when those SDKs already create useful child spans.

Operational check: run a housing RAG request, open the end-to-end trace, and verify separate spans for embedding, retrieval, and generation with durations that sum sensibly toward total latency; inject artificial delay in one stage and confirm only that span’s duration moves.