Emergency after-action PDFs will feed both retrieval and fine-tuning. What organization practice keeps those sets aligned?
Select an answer to reveal the explanation.
Short Explanation
If retrieval and fine-tuning can’t point at the same after-action memo, the left hand loses the right. Chunk with stable document IDs so both pipelines stay aligned.
Full Explanation
Emergency after-action PDFs that feed both retrieval and fine-tuning should be chunked while preserving stable document IDs shared by both sets. Shared IDs keep chunks traceable to the same source memo so retrieval evidence and fine-tune examples stay aligned for audits, refreshes, and deduplication. Organization with stable identifiers is what lets two municipal pipelines talk about the same incident record. Chunking anonymously with no IDs so sets stay unlinkable blocks exactly the alignment needed when retrieval and fine-tuning must stay consistent. Assigning a new random UUID to every chunk on each pipeline run breaks idempotency: re-ingesting the same PDF looks like new documents and fragments lineage. Dropping page order and shuffling sentences across incidents before storage destroys narrative structure and mixes evidence across emergencies. Exam caveat: stable IDs should identify source documents (and optionally chunk offsets); they are not a substitute for access control—after-action material may still need role-based redaction even when IDs align. Operational check: embed document_id and chunk_index in both the vector store and the fine-tune JSONL, re-run chunking twice and confirm IDs match, and spot-check that a retrieved chunk ID resolves to the same fine-tune source text.