Crane-maintenance log entries at Meridian's terminal are short, telegraphic lines like 'Crane C-14 hydraulic pressure low, replaced seal, cleared 08:42.' A data engineer is deciding how much character overlap to configure between chunks for this source. What is the most defensible choice?
Select an answer to reveal the explanation.
Short Explanation
Overlap exists to keep an idea from getting severed mid-thought across a chunk boundary - but these maintenance lines are already tiny, self-contained facts, so there's nothing straddling a boundary to protect, and copying a fat overlap from the contract pipeline just wastes embedding budget on duplicate text.
Full Explanation
A small or zero overlap is correct because chunk overlap is a mitigation for content that gets severed across a boundary, and telegraphic maintenance-log lines are already short, complete, self-contained statements - there is rarely meaningful context split across two consecutive log entries the way there is across two sentences of a legal clause. Copying the large overlap used for charter contracts is wrong because that overlap value was chosen for long, discursive legal prose where an idea can span hundreds of characters; applying the same setting to one-line log entries mostly duplicates near-identical short strings across chunks, inflating the index and diluting retrieval precision without protecting anything. Setting overlap equal to the full previous chunk is wrong because that makes every chunk redundant with the one before it, doubling storage and embedding cost for zero retrieval benefit. Claiming overlap is fixed by the embedding model is wrong because chunk overlap is a data-preparation parameter set before text ever reaches the embedding model - the embedding model has no say in how the source text was split. The exam-relevant point is that chunking parameters are chosen per source-document shape, not applied uniformly across every corpus in the pipeline.