The county assessor wants photo captions and deed snippets for the same parcel to land near each other in embedding space. What training signal should staff use?

When the county assessor wants photo captions and deed snippets for the same parcel to land near each other in embedding space, the training signal must explicitly pull matching pairs together and push mismatches apart. Contrastive loss on positive caption–deed pairs versus negatives creates that shared multimodal-style text embedding geometry: true parcel mates attract, hard negatives repel, and retrieval quality for civic parcel lookup improves. Staff can then search by caption and surface the right deed language (or the reverse) because both modalities were trained into one comparable space.

Cross-entropy over random parcel ID integers only fails by concept: classifying opaque IDs does not align caption and deed representations with each other—it learns discrete labels, not a shared retrieval space. Language-model next-token loss with no paired examples fails because next-token prediction alone does not enforce caption–deed proximity; unpaired LM training optimizes fluency, not cross-text embedding alignment. Beam-search decoding length penalties at inference time fail because they are decode heuristics after training; they never reshape the embedding space where caption and deed vectors live.

Exam caveat: contrastive civic datasets need careful negative mining—random negatives can be too easy, while same-neighborhood parcels can be hard negatives that actually teach discrimination. Operational check: embed a fixed parcel probe set before and after contrastive training, measure recall@k for caption→deed and deed→caption, and verify true pairs rank above cross-parcel negatives without collapsing all county embeddings to one vector.