The assessor similarity-search service must stay under a hard GPU memory ceiling. Which decision is correct?
Select an answer to reveal the explanation.
Short Explanation
Similarity search doesn’t need a novelist—it needs a tape measure. Under a hard VRAM ceiling, pick a compact encoder embedding model and leave the giant decoder novels on the shelf.
Full Explanation
Assessor similarity-search under a hard GPU memory ceiling should rule out a large decoder generative footprint and serve a compact encoder embedding model that produces vectors for ANN or dense retrieval. Pairwise similarities then become vector operations, not autoregressive generation, which is the right cost model for parcel and ordinance search. Deploying a large decoder solely to compute pairwise similarities wastes decode memory and compute against the ceiling and adds latency residents never needed for a lookup. Loading three full decoder checkpoints and picking one at random multiplies weight memory without improving embedding efficiency or consistency. Ignoring the ceiling and relying on host swap for model weights introduces thrashing latency that destroys interactive search SLAs. Exam caveat: compact encoders still need embedding dimension and index memory counted against the same operational budget; a huge on-GPU FAISS or IVF working set can break the ceiling even when the encoder itself is small. Operational check: measure peak GPU memory for encoder weights plus index working set on the assessor query mix, confirm it stays under the hard ceiling, and verify recall@k against a labeled parcel or ordinance similarity set before cutover.