A county needs fast ordinance retrieval-only search under a tight VRAM budget. Which model-type choice fits?
Select an answer to reveal the explanation.
Short Explanation
You’re looking for pages, not asking a model to write an essay. A compact encoder bi-encoder is a lightweight librarian for ordinance search, so you can avoid spending scarce VRAM on a giant decoder at every lookup.
Full Explanation
For fast ordinance retrieval under a tight VRAM budget, prefer a compact encoder bi-encoder: it embeds documents and queries into a shared vector space, making similarity search inexpensive without autoregressive decoding at each lookup. Deploying a large decoder generative stack for every similarity lookup spends compute and memory on ranking rather than drafting. Running an encoder-decoder translation model when no generation is required adds cross-attention and a decoding path that retrieval does not use. Serving only an untrained random embedding table with no encoder yields meaningless vectors, so its low VRAM use cannot deliver reliable civic search. Exam caveat: bi-encoders work well for first-stage retrieval, but nuanced query–passage comparisons may benefit from a small cross-encoder reranker afterward; that does not require replacing the compact first stage with a full generative model. Practical check: benchmark embedding latency, peak VRAM, and recall@k on labeled ordinance queries against a decoder baseline, and use the encoder path only if it stays within the county’s VRAM ceiling while meeting the retrieval SLA.