A county must compare ordinance QA quality and latency on on-prem DGX versus cloud GPUs. What approach is correct?
Select an answer to reveal the explanation.
Short Explanation
Fair race, same course: identical ordinance QA harness and metrics on DGX and cloud so latency and exact-match deltas mean something. Changing the test mid-race just confuses the county.
Full Explanation
A county comparing ordinance QA quality and latency on on-prem DGX versus cloud GPUs must run the same ordinance QA harness with standardized metrics on both platforms. Identical prompts, batching policy, decoding settings, and scorers (for example exact-match plus p50/p95 latency) make platform deltas meaningful. That is how municipal IT decides hosting without confounding hardware with a changing test.
Changing prompts and metrics freely on each platform to maximize scores fails by concept: you optimize the exam to the machine instead of measuring the machine on a fixed exam. Benchmarking only marketing FLOPS claims without task metrics fails because peak FLOPS do not equal ordinance-answer quality or real serving latency. Evaluating DGX with exact-match and cloud with unrelated sentiment scores fails because mismatched task scores cannot be compared across hosts.
Exam caveat: hold model weights, tokenizer, and adapter set constant across platforms so you isolate infrastructure, not an accidental checkpoint drift. Operational check: pin a harness version and metric config, run it on DGX and cloud under agreed load, publish side-by-side exact-match and latency tables, and choose the platform only after the standardized civic SLA comparison.