GPU costs spike while the city runs off-peak permit batch jobs. How should tuning address the latency–cost–accuracy trade-off?
Select an answer to reveal the explanation.
Short Explanation
Think bulk printing after hours when the shop rate is cheaper. Size batches and concurrency for off-peak so the city spends less without shredding accuracy. Peak-only mega fleets, killing the jobs, or randomly dropping permits are budget theater, not tuning.
Full Explanation
Municipal inference tuning should adjust batch size and concurrency to exploit off-peak capacity while preserving the accuracy required for permit workloads. When GPU costs spike while the city runs off-peak permit batch jobs, the latency–cost–accuracy trade-off is managed by sizing batches and concurrency for those cheaper windows so spend drops without abandoning quality gates. Off-peak scheduling, measured throughput, and acceptance thresholds together keep permit processing both affordable and trustworthy.
Running every batch job on the largest GPU fleet at peak daytime rates only maximizes unit cost and contends with interactive resident traffic. Disabling all permit batch processing permanently to save money abandons a required municipal function. Randomly dropping half the permits from each batch to cut GPU minutes creates unfair, incomplete case handling and audit failures.
Exam caveat: cost optimization for civic batch inference is about scheduling and throughput parameters under accuracy constraints—not deleting work or running only the most expensive footprint. Operational check: compare GPU-minutes and accuracy metrics for the same permit batch at peak versus off-peak with tuned batch size/concurrency; confirm accuracy thresholds still pass; record the chosen off-peak profile and a kill-switch if quality regresses.