Model Optimization

NCP-GENL · 34 questions

• City hall’s older inference GPU runs out of memory on a 70B permit assistant. Which optimization approach best fits the hardware?
• A library kiosk must keep a summarizer interactive on a single workstation GPU. Which technique specifically targets activation memory?
• Transit edge boxes need sparsity that maps to Tensor Core acceleration for delay explainers. Which choice fits?
• Public-health overnight jobs want to prune rarely used FFN weights using clinic FAQ statistics. Which pruning approach matches that goal?
• Courts want smaller continuance-notice models by cutting unused attention capacity and lowering weight precision. What combination is appropriate?
• On A100 nodes, county IT wants a stable tax FAQ model quantized after training and must report quality change versus FP16. What strategy fits?
• A housing eligibility model lost too much minority-language accuracy under PTQ. Which quantization path should the team choose next?
• H100 Tensor Core profiling for emergency alert rewriting shows risk if INT8 is too aggressive. What precision choice aligns with GPU generation and task risk?
• Water quality classifiers will ship to field tablets. What should engineers compare before choosing INT8 scope?
• School grading-rubric assistants stay on FP16 after INT8 calibration sets prove too small for reliable PTQ. Why is that decision sound?
• Neighborhood kiosks cannot host a 70B city-code teacher. How should the city create a deployable model?
• Clinic waiting-room tablets need a compact summarizer. How can a large public-health teacher transfer knowledge effectively?
• Transit keeps a large teacher on-prem DGX but must run inference on bus-yard rugged PCs. What deployment split is correct after distillation?
• Courts distill a multilingual teacher so the smaller student must keep Spanish continuance phrasing. What should the distillation objective emphasize?
• Housing chat distillation should teach the student to cite policy IDs the way the larger teacher does. How should losses be designed?
• County ML runs a distributed learning-rate sweep for LoRA on ordinance QA. What is the primary goal of that search?
• 311 intent fine-tuning saturates GPU memory as batch size rises. What adjustment correctly continues effective batch growth?
• Parks captioning needs overnight trials of dropout and weight decay combinations. Which approach matches systematic tuning?
• Assessor deed summarization validation loss improves with warmup plus cosine decay versus a constant learning rate. What should the team select?
• Emergency management wants to lock classifier hyperparameters. What confirms the chosen settings are stable?
• After quantizing a city clerk letter model, staff need consistent legal boilerplate. How should they judge optimization impact on wording consistency?
• Transit chatbot latency improved after several optimizations. How should the team isolate each contribution?
• A public-health summarizer feels flat after distillation. What evaluation step helps recover diversity without factual drift?
• Housing eligibility answers got worse after pruning while staff also lowered temperature. How can they prove what caused the drop?
• Library catalog blurbs lost quality after sparsity was applied. What ablation helps test whether decoding can recover quality?
• Residents ask follow-up ordinance questions in one long municipal code session. Which optimization keeps those turns fast?
• City ML validated FP16 accuracy for a permitting assistant on A100 hardware. What optimization export fits next?
• Council-meeting transcript assistants must cover multi-hour sessions without paying full-context attention cost. Which method fits?
• Edge emergency kiosks lack GPU tactics needed for TensorRT rebuilds. What optimization choice matches the hardware?
• A water utility chatbot streams tokens to residents and needs low first-token latency. Which combination helps?
• County IT wants encoder weights that already understand property-record language before classifiers train. What pretraining approach fits?
• Clerk staff want an encoder backbone that links agenda items to matching minutes paragraphs. Which objective helps?
• After MLM pretraining on ordinances, the county needs a smaller encoder for embedding search. What conceptual next step fits?
• Public-health staff distill models for clinic FAQ chat versus for embedding search. What distinction should they keep clear?