A clerk’s office must containerize a memory-heavy local helper model for an air-gapped evidence-review room where policy allows on-prem inference. Which deployment consideration is most distinctive for that LLM workload versus a typical lightweight web API?

Containerizing a memory-heavy local helper LLM for an air-gapped evidence-review room means planning capacity around model weights, accelerator or large-memory needs, and token and context working sets—not treating it like a thin REST microservice. The weights can dominate RAM or GPU capacity, and long document contexts increase runtime memory beyond the application code; start with model size and context needs, then set container limits and provide appropriately sized storage for the weights. Pinning the container to a single vCPU because foundation models rarely benefit from more cores is not a sound rule: inference stacks may need CPU for tokenization, orchestration, or non-GPU paths, while GPU or accelerator planning is often the primary capacity concern. Disabling all persistent volumes because LLM containers never need model-weight storage overlooks the large weights and caches that typically require suitable volumes or image layers. Estimating capacity from request rate and database connections as if local inference had a modest, steady CRUD-service footprint ignores weight and context memory, risking out-of-memory errors or unusable latency. Exam caveat: an air gap does not remove patch, model-version, or access-control requirements; approved media and gates govern how artifacts enter the room. Practical check: measure weight size and peak memory at the target context length using representative evidence PDFs, set memory and GPU requests above the peak with headroom, mount approved weight storage, and soak-test in the air-gapped environment before go-live.