An enterprise is rolling out a real-time computer vision application across hundreds of physical retail locations to analyze customer traffic patterns. The design demands minimal latency for video inference, optimal sharing of local GPU resources, and centralized management of model updates. The infrastructure utilizes NVIDIA edge hardware and cloud environments. Which combination of NVIDIA technologies provides the best architecture to fulfill these deployment goals?
Select an answer to reveal the explanation.
Short Explanation
You need to process video right at the edge, so Triton Inference Server on edge nodes gives you low latency and efficient GPU sharing. Then you use NGC as the central registry to manage and update models across hundreds of stores. That combo keeps inference fast and operations centralized.
Full Explanation
Deploying computer vision models at scale across multiple physical locations requires balancing latency, bandwidth, and manageability. For low latency, video frames must be processed near the camera source at the edge to avoid the network delays and massive egress/ingress costs of streaming raw high-definition video to a centralized cloud. The best architecture uses NVIDIA Triton Inference Server on edge nodes for local video processing and NVIDIA NGC for model registry and management. Triton is an open-source inference serving software that optimizes GPU utilization through features like dynamic batching, concurrent model execution, and model pipelining. It supports multiple frameworks, including PyTorch, TensorFlow, TensorRT, and ONNX, and runs efficiently on edge hardware such as Jetson or small GPU-equipped edge servers. To manage these distributed deployments, NVIDIA NGC serves as a centralized hub providing containerized software, pre-trained AI models, and a model registry, facilitating seamless deployment, versioning, and updating of models across the edge infrastructure.
The wrong choices fail for specific reasons. Setting up NVIDIA RAPIDS on local servers to process video frames and write structured data to a local database is not the best fit because RAPIDS is a suite of software libraries for executing end-to-end data science and analytics pipelines on GPUs, similar to pandas or scikit-learn, and it is not designed as a real-time model inference server. Developing custom inference engines using raw CUDA APIs and deploying them directly on general-purpose cloud virtual machines is high-maintenance, lacks built-in model serving optimizations such as dynamic batching, and sending video to the cloud introduces severe latency and bandwidth bottlenecks. Centralizing all video streams onto a cluster of high-end NVIDIA DGX systems located in a remote data center without deploying edge hardware bypasses edge computing entirely. While DGX systems are incredibly powerful, streaming hundreds of real-time video feeds over the WAN to a single data center introduces a severe latency penalty and unsustainable network bandwidth requirements.