A city data center is specifying a node that will run RAG embedding pipelines alongside LLM serving. What compute evaluation should guide the CPU-to-GPU ratio?

City data-center nodes that combine RAG embedding pipelines with LLM serving stress both host CPUs and GPUs. Balancing CPU cores and host memory against GPU count works because embeddings, tokenization, retrieval orchestration, and request scheduling are CPU- and memory-heavy prep work that must keep accelerators fed while serving remains responsive. Evaluating AI compute deployment for mixed civic workloads means sizing the ratio so neither role becomes the permanent bottleneck.

Balancing CPU cores and host memory against GPU count so ingestion and index sharding keep pace with GPUs while favoring batch build speed over serving latency fails because it optimizes offline build throughput at the expense of the live objective; fast index construction does not help if LLM serving becomes sluggish under peak query load. Balancing CPU cores and host memory against GPU count so embedding and tokenization overlap inference while staging LLM weights in host memory for failover fails because host memory on this node must hold active preprocessing state to keep GPUs fed, not standby weight copies; failover staging does not sustain throughput or keep serving responsive. Balancing CPU cores and host memory against GPU count so orchestration and retrieval overlap inference while queuing token generation behind embedding batches fails because it lets background embedding block foreground serving; token generation must proceed concurrently with fair scheduling, not wait behind bulk batches. Operational check: profile host CPU, memory bandwidth, and GPU SM/utilization together under peak citizen query load; if GPUs idle while CPUs peg, add host capacity or split roles before adding more GPUs.