Relying on a single accuracy number for LLM evals is risky because:

Question

Accepted Answer

Quality is multi-dimensional: factuality, safety, format, latency, cost, and UX matter — One accuracy number hides safety fails, format breaks, and latency blowups — score multiple axes.

Answer

Accuracy is illegal to measure

Answer

Models only have one behavior ever

Answer

Evals are only for images

Relying on a single accuracy number for LLM evals is risky because:

Short Explanation and Infographic

Full Explanation