When configuring Databricks Model Serving, when is GPU serving MOST…

Question

When configuring Databricks Model Serving, when is GPU serving MOST justified?

Accepted Answer

For large deep learning models where GPU latency/throughput gains outweigh cost — GPUs for big neural nets at serving time — not for tiny sklearn logits.

Answer

For tiny sklearn logistic regression with 5 features always

Answer

For any SQL VIEW

Answer

Never; GPUs cannot serve models

When configuring Databricks Model Serving, when is GPU serving MOST justified?

Short Explanation and Infographic

Full Explanation