A water utility wants Databricks/Spark diagnostic logs queryable alongside other Azure ops signals. What should they implement?

A water utility that wants Databricks/Spark diagnostic logs queryable alongside other Azure ops signals should stream those diagnostic logs to Log Analytics in Azure Monitor. Centralized Log Analytics lets plant and platform operators correlate Spark driver and executor diagnostics with Azure resource metrics, alerts, and incident timelines using a common query plane. Streaming Databricks/Spark diagnostic logs to Azure Blob Storage for joint queries fails because Blob Storage is object storage without the Log Analytics query plane and cannot support correlated KQL queries alongside other Azure ops signals during plant incidents. Streaming Databricks/Spark diagnostic logs to Application Insights in Azure Monitor fails because Application Insights is an APM store for application traces, not the common Log Analytics query plane where Databricks diagnostics must sit alongside Azure resource metrics and alerts. Streaming Databricks/Spark diagnostic logs to Azure Managed Prometheus for analysis fails because Prometheus is a metrics store for time-series data, not diagnostic logs, and cannot be searched together with Azure Monitor logs and alerts during an incident. Exam caveat: DP-750 monitoring guidance points to Azure Monitor and Log Analytics integration for Databricks diagnostics, not Blob Storage streams, Application Insights, or Managed Prometheus. Operational check: enable diagnostic settings to Log Analytics, confirm Databricks/Spark tables populate, and run a sample query joining job failures with related Azure signals during a utility incident drill.