How can administrators monitor running Spark jobs in IBM watsonx.data?
Select an answer to reveal the explanation.
Short Explanation and Infographic
The Spark History Server and Spark UI provide detailed metrics for running and completed Spark jobs — stages; tasks; executor memory; shuffle data — accessible through the watsonx.data Spark engine console.
Full explanation below image
Full Explanation
The Spark History Server and Spark UI provide detailed metrics for running and completed Spark jobs — stages; tasks; executor memory; shuffle data — accessible through the watsonx.data Spark engine console. The incorrect options ("Through the Presto query history UI which covers all engine types", "By reading log files directly from the IBM Cloud Object Storage bucket", "Using IBM Cloud Monitoring service only; no built-in Spark monitoring is available") are distractors that don't fully capture the concept described.