How does a Spark engine in watsonx.data access tables managed by the IBM Hive Metastore?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Spark connects to the Hive Metastore via the Hive Thrift protocol (SparkSession configured with hive.metastore.uris) to read table schemas; partition info; and data file locations.
Full explanation below image
Full Explanation
Spark connects to the Hive Metastore via the Hive Thrift protocol (SparkSession configured with hive.metastore.uris) to read table schemas; partition info; and data file locations. The incorrect options ("Spark reads metadata files directly from object storage without using the metastore", "Spark requires the table to be exported as a CSV before it can process it", "Spark uses a dedicated REST API separate from the Hive Metastore") are distractors that don't fully capture the concept described.