How does Hyperopt with SparkTrials help hyperparameter tuning on Databricks?
Select an answer to reveal the explanation.
Short Explanation and Infographic
SparkTrials = parallel Hyperopt trials across the cluster, still log to MLflow.
Full explanation below image
Full Explanation
SparkTrials runs Hyperopt trials distributed on Spark executors, accelerating search. You still define a search space; MLflow is complementary for logging.