When training an XGBoost model on Databricks, which runtime environment is recommended to avoid manually installing XGBoost?
Select an answer to reveal the explanation.
Short Explanation and Infographic
DBR ML is the 'batteries included' version of the Databricks runtime — XGBoost, LightGBM, scikit-learn, TensorFlow, and PyTorch come pre-installed so you can start training immediately.
Full explanation below image
Full Explanation
Databricks Runtime for Machine Learning (e.g., 14.3 LTS ML) is a pre-configured cluster runtime that includes: XGBoost, LightGBM, scikit-learn, TensorFlow, PyTorch, Horovod, MLflow, and other popular ML libraries pre-installed and version-compatible. Using DBR ML avoids: library conflicts, installation time at cluster startup, and version incompatibility between frameworks. Key features of DBR ML: 1) Single-node and distributed ML support. 2) GPU support (DBR ML GPU variants). 3) Pre-tuned configurations for ML workloads. 4) Built-in MLflow integration. Standard DBR does NOT include XGBoost or most ML frameworks. DBR for Genomics is a specialized runtime for bioinformatics. Custom Docker containers work but require significant maintenance overhead compared to the managed DBR ML.