Which Spark MLlib transformer creates polynomial features and interaction terms from numeric features?
Select an answer to reveal the explanation.
Short Explanation and Infographic
PolynomialExpansion is Spark MLlib's built-in polynomial feature generator — set degree=2 and it outputs all squared terms and cross-products (x1², x2², x1×x2) from your feature vector.
Full explanation below image
![Infographic 1024x768 showing PolynomialExpansion: input vector [x1=2, x2=3] → PolynomialExpansion(degree=2) box → output vector [x1=2, x2=3, x1²=4, x1×x2=6, x2²=9]. Feature count explosion table: 2 features degree=2 → 5, degree=3 → 10; 5 features degree=2 → 20, degree=3 → 56. Warning label 'Feature explosion — use regularization'. Code snippet showing pipeline stage.](/media/databricks/dbml/DBML-q069-infographic.webp)
Full Explanation
pyspark.ml.feature.PolynomialExpansion(degree=2, inputCol='features', outputCol='poly_features') generates polynomial feature combinations. For 2 input features [x1, x2] with degree=2: output = [x1, x2, x1², x1×x2, x2²] — 5 features from 2. For degree=3 with 2 features: output has C(2+3, 3) = 10 features. Warning: polynomial expansion grows combinatorially — 10 features with degree=3 produces C(13,3)=286 features. Use cases: capturing non-linear relationships in linear models (LinearRegression + PolynomialExpansion can model curves). Typically paired with regularization (Ridge/Lasso) due to the feature explosion. FeatureInteraction, VectorExpander, and CrossFeatureTransformer are not valid Spark MLlib class names. Scikit-learn equivalent: sklearn.preprocessing.PolynomialFeatures.