A very flexible approach (versus a less flexible)
- Less bias.
- Given enough data, better results.
- More risk of overfitting.
- Harder to train.
- Longer to train.
- Computational more demanding.
- Less clear interpretability.
When is one approach preferable?
- Large sample size and small number of predictors.
- Non-linear relationship between the predictors and response.
- Small sample size and large number of predictors.
- High variance of the error terms.