Exercise 2.5

A very flexible approach (versus a less flexible)

Advantages

• Less bias.
• Given enough data, better results.

Disadvantages

• More risk of overfitting.
• Harder to train.
• Longer to train.
• Computational more demanding.
• Less clear interpretability.

When is one approach preferable?

More flexible

• Large sample size and small number of predictors.
• Non-linear relationship between the predictors and response.

Less flexible

• Small sample size and large number of predictors.
• High variance of the error terms.