Exercise 7.2

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

\hat{g} = arg min_g(\sum_{i=1}^{n} (y_i - g(x_i))^2 + \lambda \int(g^{(m)}(x))^2 dx)

arg min_g is the value of g that minimizes the function.

(a)

When \lambda = \infty the first term loses significance and can be ignored. If g^{(0)}(x) = g(x) = 0 the function will be minimized, which means that \hat{g} must be 0.

# Example sketch
x = np.arange(0,10,1)
y = np.full(10,0,dtype='int')

plt.plot(x,y,'-r');

png

(b)

When \lambda = \infty the first term loses significance and can be ignored. If g^{(1)}(x) = g'(x) = c the function will be minimized because the first derivative of a constant is 0. This means that \hat{g} must be an horizontal line.

# Example sketch
# We used c=5 but it could have been done with any other c.
x = np.arange(0,10,1)
y = np.full(10,5,dtype='int')

plt.plot(x,y,'-r');

png

(c)

When \lambda = \infty the first term loses significance and can be ignored. If g^{(2)}(x) = g''(x) = bx + c the function will be minimized because the second derivative of a linear function is 0. This means that \hat{g} must be a linear function.

# Example sketch
# We used y=x but it could have been done with any other linear function.
x = np.arange(0,10,1)
y = np.arange(0,10,1)

plt.plot(x,y,'-r');

png

(d)

When \lambda = \infty the first term loses significance and can be ignored. If g^{(3)}(x) = g'''(x) = ax^2 + bx + c the function will be minimized because the third derivative of a quadratic function is 0. This means that \hat{g} must be a quadratic function.

# Example sketch
# We used y=x^2 but it could have been done with any other quadratic function.
x = np.arange(0,10,1)
y = np.arange(0,10,1)**2

plt.plot(x,y,'-r');

png

(e)

This situation corresponds to a linear regression least squares fit. If \lambda = 0, the second term loses significance and can be ignored. Therefore, the function will be minimized when \sum_{i=1}^{n} (y_i - g(x_i)^2 is minimum. This means that \hat{g} must be such that it interpolates all of the y_i.

Since there are many different shapes that can express this situation, we didn't draw any example sketch.