Exercise 7.2

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

$\hat{g} = arg$ $min_g(\sum_{i=1}^{n} (y_i - g(x_i))^2 + \lambda \int(g^{(m)}(x))^2 dx)$

$arg$ $min_g$ is the value of g that minimizes the function.

(a)

When $\lambda = \infty$ the first term loses significance and can be ignored. If $g^{(0)}(x) = g(x) = 0$ the function will be minimized, which means that $\hat{g}$ must be 0.

# Example sketch
x = np.arange(0,10,1)
y = np.full(10,0,dtype='int')

plt.plot(x,y,'-r');

png

(b)

When $\lambda = \infty$ the first term loses significance and can be ignored. If $g^{(1)}(x) = g'(x) = c$ the function will be minimized because the first derivative of a constant is 0. This means that $\hat{g}$ must be an horizontal line.

# Example sketch
# We used c=5 but it could have been done with any other c.
x = np.arange(0,10,1)
y = np.full(10,5,dtype='int')

plt.plot(x,y,'-r');

png

(c)

When $\lambda = \infty$ the first term loses significance and can be ignored. If $g^{(2)}(x) = g''(x) = bx + c$ the function will be minimized because the second derivative of a linear function is 0. This means that $\hat{g}$ must be a linear function.

# Example sketch
# We used y=x but it could have been done with any other linear function.
x = np.arange(0,10,1)
y = np.arange(0,10,1)

plt.plot(x,y,'-r');

png

(d)

When $\lambda = \infty$ the first term loses significance and can be ignored. If $g^{(3)}(x) = g'''(x) = ax^2 + bx + c$ the function will be minimized because the third derivative of a quadratic function is 0. This means that $\hat{g}$ must be a quadratic function.

# Example sketch
# We used y=x^2 but it could have been done with any other quadratic function.
x = np.arange(0,10,1)
y = np.arange(0,10,1)**2

plt.plot(x,y,'-r');

png

(e)

This situation corresponds to a linear regression least squares fit. If $\lambda = 0$ , the second term loses significance and can be ignored. Therefore, the function will be minimized when $\sum_{i=1}^{n} (y_i - g(x_i)^2$ is minimum. This means that $\hat{g}$ must be such that it interpolates all of the $y_i$ .

Since there are many different shapes that can express this situation, we didn't draw any example sketch.