CS 189 Discussion 7

$y_i=g(x_i)+ϵ$

$\hat{h}=argmi{n}_h{\mathbb{E}}_g[(h-g{)}^2]$ $\hat{h}=argmi{n}_h{\mathbb{E}}_g[(h(X)-y{)}^2]$ By law of large numbers,

$$argmi{n}_h=\frac{1}{n}\sum _{i=1}^n(h(x_i-y_i{)}^2)=RSS$$

---

Q1. Simple Bias-Variance Tradeoff

$Bias(\hat{X})=\mathbb{E}[\hat{X}]-\mu$

1. $\hat{X}=\frac{1}{n}{\sum }_{i=1}^n{x}_i=\mu$ Bias = 0

c. $E[(\hat{x}-x^{\prime }{)}^2]$ in terms of ${\sigma }^2$ and the bias and variance of the estimator $\hat{X}$ $E[(\hat{x}-x^{\prime }{)}^2]=E[{\hat{x}}^2-2\hat{x}{x}^{\prime }+x^{\prime 2}]$

$=E[{\hat{x}}^2]-2E[\hat{x}{x}^{\prime }]+E[x^{\prime 2}]$

$=Var(\hat{x})+E[x{]}^2-2E[\hat{x}{x}^{\prime }]+E[x^{\prime }{]}^2+Var(x^{\prime })$

$=E[\hat{x}{]}^2-2E[\hat{x}{x}^{\prime }]+{\mu }^2+Var(\hat{x})+{\theta }^2$

More n0 → bias becomes larger, Variance becomes smaller

if u → 0, add more zeros, shrink the estimate to the zero. if data ${\sigma }^2$ is high, (noisy data) → better to reduce variance, add more zeros.

What if the mean is also 0?