---
tags: ISLR
---
# ISLR hw6
Range: 「課本第三章習題：第2題、第14題、第15題、第四章習題：第3題、第5題」
## ch03 Q2
classifier search k nearest-neighbor "voted" to output the answer
regression search k nearest-neighbor "calculate mean" to output the anser
## Q14
```r
set.seed(1)
x1 <- runif(100)
x2 <- 0.5*x1 + rnorm(100) / 10
y <- 2 + 2*x1 + 0.3*x2 + rnorm(100)
```
### a
$y = \beta_0 + \beta_1 * x_1 + \beta_2 * x_2 + c$
Regression coefficients is $\beta_0 = 2$, $\beta_1 = 2$ and $\beta_2 = 0.3$
### b
```r
cor(x1, x2)
# [1] 0.8351212
```
### c
```r
x1x2_lm <- lm(y ~ x1 + x2)
summary(x1x2_lm)
# Call:
# lm(formula = y ~ x1 + x2)
# Residuals:
# Min 1Q Median 3Q Max
# -2.8311 -0.7273 -0.0537 0.6338 2.3359
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 2.1305 0.2319 9.188 7.61e-15 ***
# x1 1.4396 0.7212 1.996 0.0487 *
# x2 1.0097 1.1337 0.891 0.3754
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 1.056 on 97 degrees of freedom
# Multiple R-squared: 0.2088, Adjusted R-squared: 0.1925
# F-statistic: 12.8 on 2 and 97 DF, p-value: 1.164e-05
```
estimated $\beta_0$ is 2.1305, $\beta_1$ is 1.4396 and $\beta_2$ is 1.0097
some of they is close with true coefficients
We can reject null hypothesis of $\beta_1$
We cannot reject null hypothesis of $\beta_2$
### d
```r
x1_lm <- lm(y ~ x1)
summary(x1_lm)
# Call:
# lm(formula = y ~ x1)
# Residuals:
# Min 1Q Median 3Q Max
# -2.89495 -0.66874 -0.07785 0.59221 2.45560
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 2.1124 0.2307 9.155 8.27e-15 ***
# x1 1.9759 0.3963 4.986 2.66e-06 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 1.055 on 98 degrees of freedom
# Multiple R-squared: 0.2024, Adjusted R-squared: 0.1942
# F-statistic: 24.86 on 1 and 98 DF, p-value: 2.661e-06
```
Yes, because p of x1 coef is < 0.05
We can reject null hypothesis of $\beta_1$
### e
```r
x2_lm <- lm(y ~ x2)
summary(x2_lm)
# Call:
# lm(formula = y ~ x2)
# Residuals:
# Min 1Q Median 3Q Max
# -2.62687 -0.75156 -0.03598 0.72383 2.44890
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 2.3899 0.1949 12.26 < 2e-16 ***
# x2 2.8996 0.6330 4.58 1.37e-05 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 1.072 on 98 degrees of freedom
# Multiple R-squared: 0.1763, Adjusted R-squared: 0.1679
# F-statistic: 20.98 on 1 and 98 DF, p-value: 1.366e-05
```
Yes, because p of x2 coef is < 0.05
We can reject null hypothesis of $\beta_1$
### f
No. In c. and d. we notice there are high correlation between x1 and x2.
d. and e. show that x1 and x2 have statistically evidence to reject null hypothesis
c. shows that x1 has more confidence in statistic than x2
### g
```r
x1 <- c(x1, 0.1)
x2 <- c(x2, 0.8)
y <- c(y,6)
```
```r
# c plot
x1x2_lm = lm(y ~ x1 + x2)
summary(x1x2_lm)
# Call:
# lm(formula = y ~ x1 + x2)
# Residuals:
# Min 1Q Median 3Q Max
# -2.73348 -0.69318 -0.05263 0.66385 2.30619
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 2.2267 0.2314 9.624 7.91e-16 ***
# x1 0.5394 0.5922 0.911 0.36458
# x2 2.5146 0.8977 2.801 0.00614 **
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 1.075 on 98 degrees of freedom
# Multiple R-squared: 0.2188, Adjusted R-squared: 0.2029
# F-statistic: 13.72 on 2 and 98 DF, p-value: 5.564e-06
```
Now x2 is significant, x1 is not.
d plot and e plot keep the same(skip)
replot the `plot(x1, x2)`
run `plot(x1x2_lm)`
Yes, it looks like outlier and high-leverage point.
## Q15
### a
```r
colnames(Boston)
lapply(colnames(Boston), FUN = function(p) summary(lm(Boston$crim ~ Boston[,p])))
# [1] "crim" "zn" "indus" "chas" "nox" "rm" "age" "dis" "rad"
# [10] "tax" "ptratio" "black" "lstat" "medv"
# > lapply(colnames(Boston), FUN = function(p) summary(lm(Boston$crim ~ Boston[,p])))
# [[1]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -3.074e-15 -2.120e-16 -7.000e-17 7.200e-17 4.023e-14
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 1.263e-15 8.895e-17 1.420e+01 <2e-16 ***
# Boston[, p] 1.000e+00 9.542e-18 1.048e+17 <2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 1.844e-15 on 504 degrees of freedom
# Multiple R-squared: 1, Adjusted R-squared: 1
# F-statistic: 1.098e+34 on 1 and 504 DF, p-value: < 2.2e-16
# [[2]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -4.429 -4.222 -2.620 1.250 84.523
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 4.45369 0.41722 10.675 < 2e-16 ***
# Boston[, p] -0.07393 0.01609 -4.594 5.51e-06 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.435 on 504 degrees of freedom
# Multiple R-squared: 0.04019, Adjusted R-squared: 0.03828
# F-statistic: 21.1 on 1 and 504 DF, p-value: 5.506e-06
# [[3]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -11.972 -2.698 -0.736 0.712 81.813
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -2.06374 0.66723 -3.093 0.00209 **
# Boston[, p] 0.50978 0.05102 9.991 < 2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.866 on 504 degrees of freedom
# Multiple R-squared: 0.1653, Adjusted R-squared: 0.1637
# F-statistic: 99.82 on 1 and 504 DF, p-value: < 2.2e-16
# [[4]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -3.738 -3.661 -3.435 0.018 85.232
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 3.7444 0.3961 9.453 <2e-16 ***
# Boston[, p] -1.8928 1.5061 -1.257 0.209
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.597 on 504 degrees of freedom
# Multiple R-squared: 0.003124, Adjusted R-squared: 0.001146
# F-statistic: 1.579 on 1 and 504 DF, p-value: 0.2094
# [[5]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -12.371 -2.738 -0.974 0.559 81.728
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -13.720 1.699 -8.073 5.08e-15 ***
# Boston[, p] 31.249 2.999 10.419 < 2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.81 on 504 degrees of freedom
# Multiple R-squared: 0.1772, Adjusted R-squared: 0.1756
# F-statistic: 108.6 on 1 and 504 DF, p-value: < 2.2e-16
# [[6]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -6.604 -3.952 -2.654 0.989 87.197
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 20.482 3.365 6.088 2.27e-09 ***
# Boston[, p] -2.684 0.532 -5.045 6.35e-07 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.401 on 504 degrees of freedom
# Multiple R-squared: 0.04807, Adjusted R-squared: 0.04618
# F-statistic: 25.45 on 1 and 504 DF, p-value: 6.347e-07
# [[7]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -6.789 -4.257 -1.230 1.527 82.849
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -3.77791 0.94398 -4.002 7.22e-05 ***
# Boston[, p] 0.10779 0.01274 8.463 2.85e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.057 on 504 degrees of freedom
# Multiple R-squared: 0.1244, Adjusted R-squared: 0.1227
# F-statistic: 71.62 on 1 and 504 DF, p-value: 2.855e-16
# [[8]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -6.708 -4.134 -1.527 1.516 81.674
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 9.4993 0.7304 13.006 <2e-16 ***
# Boston[, p] -1.5509 0.1683 -9.213 <2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.965 on 504 degrees of freedom
# Multiple R-squared: 0.1441, Adjusted R-squared: 0.1425
# F-statistic: 84.89 on 1 and 504 DF, p-value: < 2.2e-16
# [[9]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -10.164 -1.381 -0.141 0.660 76.433
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -2.28716 0.44348 -5.157 3.61e-07 ***
# Boston[, p] 0.61791 0.03433 17.998 < 2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 6.718 on 504 degrees of freedom
# Multiple R-squared: 0.3913, Adjusted R-squared: 0.39
# F-statistic: 323.9 on 1 and 504 DF, p-value: < 2.2e-16
# [[10]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -12.513 -2.738 -0.194 1.065 77.696
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -8.528369 0.815809 -10.45 <2e-16 ***
# Boston[, p] 0.029742 0.001847 16.10 <2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 6.997 on 504 degrees of freedom
# Multiple R-squared: 0.3396, Adjusted R-squared: 0.3383
# F-statistic: 259.2 on 1 and 504 DF, p-value: < 2.2e-16
# [[11]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -7.654 -3.985 -1.912 1.825 83.353
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -17.6469 3.1473 -5.607 3.40e-08 ***
# Boston[, p] 1.1520 0.1694 6.801 2.94e-11 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.24 on 504 degrees of freedom
# Multiple R-squared: 0.08407, Adjusted R-squared: 0.08225
# F-statistic: 46.26 on 1 and 504 DF, p-value: 2.943e-11
# [[12]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -13.756 -2.299 -2.095 -1.296 86.822
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 16.553529 1.425903 11.609 <2e-16 ***
# Boston[, p] -0.036280 0.003873 -9.367 <2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.946 on 504 degrees of freedom
# Multiple R-squared: 0.1483, Adjusted R-squared: 0.1466
# F-statistic: 87.74 on 1 and 504 DF, p-value: < 2.2e-16
# [[13]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -13.925 -2.822 -0.664 1.079 82.862
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -3.33054 0.69376 -4.801 2.09e-06 ***
# Boston[, p] 0.54880 0.04776 11.491 < 2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.664 on 504 degrees of freedom
# Multiple R-squared: 0.2076, Adjusted R-squared: 0.206
# F-statistic: 132 on 1 and 504 DF, p-value: < 2.2e-16
# [[14]]
# Call:
# lm(formula = Boston$crim ~ Boston[, p])
# Residuals:
# Min 1Q Median 3Q Max
# -9.071 -4.022 -2.343 1.298 80.957
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 11.79654 0.93419 12.63 <2e-16 ***
# Boston[, p] -0.36316 0.03839 -9.46 <2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.934 on 504 degrees of freedom
# Multiple R-squared: 0.1508, Adjusted R-squared: 0.1491
# F-statistic: 89.49 on 1 and 504 DF, p-value: < 2.2e-16
```
all parameters except chas has significant with crime.rate
### b
```r
summary(lm(crim ~ ., data = Boston))
# Call:
# lm(formula = crim ~ ., data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -9.924 -2.120 -0.353 1.019 75.051
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 17.033228 7.234903 2.354 0.018949 *
# zn 0.044855 0.018734 2.394 0.017025 *
# indus -0.063855 0.083407 -0.766 0.444294
# chas -0.749134 1.180147 -0.635 0.525867
# nox -10.313535 5.275536 -1.955 0.051152 .
# rm 0.430131 0.612830 0.702 0.483089
# age 0.001452 0.017925 0.081 0.935488
# dis -0.987176 0.281817 -3.503 0.000502 ***
# rad 0.588209 0.088049 6.680 6.46e-11 ***
# tax -0.003780 0.005156 -0.733 0.463793
# ptratio -0.271081 0.186450 -1.454 0.146611
# black -0.007538 0.003673 -2.052 0.040702 *
# lstat 0.126211 0.075725 1.667 0.096208 .
# medv -0.198887 0.060516 -3.287 0.001087 **
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 6.439 on 492 degrees of freedom
# Multiple R-squared: 0.454, Adjusted R-squared: 0.4396
# F-statistic: 31.47 on 13 and 492 DF, p-value: < 2.2e-16
```
We can reject null hypothesis with zn, dis, rad, black and medv
### c
```r
uni_lm <- lapply(colnames(Boston)[-1], FUN = function(p) lm(Boston$crim ~ Boston[,p])$coefficients[2])
mul_lm <- lm(crim ~ ., data = Boston)$coefficients[-1] # remove beta_0
plot(uni_lm, mul_lm)
```
### d
```r
lm_formula <- function(p) {
crim_formula <- as.formula(paste("crim ~ ", p, " + I(", p, "^2)", " + I(", p, "^3)"))
return(summary(lm(crim_formula, data = Boston)))
}
lapply(colnames(Boston), FUN = lm_formula)
# [[1]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -12.420 -1.403 -1.249 1.071 13.763
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 1.4813187 0.1170849 12.65 <2e-16 ***
# I(crim^2) 0.0421933 0.0009366 45.05 <2e-16 ***
# I(crim^3) -0.0003695 0.0000122 -30.30 <2e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 2.519 on 503 degrees of freedom
# Multiple R-squared: 0.9146, Adjusted R-squared: 0.9143
# F-statistic: 2694 on 2 and 503 DF, p-value: < 2.2e-16
# [[2]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -4.821 -4.614 -1.294 0.473 84.130
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 4.846e+00 4.330e-01 11.192 < 2e-16 ***
# zn -3.322e-01 1.098e-01 -3.025 0.00261 **
# I(zn^2) 6.483e-03 3.861e-03 1.679 0.09375 .
# I(zn^3) -3.776e-05 3.139e-05 -1.203 0.22954
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.372 on 502 degrees of freedom
# Multiple R-squared: 0.05824, Adjusted R-squared: 0.05261
# F-statistic: 10.35 on 3 and 502 DF, p-value: 1.281e-06
# [[3]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -8.278 -2.514 0.054 0.764 79.713
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 3.6625683 1.5739833 2.327 0.0204 *
# indus -1.9652129 0.4819901 -4.077 5.30e-05 ***
# I(indus^2) 0.2519373 0.0393221 6.407 3.42e-10 ***
# I(indus^3) -0.0069760 0.0009567 -7.292 1.20e-12 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.423 on 502 degrees of freedom
# Multiple R-squared: 0.2597, Adjusted R-squared: 0.2552
# F-statistic: 58.69 on 3 and 502 DF, p-value: < 2.2e-16
# [[4]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -3.738 -3.661 -3.435 0.018 85.232
# Coefficients: (2 not defined because of singularities)
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 3.7444 0.3961 9.453 <2e-16 ***
# chas -1.8928 1.5061 -1.257 0.209
# I(chas^2) NA NA NA NA
# I(chas^3) NA NA NA NA
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.597 on 504 degrees of freedom
# Multiple R-squared: 0.003124, Adjusted R-squared: 0.001146
# F-statistic: 1.579 on 1 and 504 DF, p-value: 0.2094
# [[5]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -9.110 -2.068 -0.255 0.739 78.302
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 233.09 33.64 6.928 1.31e-11 ***
# nox -1279.37 170.40 -7.508 2.76e-13 ***
# I(nox^2) 2248.54 279.90 8.033 6.81e-15 ***
# I(nox^3) -1245.70 149.28 -8.345 6.96e-16 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.234 on 502 degrees of freedom
# Multiple R-squared: 0.297, Adjusted R-squared: 0.2928
# F-statistic: 70.69 on 3 and 502 DF, p-value: < 2.2e-16
# [[6]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -18.485 -3.468 -2.221 -0.015 87.219
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 112.6246 64.5172 1.746 0.0815 .
# rm -39.1501 31.3115 -1.250 0.2118
# I(rm^2) 4.5509 5.0099 0.908 0.3641
# I(rm^3) -0.1745 0.2637 -0.662 0.5086
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.33 on 502 degrees of freedom
# Multiple R-squared: 0.06779, Adjusted R-squared: 0.06222
# F-statistic: 12.17 on 3 and 502 DF, p-value: 1.067e-07
# [[7]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -9.762 -2.673 -0.516 0.019 82.842
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -2.549e+00 2.769e+00 -0.920 0.35780
# age 2.737e-01 1.864e-01 1.468 0.14266
# I(age^2) -7.230e-03 3.637e-03 -1.988 0.04738 *
# I(age^3) 5.745e-05 2.109e-05 2.724 0.00668 **
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.84 on 502 degrees of freedom
# Multiple R-squared: 0.1742, Adjusted R-squared: 0.1693
# F-statistic: 35.31 on 3 and 502 DF, p-value: < 2.2e-16
# [[8]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -10.757 -2.588 0.031 1.267 76.378
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 30.0476 2.4459 12.285 < 2e-16 ***
# dis -15.5543 1.7360 -8.960 < 2e-16 ***
# I(dis^2) 2.4521 0.3464 7.078 4.94e-12 ***
# I(dis^3) -0.1186 0.0204 -5.814 1.09e-08 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.331 on 502 degrees of freedom
# Multiple R-squared: 0.2778, Adjusted R-squared: 0.2735
# F-statistic: 64.37 on 3 and 502 DF, p-value: < 2.2e-16
# [[9]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -10.381 -0.412 -0.269 0.179 76.217
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -0.605545 2.050108 -0.295 0.768
# rad 0.512736 1.043597 0.491 0.623
# I(rad^2) -0.075177 0.148543 -0.506 0.613
# I(rad^3) 0.003209 0.004564 0.703 0.482
# Residual standard error: 6.682 on 502 degrees of freedom
# Multiple R-squared: 0.4, Adjusted R-squared: 0.3965
# F-statistic: 111.6 on 3 and 502 DF, p-value: < 2.2e-16
# [[10]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -13.273 -1.389 0.046 0.536 76.950
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 1.918e+01 1.180e+01 1.626 0.105
# tax -1.533e-01 9.568e-02 -1.602 0.110
# I(tax^2) 3.608e-04 2.425e-04 1.488 0.137
# I(tax^3) -2.204e-07 1.889e-07 -1.167 0.244
# Residual standard error: 6.854 on 502 degrees of freedom
# Multiple R-squared: 0.3689, Adjusted R-squared: 0.3651
# F-statistic: 97.8 on 3 and 502 DF, p-value: < 2.2e-16
# [[11]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -6.833 -4.146 -1.655 1.408 82.697
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 477.18405 156.79498 3.043 0.00246 **
# ptratio -82.36054 27.64394 -2.979 0.00303 **
# I(ptratio^2) 4.63535 1.60832 2.882 0.00412 **
# I(ptratio^3) -0.08476 0.03090 -2.743 0.00630 **
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 8.122 on 502 degrees of freedom
# Multiple R-squared: 0.1138, Adjusted R-squared: 0.1085
# F-statistic: 21.48 on 3 and 502 DF, p-value: 4.171e-13
# [[12]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -13.096 -2.343 -2.128 -1.439 86.790
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 1.826e+01 2.305e+00 7.924 1.5e-14 ***
# black -8.356e-02 5.633e-02 -1.483 0.139
# I(black^2) 2.137e-04 2.984e-04 0.716 0.474
# I(black^3) -2.652e-07 4.364e-07 -0.608 0.544
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.955 on 502 degrees of freedom
# Multiple R-squared: 0.1498, Adjusted R-squared: 0.1448
# F-statistic: 29.49 on 3 and 502 DF, p-value: < 2.2e-16
# [[13]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -15.234 -2.151 -0.486 0.066 83.353
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 1.2009656 2.0286452 0.592 0.5541
# lstat -0.4490656 0.4648911 -0.966 0.3345
# I(lstat^2) 0.0557794 0.0301156 1.852 0.0646 .
# I(lstat^3) -0.0008574 0.0005652 -1.517 0.1299
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 7.629 on 502 degrees of freedom
# Multiple R-squared: 0.2179, Adjusted R-squared: 0.2133
# F-statistic: 46.63 on 3 and 502 DF, p-value: < 2.2e-16
# [[14]]
# Call:
# lm(formula = crim_formula, data = Boston)
# Residuals:
# Min 1Q Median 3Q Max
# -24.427 -1.976 -0.437 0.439 73.655
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 53.1655381 3.3563105 15.840 < 2e-16 ***
# medv -5.0948305 0.4338321 -11.744 < 2e-16 ***
# I(medv^2) 0.1554965 0.0171904 9.046 < 2e-16 ***
# I(medv^3) -0.0014901 0.0002038 -7.312 1.05e-12 ***
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# Residual standard error: 6.569 on 502 degrees of freedom
# Multiple R-squared: 0.4202, Adjusted R-squared: 0.4167
# F-statistic: 121.3 on 3 and 502 DF, p-value: < 2.2e-16
```
There are statistically siginificent in indus, nox, age, dis, pratio and medv.
## ch04 Q03
from 4.11
$$
p_k(x)=\frac{\pi_k\frac{1}{\sqrt{2\pi}\delta_k}exp(-\frac{1}{2\delta^2}(x-\mu_k)^2)}{\sum_{l=1}^K \pi_l \frac{1}{\sqrt{2\pi}}exp(-\frac{1}{2\delta_k^2}(x-\mu_l)^2)}
$$
LaTeX refrence from website
then we apply log into it
$$
p1 = log(\pi_k)log(\frac{1}{\sqrt{2\pi}\delta_k})-\frac{1}{2\delta^2}(x-\mu_k)^2 \\
p2 = log(\sum_{l=1}^K \pi_l \frac{1}{\sqrt{2\pi}}exp(-\frac{1}{2\delta_k^2}(x-\mu_l)^2)) \\
log(p_k(x)) = p1 / p2
$$
This LaTeX from myself
because p2 in always a constant, so it could be ignored
$$
log(p_k(x)) = log(\pi_k)log(\frac{1}{\sqrt{2\pi}\delta_k})-\frac{1}{2\delta^2}(x-\mu_k)^2
$$
The last x^2 is quadratic, so the Bayes classifier is non-linear
## Q5
### a
QDA is better on train set, LDA is better on test set.
### b
QDA is better in the both set.
### c
Bigger size n will reduces the variance. QDA will get benefit from that.
### d
False
QDA have higher variance. LDA will perform better if the boundary is linear(means model is not flexible)
Sign in with Wallet
Connect another wallet