# The Bias-Variance decomposition <!-- Put the link to this slide here so people can follow --> slide: https://hackmd.io/@ccornwell/bias-variance --- <h3>High model variance</h3> - <font size=+2>When a model is overly responsive to which points are in the training set.</font>  - <font size=+2 style="color:#181818;">Were some values of $x$ where values of two models differed by more than 24; average difference over interval was more than 0.3.</font> ---- <h3>High model variance</h3> - <font size=+2>When a model is overly responsive to which points are in the training set.</font>  - <font size=+2 style="color:#181818;">Were some values of $x$ where values of two models differed by more than 24; average difference over interval was more than 0.3.</font> ---- <h3>High model variance</h3> - <font size=+2>When a model is overly responsive to which points are in the training set.</font>  - <font size=+2>Were some values of $x$ where values of two models differed by more than 24; average difference over interval was more than 0.3.</font> --- <h3>Compare to linear model</h3> - <font size=+2>With less parameters, a model can be less responsive to the training set.</font>  - <font size=+2 style="color:#181818;">The max difference over $x$ was about 0.03, and average difference in values over the interval, around 0.01.</font> ---- <h3>Compare to linear model</h3> - <font size=+2>With less parameters, a model can be less responsive to the training set.</font>  - <font size=+2 style="color:#181818;">The max difference over $x$ was about 0.03, and average difference in values over the interval, around 0.01.</font> ---- <h3>Compare to linear model</h3> - <font size=+2>With less parameters, a model can be less responsive to the training set.</font>  - <font size=+2>The max difference over $x$ was about 0.03, and average difference in values over the interval, around 0.01.</font> --- <h3>"Bias" of a model</h3> - <font size=+2>If the (true) labels (or target values) vary _a lot_ as you change input variable $x$, then you want your model to do the same.</font> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> ---- <h3>"Bias" of a model</h3> - <font size=+2>If the (true) labels (or target values) vary _a lot_ as you change input variable $x$, then you want your model to do the same.</font> - <font size=+2>If this does not occur, model has too much _bias_.</font>  --- <h3>The Bias-Variance decomposition</h3> - <font size=+2>Having chosen a (type of) model to work with...$h$</font> - <font size=+2>Training set $S$. Will talk about the expected value of $h_S(x)$. Write as $\overline{h}(x)$.</font> <br /> <br /> <br /> <br /> <br /> <br /> <br /> --- <h3>More Discussion on Bias-Variance</h3> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br />