# Logistic regression <!-- Put the link to this slide here so people can follow --> slide: https://hackmd.io/@ccornwell/Logistic-regression --- <h3>Coordinate system for half-space model?</h3> <br /> <br /> <br /> <br /> <br /> <br /> <br /> --- <h3>Half-space model: Concept Problem</h3> <font size=+3>Often data is messy, and classification questions on decision boundaries are not so clear.</font> <br /> <br /> <br /> --- <h3>A more probabilistic approach</h3> - <font size=+2>Rather than $\texttt{sign}({\bf w}\cdot{\bf x_i}+b)$ for prediction $y_i$, compose $z = {\bf w}\cdot{\bf x_i}+b$ with something else...</font> - <font size=+2>e.g. $\sigma(z) = \frac{1}{1+e^{-z}}$ (the *logistic function*, sometimes *sigmoid function*).</font>  ---- <h3>A more probabilistic approach</h3>  - <font size=+2>When $z = {\bf w}\cdot{\bf x_i}+b$ is large and positive, then $\sigma(z) \approx 1$. When $z < 0$ but $|z|$ is large, then $\sigma(z) \approx 0$. Also, if $z=0$, then $\sigma(z) = 1/2$.</font> - <font size=+2>Output $\sigma({\bf w}\cdot{\bf x_i}+b)$ thought of as probability that $y_i$ is $+1$.</font> --- <h3>Fitting a model using data</h3> <h5>Given your data, how do you find "best" parameters?</h5> - <font size=+3>Don't want to assume data is linearly separable. Need to measure how well given parameters fit data.</font> > $\leadsto$ <font size=+2>A **loss function** (sometimes *error function* or *objective function*).</font> ---- <h3>Fitting a model using data</h3> > $\leadsto$ <font size=+2>A **loss function**</font> - <font size=+3>Could use *accuracy* (percentage of predicted labels that are correct), but...</font> 1. <font size=+3>Ignores distance-to-hyperplane info the model now gives us;</font> 2. <font size=+3>*Very* sensitive to noise; also not differentiable (in the parameters).</font> - <font size=+3>Later: more about negative log-loss function -- the right loss function here. First, detour into linear regression.</font>