### Convolution Neural Network  --- ## 1. Perceptron ---- ### Binatry Classification Problem Given a trainging data set $S = \{ \ (x^{i},y_i) \ | \ x^i\in R, \ y_i \in \{-1,1\}\}$ $x^i \in A_{+}$ if and only if $y_i \ = \ 1$, $x^i \in A_{-}$ if and only if $y_i \ = \ -1$,  ----  ---- ### Rosenblatt's Perceptron  Demo! --- ## 2. Neural Network  ---- ### Neuron  ----  $\vec{H}^{*}= \begin{bmatrix} \vec{H}^{*}_0\\ \vec{H}^{*}_1\\ \vec{H}^{*}_2 \end{bmatrix}= \begin{bmatrix} f \ (\vec{w_0} \cdot \vec{x})\\ f \ (\vec{w_1} \cdot \vec{x})\\ f \ (\vec{w_2} \cdot \vec{x})\\ \end{bmatrix}= f \ \Big{(} W \cdot \vec{x} \Big{)}$ ---- ### Feed Forward  ---- ### Gradient Descent Minimization problem: $min \ L(x)$  $$ \vartriangle x^{i+1} := x^{i+1} - x^{i} = - \eta \ \triangledown L(x^i)$$ ---- ### Backpropogation ----  $\frac{\partial L}{\partial V[i][j]}=\frac{\partial L}{\partial O[i]}\frac{\partial O[i]}{\partial V[i][j]}=\frac{\partial L}{\partial O[i]}H^{*}[j]$ ---- $\frac{\partial L}{\partial O[k]}=\frac{\partial L}{\partial O^*[k]}\frac{\partial O^*[k]}{\partial O[k]}=-(T[k]-O^{*}[k])\sigma'(O[k])$ $\sigma'(O[k])=\sigma(O[k])(1-\sigma(O[k]))=O^*[k](1-O^*[k])$ $\triangle V[i][j]=-\eta\frac{\partial L }{\partial V[i][j]}=-\eta \frac{\partial L}{\partial O[i]} H^{*}[j]$ $\triangle c[k] = -\eta\frac{\partial L}{\partial c[k]}=-\eta\frac{\partial L}{\partial O[k]}$ ----  $\frac{\partial L}{\partial W[i][j]}=\frac{\partial L}{\partial H[i]}\frac{\partial H[i]}{\partial W[i][j]}=\frac{\partial L}{\partial H[i]}I[j]$ ---- $\frac{\partial L}{\partial H[i]}=\frac{\partial L}{\partial H^{*}[i]}\frac{\partial H^{*}[i]}{\partial H[i]}=\Big{(} \sum_{l=0}^{2}\frac{\partial L}{\partial O[l]}\frac{\partial O[l]}{\partial H^{*}[i]}\Big{)}\sigma'(H[i])$ $=\Big{(} \sum_{l=0}^{2}\frac{\partial L}{\partial O[l]}\frac{\partial O[l]}{\partial H^{*}[i]}\Big{)}H^*[i](1-H^*[i])$ $=\Big{(} \sum_{l=0}^{2}\frac{\partial L}{\partial O[l]}V[l][i]\Big{)}H^*[i](1-H^*[i])$ $\triangle W[i][j]=-\eta\frac{\partial L}{\partial W[i][j]}=-\eta\frac{\partial L}{\partial H[i]}I[j]$ $\triangle b[k] = -\eta\frac{\partial L}{\partial b[k]}=-\eta\frac{\partial L}{\partial H[k]}$ --- ## 3. CNN  ---- ### Convolution  ----  [Demo](https://doodle-draw.herokuapp.com/conv) ---- ### Pooling  --- ## 4. Demo [Doodle Draw](https://doodle-draw.herokuapp.com/draw) [Github](https://github.com/jarcomatolsavisch/doodle-drawing) --- ## 5. Resources [Neural Networks Chart](https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks-explained-3fb6f2367464)