---
# System prepended metadata

title: Untitled

---

## Task 1
No it isn't because we get a linear regression model that can only solve linear problems.

weight --  #neurons in X #neurons out
input vector X - 1xn
W1 - n x h1
W2 - h1 x h2
W3 - h2 x h3

b1 - 1 x h1
b2 - 1 x h2
b3 - 1 x h3

f(x)=x - activation function


<!--$Y = f(f(f(X*W1 + b1) * W2 + b2) * W3 + b3) =$
$(((X * W1 + b1) * W2 + b2) * W3 + b3) =$
$(X*W1*W2 + b1*W2 + b2) * W3 + b3 =$
$X*W1*W2*W3 + b1*W2*W3 + b2*W3 + b3$


Let:
$a = W1*W2*W3$
$b = b1*W2*W3 + b2*W3 + b3$-->


$Y = f(f(f(X\cdot W1 + b1) \cdot  W2 + b2) \cdot W3 + b3) =$
$(((X \cdot W1 + b1) \cdot W2 + b2) \cdot W3 + b3) =$
$(X\cdot W1\cdot W2 + b1 \cdot W2 + b2) \cdot W3 + b3 =$
$X \cdot W1 \cdot W2 \cdot W3 + b1\cdot W2\cdot W3 + b2\cdot W3 + b3$


Let:
$a = W1\cdot W2 \cdot W3$
$b = b1\cdot W2\cdot W3 + b2\cdot W3 + b3$

Then:
$Y = a\cdot X + b$


Conlcusion - Linear combination of input elements, same as linear regression. Linear activation does not introduce anything.

## Task 2
$Q^j = \frac{(z^j - y^j)^2}  {2}$
$\delta y/\delta W =  \delta y/\delta e \cdot \delta e/\delta W$
(e is the input of activation function in the forward phase, that is the weighted sum)
$\delta y/\delta e = \delta y/\delta f(e) \cdot \delta f(e)/\delta e = f'(e)$ as y=f(e)
Solve:
$\frac{\delta Q^j}{\delta W}= \frac{\delta \frac{1}{2}(z^j - y^j)^2}{\delta W}=(z^j - y^j) \cdot (\frac{\delta z^j}{\delta W} - \frac{\delta y^j}{\delta W} )=-(z^j - y^j)\frac{\delta y^j}{\delta W}=-(z^j - y^j)\frac{\delta f(e^j)}{\delta W} =$
$=-(z^j - y^j)(f'(e^j) \cdot \frac{\delta e^j}{\delta W} )=-(z^j - y^j)\cdot f'(e^j) \cdot \frac{\delta \sum x_k^j\cdot w_k}{\delta w_i}=-(z^j - y^j)\cdot f'(e^j) \cdot x_i^j$

$\Delta w_i=-\eta \frac{\delta Q^j}{\delta W}=\eta(z^j - y^j)\cdot f'(e^j) \cdot x_i^j$
## Task 3
### a
F-$[0]$ for height, $[1]$ for width
Number of weights  $= (K \cdot F[0]\cdot F[1] \cdot C_1)$, assuming every filter is applied to every channel of input ($K\cdot C_1$)
plus one bias per filter, so K biases
### b
K-no. different filters
C_2 could be simply number of our filters (K) if applied on single channel image (C_1 = 1), or $C_1\cdot C_2$ if we want to compute every possible combination, or else as desired 

From torch docs on convolutional layers:
F-kernel_size
Dilation - we dont use, assume =1
![](https://i.imgur.com/bAPq1NB.png)
