---
# System prepended metadata

title: DLFNLP Exercise 1

---

# DLFNLP Exercise 1

## 1
### a)
11

### b)
$$
  \begin{aligned}
     \begin{bmatrix}
           14 \\
           10 \\
         \end{bmatrix}
  \end{aligned}
$$

### c)
$$
  \begin{aligned}
     \begin{bmatrix}
           8 & 20 \\
           5 & 13 \\
     \end{bmatrix}
  \end{aligned}
$$

### d)

$det(X) = 2*3 - 1*4 = 2 \implies \text{invertible}$

$$
  \begin{aligned}
     \begin{bmatrix}
           1.5 & -2 \\
           -0.5 & 1 \\
     \end{bmatrix}
  \end{aligned}
$$

### e)
2
    
$$
\begin{aligned}
    \begin{bmatrix}
        1.5 & -2 \\
        -0.5 & 1 \\
    \end{bmatrix}
\end{aligned}
$$
            
## 2
### a)

$3 * x^2 + 1$

### b)
$40x^3 - 8x$

### c)
$\frac{16x^2+4x-24}{64x^2+16x+1}$

### d)
$24*(3x-2)^7$

### e)
$\frac{2x + 1}{x^2 + x}$

## 3.
![](https://i.imgur.com/qV33W0y.png)

## 4.
### b)
Fully connected layer => every beuron has weights to the next layer + every neuron (besides input neurons) have a bias

From this follows there are:
$D_X * D_H + D_H + D_H * D_Y + D_Y = (D_X+1)D_H + (D_H+1)D_Y$ 
neurons. 


## 5.

We had some problems with mismatched dimensionality in our backpropagation method. Because of this we sadly didn't manage to calculate the accuracy before the deadline.

```
import numpy as np
import random
from sklearn.datasets import load_iris

class nn:
    W_1 = np.random.rand(8,4)
    W_2 = np.random.rand(3,8)

    B_1 = np.random.rand(8)
    B_2 = np.random.rand(3)
    
    h_1 = None
    h_2 = None
    a_2 = None
    
    @staticmethod
    def cross_entropy(y,y_hat):
        return -np.sum([y[i] * np.log(y_hat[i]) for i in range(len(y))])
    
    @staticmethod
    def relu(x):
        return np.array([xi if xi > 0 else 0 for xi in x])
    
    
    
    @staticmethod
    def softmax(x):
        denom = np.sum([np.exp(xi) for xi in x])
        return np.array([np.exp(xi)/denom for xi in x])
    
    def softmax_cross_d(self,x,y):
        return self.softmax(x) - y
    
    def forward(self,X):
        a_1 = self.W_1 @ X - self.B_1
        self.h_1 = self.relu(a_1)
        
        self.a_2 = self.W_2 @ self.h_1 - self.B_2
        self.h_2 = self.softmax(self.a_2)
        return self.h_2
    
    def fit(self,train,target,epochs,learning_rate):
        for epoch in range(epochs):
            for i,X in enumerate(train):
                y_hat = self.forward(X)
                err = self.cross_entropy(target[i],y_hat)
                l2_err = self.softmax_cross_d(y_hat,target[i])
                l1_err = np.transpose(self.W_2) @ l2_err
                
                 
                self.W_2 = self.W_2 - (learning_rate * l2_err.dot(np.transpose(self.h_2)))
                self.W_1 = self.W_1 - (learning_rate * l1_err.dot(np.transpose(self.h_1)))
                
                self.B_2 = self.B_2 - learning_rate * l2_err 
                self.B_1 = self.B_1 - learning_rate * l1_err 
            print(err)
            
data = load_iris()
data_len = len(data["target"])
train_num = int(0.8 * data_len)
test_num = data_len - train_num

test_data = []
test_targets = []

test_rows = random.sample(range(data_len),test_num)
for i in test_rows:
    test_data.append(data["data"][i])
    test_targets.append(data["target"][i])

train_data = []
train_targets = []

for i in range(data_len):
    if i not in test_rows:
        train_data.append(data["data"][i])
        train_targets.append(data["target"][i])

for i in range(len(train_targets)):
    train_targets[i] = np.eye(3)[train_targets[i]]
    
for i in range(len(test_targets)):
    train_targets[i] = np.eye(3)[test_targets[i]]

model = nn()

model.fit(np.array(train_data),np.array(train_targets),200,10 ** -4)
```



## 6.
### a)
1) holy
```
hearken
serpent
perished
faithfulness
appear
sayingspeak
baruch
common
solemn
grain

`````
2) woman
`````
olivetree
threshingfloor
jacob
sinoffering
nathan
pot
philip
angel
fountain
nebuzaradan
`````

3) light
`````
light
delight
delighteth
lightnings
lightning
delighted
lights
twilight
flight
`````
