[TOC]
# Project3 - RNN 報告

## 題目敘述
* 給1000個輸入data用RNN或RLU或LSTM做一個模型預測第1001到1500個數字
---
### Code Description
- models

import keras
import numpy as np
from keras.models import Sequential
from keras.layers import Embedding, SimpleRNN, LSTM, GRU,Dense,Dropout,Activation,Flatten
from keras.optimizers import RMSprop
from keras.utils import plot_model
# draw the graphs
import matplotlib.pyplot as plt

* Data spliting.
* Read A3_train.txt which I copied all 1000 data in it.
* split by endline character and change the data type to float64 make it accrate
* A3_train.txt is in the folder of my report

import os
f = open('A3_train.txt')
data = f.read()
f.close()
lines = data.split('\n')
lines = np.asarray(lines, dtype=np.float64)

- split the train set by data
- Our model's input sequence is 100 and return 1 output which is it's prediction
- We have 1000 data.
- Split 1-100,2-101,...,900-999 => 900 sets as train set
- The answer is from the data's 101th number to 1000th number.

train_set=[]
for i in range(900):
    train_set=np.append(train_set,lines[i:i+100],axis=0)
train_set = train_set.reshape(900,1,100)
ytrain = lines[100:1000]
ytrain = np.reshape(ytrain,(ytrain.shape[0],1))

- model is using LSTM layer
- we have 2 dense layer to calculate the prediction data.

# model creation
model = Sequential()
model.add(LSTM(units = 32,input_shape=(1,100)))
model.add(Dense(8))
model.add(Activation("relu"))
model.add(Dense(1))
model.add(Activation("relu"))
model.compile(loss="mse", optimizer="rmsprop",metrics=['mse','mae'])
model.summary()

- model training and prediction
- We train the model 50 times.

# model training
history = model.fit(train, ytrain, epochs=50, batch_size=20,verbose=1)

- Predictions:
- the process is we continully get our last 100 data and make prediction
- then append our data to the original data sequence.

sq = lines
for i in range(500):
    last100 = np.reshape(sq[-100:],(1,1,100))
    predict = model.predict(last100)[-1][0]
    sq = np.append(sq,predict)
plt.plot(sq,'c-')
plt.grid()
plt.savefig('data1500.png')
plt.show()

- We need to reformat our data and produce a txt file.

sq1 = np.round(sq,decimals=2)
sq1 = np.float32(sq1)
sq1 = sq1[1000:1500]
np.savetxt("out.txt",sq1,fmt='%.2f' , delimiter="",newline=', ')

- My gpu setting have some problems
- this is the code I use to solve it.

import tensorflow as tf
gpu_options = tf.compat.v1.GPUOptions(per_process_gpu_memory_fraction=0.2)
tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(gpu_options=gpu_options,allow_soft_placement=True))

### Result
- the final result visualization
- the loss graph
- the last 500 data visualization
- the model graph

### My reflection
- The RNN model needs a lot of space resources.
- The model will be very large if I add some more layers.
- The prediction loss I use is mean squared error, I think this way may have some bias, so I add mae beside the mse at the same graph.