---
title: Open project - Chatbot
tags: APAC HPC-AI competition
---
[TOC]

# We are almost there!!!

On first terminal:
`rasa x`: open the bot GUI

Second terminal:
1. `cd Go_to_chatbot_movie.../`
2. `./ngrok http 5002`: to accept access from external connection
3. replace the `http://localhost:5002` portion when `Share your bot` with `https://831897ef2f98.ngrok.io/guest/conversations/production/29fbd34c2a06481f`
4. Then send the new URL to others!!!

Third terminal:
`rasa run actions`: open server `:5055/webhook`

# WTF is Chatbot?

A:聊天機器人

# Building chatbot with Python
## Ch. 1 The beloved chatbot
### Q: Can the problem be solved by simple question and answer or back-and-forth communication?
- Yes, FAQs are nothing but simple frequently asked questions and their relative answers.

### Q: Can your bot's task be automated and fixed?
- Yes, an FAQ bot would need to get the question, analyze the question, fetch information from the database, and give it back to the user. ==There is nothing here that can't be done using coding!!!==

### Components of a chatbot
- Intent
- Eg. Like when a user says, "Find me a girl friend", to a chatbot. This is an intent for a bot. It could be named "find_girl".
- Entities
- Intents have metadata about the intent called "**Entities**". In the example, "Find me a girl friend", finding a friend could be an intent and the entity is "**girl**", which could have been something else, like boy, ugly.
- Utterances
- different forms of the same question/intent your user may show.
- find me a girl friend
- I want to sleep with a girl
- I need a girl to fill me. (yew...)

# Ch. 2 Natural Language Processing for Chatbots
==There is nothing artificial in AI; It's actually machine learning and deep learning written by great people.==

## What is spaCy?
- an open source software library for advanced NLP, written in Python and Cython.
- offers the fastest syntactic parser in the world.
- offers statistical neural network for a wide range of languages. (But hell no Chinese :cry:)
- designed with "**get things done**" in mind.

---
# Homework

1. [Deep Learning for NLP: Creating a Chatbot with Keras!](https://towardsdatascience.com/deep-learning-for-nlp-creating-a-chatbot-with-keras-da5ca051e051)

2. [NLP with Keras](https://learning.oreilly.com/videos/keras-tips-tricks/9781838644239/9781838644239-video1_1)
- An introduction to NLP and Keras
- Builing a sentiment Analysis Engine with NLP
- Training a Chatbot with NLP
- Implementing your trained Chatbot

:::danger
禮拜五記得回報進度ㄚ no
:::
>雖然硬著頭皮看了但4有看沒有懂 QAQ
>看懂除了flask, code某些有點茫哈哈

---
# Deep Learning for NLP: Creating a Chatbot with Keras
Neural network structures for exploiting sequential data like text or audio were introduced [here](https://towardsdatascience.com/deep-learning-for-nlp-anns-rnns-and-lstms-explained-95866c1db2e4).
- Artificial Neural Network (ANN), Machine Learning models that try to mimic the functioning of the human brain, whose structure is built from **a large number of neurons connected in between them**.
- These individual neurons can be stacked on top of each other forming layers of the size that we want, and then these layers can be sequentially put next to each other to make the network deeper.
- When networks are built in this way, the neurons that don’t belong to the input or output layers are considered part of the **hidden layers**.
> Image of a larger neural network, composed of many individual neurons and layers: an input layer, 2 hidden layers and an output layer.

---
## What is Deep Learning then?
It's really just the use of **multiple hidden layers** to enhance the performance of our neural models.
> Deep Learning is GOOD! :+1:

When we train a neural network (training a neural network is the ML expression for making it learn) we **feed it a set of known data** (in ML this is called labelled data), have it **predict a characteristic** that we know about such data (like if an image represents a dog or a cat) and then compare the predicted result to the actual result.

As this process goes on and the network makes mistakes, it **adapts the weights of the connections in between the neurons** to reduce the number of mistakes it makes. Because of this, as shown before, if we give the network more and more data most of the time it will improve it’s performance.

---
## Learning from sequential data — Recurrent Neural Networks

---
## Keras: Easy Neural Networks in Python
- Keras is an **open source, high level library** for developing neural network models.
- It’s core principle is to make the process of **building a neural network, training it, and then using it to make predictions**.
- Keras is actually just an interface that can run on top of different Deep Learning frameworks like CNTK, Tensorflow, or Theano for example.


In a Neural Network each node in a specific layer takes the weighted sum of the outputs from the previous layer, applies a mathematical function to them, and then passes that result to the next layer.

## Steps for creating a Keras model
### Step 1: Define a network model

#Define Sequential Model
model = Sequential() 
#Create input layer
model.add(Dense(32, input_dim=784))
#Create hidden layer 
model.add(Activation('relu')) 
#Create Output layer
model.add(Activation('sigmoid'))  

> Most common network model: Sequential model
> - will be defined as a sequence of layers
> - each with its own customisable size and activation function.
> - In these models the first layer will be the input layer, which requires us to define the size of the input that we will be feeding to the network. After this more and more layers can be added and customised until we reach the final output layer.

### Step 2: Compile it

#Compiling the model with a mean squared error loss and RMSProp #optimizer
model.compile(optimizer='rmsprop',loss='mse')

> Compiling the structure transforms the simple sequence of layers that we have previously defined, into a complex group of matrix operations that dictate how the network behaves.
>

### Step 3: Train the network

# Train the model, iterating on the data in batches of 32 samples 
model.fit(data, labels, epochs=10, batch_size=32)

> Use back-propagation algorithm
>

### Step 4: Use it to make predictions on new data

## Training our own chatbot
### Key elements of a chatbot
- Rules vs chatbot
- Goal-oriented
- best if the chatbot does only one thing
- Platforms that ware supported

:::success
The Best chatbots are those that pass the Turing Test.
:::

### Natural language toolkit
- NLTK
- A leading platform for building Python programs to work with human language data

# Code
[Chatbot example code](https://github.com/PacktPublishing/Keras-Tips-Tricks-and-Techniques/tree/master/Section%205/CODE)

- chatbot.py

from flask import Flask, jsonify, request
import numpy as np
from flask_cors import CORS, cross_origin
import sys
import io
from keras.models import load_model
import nltk
import pickle
nltk.download('punkt')
from nltk.stem.lancaster import LancasterStemmer
stemmer = LancasterStemmer()

#setup the flask app
app = Flask(__name__)
cors = CORS(app)
app.config['CORS_HEADERS'] = 'Content-Type'

model = load_model('chatbot_model.h5')
data = pickle.load( open( "chatbot-data.pkl", "rb" ) )
words = data['words']
classes = data['classes']

model._make_predict_function()


#prediction route
@app.route('/chat_with_me', methods=["POST"])
@cross_origin()
def get_answer():
    data = request.json
    question = data['question']
    print(question)
    p = parse_sentence(question, words) 
    input = np.array([p])
    prediction = model.predict(inputvar)
    answer= classes[np.argmax(prediction)]
    prediction = {'answer':answer}

    return jsonify(prediction)

#this function is for cleanup and tokenizing
def tokenize_sentence(sentence):
    sentence_words = nltk.word_tokenize(sentence)
    sentence_words = [stemmer.stem(word.lower()) for word in sentence_words]
    return sentence_words

#translate the sentence into words
def parse_sentence(sentence, words):
    sentence_results = tokenize_sentence(sentence)
    bag = [0]*len(words)  
    for s in sentence_results:
        for i,w in enumerate(words):
            if w == s: 
                bag[i] = 1
    return(np.array(bag))

if __name__ == "__main__":
        app.run()

> #tokenize_sentence()
> NLTK provides a function called word_tokenize() for splitting strings into tokens (nominally words).

- intents.json

{"intents": [
        {"tag": "greeting",
         "patterns": ["Hi there", "How are you", "Is anyone there?", "Hello", "Good day"],
         "responses": ["Hello, thanks for asking", "Good to see you again", "Hi there, how can I help?"]
        },
        {"tag": "goodbye",
         "patterns": ["Bye", "See you later", "Goodbye", "Nice chatting to you, bye", "Until next time"],
         "responses": ["See you!", "Have a nice day", "Bye! Come back again soon."]
        },
        {"tag": "thanks",
         "patterns": ["Thanks", "Thank you", "That's helpful", "Awesome, thanks", "Thanks for helping me"],
         "responses": ["Happy to help!", "Any time!", "My pleasure"]
        },
        {"tag": "noanswer",
         "patterns": [],
         "responses": ["Sorry, I can't understand you", "Please give me more info", "Not sure I understand"]\
        },
        {"tag": "options",
         "patterns": ["How you could help me?", "What you can do?", "What help you provide?", "How you can be helpful?", "What support is offered"],
         "responses": ["I can help manage your library books. What do you want to do?"]
        },
        {"tag": "book_search",
         "patterns": ["Show me adventure books", "I am looking for some books", "Give me a list of books", "Show me new books" ],
         "responses": ["Searching for books..."]
        },
        {"tag": "return_book",
         "patterns": ["I want to return a book", "returning a book", "book return", "Can I return a book" ],
         "responses": ["Navigating to book return module"]
        },
        {"tag": "renew_book",
                "patterns": ["Can I renew a book", "Renew a book", "I would like to renew my books", "Renew all my books" ],
                "responses": ["Navigating to renew book module"]
               }
   ]
}

- Keras chatbot example

import nltk
nltk.download('punkt')

from nltk.stem.lancaster import LancasterStemmer
stemmer = LancasterStemmer()

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout
from keras.optimizers import SGD
import random

> **nltk** -> Natural Language ToolKit
> **punkt** -> a tokenizer model, it divides a text into a list of sentenses.
> **stemmer** -> an important process in NLP, it chopout the end of words, better when training.
> eg. the boy's cars are different colors $\Rightarrow$ the boy car be differ color
> **Sequential** -> defining the structure of the model
>

import json
with open('intents.json') as json_data:
    intents = json.load(json_data)

> Import our intents data. Our intents are made of 2 parts: the **tag** which is the intent we want to match, **patterns** natural language ways to learn which intent to match to.

words = []
classes = []
documents = []
ignore_words = ['?']
for intent in intents['intents']:
    for pattern in intent['patterns']:
        w = nltk.word_tokenize(pattern)
        words.extend(w)
        documents.append((w, intent['tag']))
        if intent['tag'] not in classes:
            classes.append(intent['tag'])

words = [stemmer.stem(w.lower()) for w in words if w not in ignore_words]
words = sorted(list(set(words)))

classes = sorted(list(set(classes)))

print (len(documents), "documents")
print (len(classes), "classes")
print (classes)
print ()
print (len(words), "unique stemmed words")
print (words)

> We need to parse out data. The tags are our labels and the patterns are our training data to learn from in order to get to the right tag.
> **parse** (v.) 從語法上分析
>
> Example output:
> 32 documents
7 classes
['book_search', 'goodbye', 'greeting', 'options', 'renew_book', 'return_book', 'thanks']
55 unique stemmed words
["'s", ',', 'a', 'adv', 'al', 'am', 'anyon', 'ar', 'awesom', 'be', 'book', 'bye', 'can', 'chat', 'could', 'day', 'do', 'for', 'giv', 'good', 'goodby', 'hello', 'help', 'hi', 'how', 'i', 'is', 'lat', 'lik', 'list', 'look', 'me', 'my', 'new', 'next', 'nic', 'of', 'off', 'provid', 'renew', 'return', 'see', 'show', 'som', 'support', 'thank', 'that', 'ther', 'tim', 'to', 'until', 'want', 'what', 'would', 'you']

training = []

output_empty = [0] * len(classes)

for doc in documents:
    bag = []
    pattern_words = doc[0]
    pattern_words = [stemmer.stem(word.lower()) for word in pattern_words]

    for w in words:
        bag.append(1) if w in pattern_words else bag.append(0)
    
    output_row = list(output_empty)
    output_row[classes.index(doc[1])] = 1
    
    training.append([bag, output_row])

random.shuffle(training)
training = np.array(training)

train_x = list(training[:,0])
train_y = list(training[:,1])

> Loop through the patterns and use the stemmer to create a base word in an attempt to represent related words and then create a set of features (patterns) and labels (tags).

model = Sequential()
model.add(Dense(128, input_shape=(len(train_x[0]),), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(len(train_y[0]), activation='softmax'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

model.fit(np.array(train_x), np.array(train_y), epochs=200, batch_size=5, verbose=1)

model.save("chatbot_model.h5")
pickle.dump( {'words':words, 'classes':classes, 'train_x':train_x, 'train_y':train_y}, 
            open( "chatbot-data.pkl", "wb" ) )

> Create our Keras model architecture. Note that the last layer matches the number of classes we learn on.
> This is a deep learning stucture.
>

:::danger
NB. If you publish work that uses NLTK, please cite the NLTK book as follows:

Bird, Steven, Edward Loper and Ewan Klein (2009), *Natural Language Processing with Python*. O’Reilly Media Inc.
:::

---
# Reference
- NLP with keras
- [NLP with keras!!!](https://ithelp.ithome.com.tw/articles/10206265)
- [Keras多層感知器MLP進行IMDb情緒分析](https://ithelp.ithome.com.tw/articles/10206267)
- [Keras RNN進行IMDb情緒分析](https://ithelp.ithome.com.tw/articles/10206268)
[Chatbot Tutorial](https://pytorch.org/tutorials/beginner/chatbot_tutorial.html)
[Creating a chatbot from scratch using Keras](https://medium.com/predict/creating-a-chatbot-from-scratch-using-keras-and-tensorflow-59e8fc76be79)
[Deep Learning for NLP: Creating a Chatbot with Keras!](https://towardsdatascience.com/deep-learning-for-nlp-creating-a-chatbot-with-keras-da5ca051e051)
[Learn Keras](https://www.tutorialspoint.com/keras/index.htm)
[keras & tensorflow introduction](https://medium.com/chiukevin0321/tensorflow%E8%88%87keras%E5%9F%BA%E6%9C%AC%E4%BB%8B%E7%B4%B9-621352fc7150)
[順序模型 tensorflow core](https://www.tensorflow.org/guide/keras/sequential_model?hl=zh-tw)
[Keras:基於Python的深度學習庫](https://keras-cn.readthedocs.io/en/latest/)