Lab 2: Adding NLP to Your Bot

Some background on how NLP works (not required, but interesting):

• Video (quick, just watch 60 seconds): https://youtu.be/d4gGtcobq8M?t=110. They are explaining the TF-IDF method (which you can read about here if you're curious: https://en.wikipedia.org/wiki/Tf–idf)
• Video (broadest): https://www.youtube.com/watch?v=fOvTtapxa9c, slight bit on chatbots at 4:53
• Article: https://becominghuman.ai/a-simple-introduction-to-natural-language-processing-ea66a1747b32.

Step 0: hit the magic button

Go to your settings, and then click "Built-In NLP". For language model, let's default to English, but also include support for Vietnamese.

Step 1: Handle incoming messages

Remember to set up ngrok again (and change your callback URL) so you can read messages. Now if we look carefully at our incoming messages, we'll see a new section: nlp.

sender { id: '3122675527859206' }
message { mid:
   'm_kzKZlgkVZTQqKnPiVHQ0AwxUI7v_foLXkkJMlQdvAaVM_aEw0Fjh3RHtwjgpIFVw5k6S5EAW2Frwi77tRktjTA',
  text: 'hello dude',
  nlp:
   { intents: [],
     entities: { 'wit$location:location': [Array] },
     traits: { 'wit$sentiment': [Array], 'wit$greetings': [Array] },
     detected_locales: [ [Object] ] } }

Let's dig more into this nlp section. Let's log it out to the screen, but let's use a little known method called console.dir to print out all the different fields:

console.dir(message.nlp, { depth: null })

Send the message "Hello from Hanoi!" or similar, and you'll see output similar to this:

{ intents: [],
  entities:
   { 'wit$location:location':
      [ { id: '624173841772436',
          name: 'wit$location',
          role: 'location',
          start: 11,
          end: 16,
          body: 'Hanoi',
          confidence: 0.9309,
          entities: [],
          resolved:
           { values:
              [ { name: 'Hanoi',
                  domain: 'region',
                  coords: { lat: 21, long: 105.75 },
                  timezone: 'Asia/Bangkok',
                  external: { geonames: '1581129' },
                  attributes: {} },
                { name: 'Hanoi',
                  domain: 'locality',
                  coords: { lat: 21.024499893188, long: 105.84117126465 },
                  timezone: 'Asia/Bangkok',
                  external:
                   { geonames: '1581130', wikidata: 'Q1858', wikipedia: 'Hanoi' },
                  attributes: {} } ] },
          type: 'resolved' } ] },
  traits:
   { 'wit$sentiment':
      [ { id: '5ac2b50a-44e4-466e-9d49-bad6bd40092c',
          value: 'positive',
          confidence: 0.7759 } ],
     'wit$greetings':
      [ { id: '5900cc2d-41b7-45b2-b21f-b950d3ae3c5c',
          value: 'true',
          confidence: 0.9997 } ] },
  detected_locales: [ { locale: 'en_XX', confidence: 0.9944 } ] }

There are some amazing things in this data, that Messenger Platform (with wit.ai) have automatically understood for us.

1. The bot is 93% sure we're talking about Hanoi, Vietnam.
2. The bot is 78% sure we're positive.
3. The bot is 99% sure we're giving greetings.
4. The bot is 99% sure we're talking in English.

Spend some time playing around with the bot and observing the output. Try the following scenarios:

• "My favorite time is studying on Wednesdays at 7PM"

{ intents: [],
  entities:
   { 'wit$datetime:datetime':
      [ { id: '698488774257035',
          name: 'wit$datetime',
          role: 'datetime',
          start: 35,
          end: 54,
          body: 'on Wednesday at 7PM',
          confidence: 0.9634,
          entities: [],
          type: 'value',
          grain: 'hour',
          value: '2020-09-02T19:00:00.000+07:00',
          values:
           [ { type: 'value',
               grain: 'hour',
               value: '2020-09-02T19:00:00.000+07:00' },
             { type: 'value',
               grain: 'hour',
               value: '2020-09-09T19:00:00.000+07:00' },
             { type: 'value',
               grain: 'hour',
               value: '2020-09-16T19:00:00.000+07:00' } ] } ] },
  traits:
   { 'wit$sentiment':
      [ { id: '5ac2b50a-44e4-466e-9d49-bad6bd40092c',
          value: 'positive',
          confidence: 0.9125 } ] },
  detected_locales: [ { locale: 'en_XX', confidence: 1 } ] }
  

In theory, you should receive values for any entity detected from the following list: https://developers.facebook.com/docs/messenger-platform/built-in-nlp/#entities_and_traits, which includes things like quantities, temperatures, volumes, and sentiment.

In this step, try to trigger all of them (in either Vietnamese or English). Here are some examples to get you started:

• Học phí của lớp này là 500 triệu đồng

{ intents: [],
  entities:
   { 'wit$amount_of_money:amount_of_money':
      [ { id: '232943834700385',
          name: 'wit$amount_of_money',
          role: 'amount_of_money',
          start: 23,
          end: 37,
          body: '500 triệu đồng',
          confidence: 1,
          entities: [],
          unit: 'VND',
          type: 'value',
          value: 500000000 } ] },
  traits: {},
  detected_locales: [ { locale: 'vi_VN', confidence: 1 } ] }

• Ngày mai gặp nhau ở hồ tây nhé

 intents: [],
  entities:
   { 'wit$location:location':
      [ { id: '237576840797015',
          name: 'wit$location',
          role: 'location',
          start: 20,
          end: 26,
          body: 'hồ tây',
          confidence: 0.9361,
          entities: [],
          suggested: true,
          value: 'hồ tây',
          type: 'value' } ],
     'wit$datetime:datetime':
      [ { id: '911522092629211',
          name: 'wit$datetime',
          role: 'datetime',
          start: 0,
          end: 8,
          body: 'ngày mai',
          confidence: 1,
          entities: [],
          type: 'value',
          grain: 'day',
          value: '2020-08-30T00:00:00.000+07:00',
          values:
           [ { type: 'value',
               grain: 'day',
               value: '2020-08-30T00:00:00.000+07:00' } ] } ] },
  traits: {},
  detected_locales: [ { locale: 'vi_VN', confidence: 1 } ] }

• The two hours of class today are so amazing that I want to dance

 intents: [],
  entities:
   { 'wit$datetime:datetime':
      [ { id: '698488774257035',
          name: 'wit$datetime',
          role: 'datetime',
          start: 23,
          end: 28,
          body: 'today',
          confidence: 0.9642,
          entities: [],
          type: 'value',
          grain: 'day',
          value: '2020-08-29T00:00:00.000+07:00',
          values:
           [ { type: 'value',
               grain: 'day',
               value: '2020-08-29T00:00:00.000+07:00' } ] } ],
     'wit$duration:duration':
      [ { id: '237879440959430',
          name: 'wit$duration',
          role: 'duration',
          start: 4,
          end: 13,
          body: 'two hours',
          confidence: 0.9672,
          entities: [],
          normalized: { unit: 'second', value: 7200 },
          unit: 'hour',
          type: 'value',
          hour: 2,
          value: 2 } ] },
  traits:
   { 'wit$sentiment':
      [ { id: '5ac2b50a-44e4-466e-9d49-bad6bd40092c',
          value: 'positive',
          confidence: 0.9787 } ] },
  detected_locales: [ { locale: 'en_XX', confidence: 1 } ] }

Step 2: Add a Custom Model with a Custom Wit App

This built-in NLP is very cool, but a bit limited because as you can see above, only the entities are being populated. Perhaps we could dig into the intent. To do so, we'll have to create a custom model.

Click "Create a new Wit app".

After filing out the forms, it's a bit confusing where to go next, but in the bottom right, you'll see "Go to my Wit app"

If you have the option, in the upper right, click "Switch to the New Wit" to arrive at a page that looks like this:

Here, in the previous step, you should have tried many phrases - and not always been successful. Here you can help train the bot to understand what you said before.

For example here I've manually told the bot about 40 Degrees being a temperature and that ha noi is a location.

You can optionally go through and train your bot, but we'll be doing this in a more specific way in the next step.

Step 3: Ordering Coffees

Note that the following instructions will be for a Chatbot that understands Englihs, but you could do it in Vietnamese instead if you change the setting here:

General NLP is useful but let's specialize around a bot that has a job: to schedule a coffee delivery. To do so, first we'll create an intent. Click "intents" to the left, and then click "+ Intent".

Now, you'll see not much is set here. What we'll have to do is go back one page, and add some utterances.

Go to the bottom, and type in some phrases that someone might type into our coffee bot.

Create a new entity, let's call it drink_type, and have it belong to the coffee ordering intent.

Let's also add a milk coffee, or whatever types of coffee you'd like to offer in your store.

Now let's go back to our server, and test out a message.

• Deliver a milk coffee to 12 Ton Dan, District 4, Ho Chi Minh City, at 8am tomorrow, you stupid robot slave.

My server responds like this:

{ intents:
   [ { id: '2965105603595779',
       name: 'Coffee_Ordering',
       confidence: 1 } ],
  entities:
   { 'wit$location:location':
      [ { id: '315900999863727',
          name: 'wit$location',
          role: 'location',
          start: 25,
          end: 65,
          body: '12 Ton Dan, District 4, Ho Chi Minh City',
          confidence: 0.9363,
          entities: [],
          suggested: true,
          value: '12 Ton Dan, District 4, Ho Chi Minh City',
          type: 'value' } ],
     'drink_type:drink_type':
      [ { id: '658420578132111',
          name: 'drink_type',
          role: 'drink_type',
          start: 10,
          end: 21,
          body: 'milk coffee',
          confidence: 1,
          entities: [],
          value: 'milk coffee',
          type: 'value' } ] },
  traits: {},
  detected_locales: [ { locale: 'en_XX', confidence: 1 } ] }

Ah, this is not 100% correct. It seems that we did not get the time properly from this. That's okay, we can re-train our detected entities.

Actually, it's already been suggested. Just click "add", and click "train and validate".

Now update your bot to respond. First, let's go back and change our handleMessage function:

const handleMessage = (sender, message) => {
  if(message.nlp) {
    if(message.nlp.intents) {
      let intent = message.nlp.intents[0];
      if(intent && intent.name === 'Coffee_Ordering' && intent.confidence > 0.6) {
        return handleCoffeeOrder(sender, message);
      }
    }
  }
}

And our handleCoffeeOrder function looks like:

const handleCoffeeOrder = (sender, message) => {
  let location = getEntityFromMessage('location', message)
  let drinkType = getEntityFromMessage('drink_type', message);
  let deliveryTime = getEntityFromMessage('datetime', message);

  sendMsg(sender, {text: `Yes my human master. Your loyal slave will send a ${drinkType} to ${location} at ${deliveryTime}`});
}

We'll just need to make a quick function called getEntityFromMessage.

const getEntityFromMessage = (entityName, message) => {
  let entities = message.nlp.entities;
  switch(entityName) {
    case 'location':
      return entities['wit$location:location'] && entities['wit$location:location'][0].value;
    case 'drink_type':
      return entities['drink_type:drink_type'] && entities['drink_type:drink_type'][0].value;
    case 'datetime':
      return entities['wit$datetime:datetime'] && entities['wit$datetime:datetime'][0].value;  
  }
}

Congratulations! You now have a coffee bot.

Step 4: Conversational Flow

One shortcoming of our bot is that it's not very good at asking for more information.

So we'lll need to build in some conversational logic. Doing this properly is beyond the scope of our lab today. Because we are running serverless, we won't be able to persist memory between requests, and it's recommended we use something like Redis to store our state.

The basic code to implement something like this is below:

const handleCoffeeOrder = (sender, message) => {
  let { location, drinkType, deliveryTime } = globalCoffeeOrders[sender.id] || {};

  location = getEntityFromMessage("location", message) || location;
  drinkType = getEntityFromMessage("drink_type", message) || drinkType;
  deliveryTime = getEntityFromMessage"datetime", message) || deliveryTime;

  globalCoffeeOrders[sender.id] = { location, drinkType, deliveryTime };

  console.dir(globalCoffeeOrders);

  if (location && drinkType && deliveryTime) {
    sendMsg(sender, {
      text: `Yes my human master. Your loyal slave will send a ${drinkType} to ${location} at ${deliveryTime}`,
    });
  } else {
    let text = "I apologize master! Please don't beat me but I need to know:";
    if (!location) {
      text += " the location to deliver";
    }
    if (!drinkType) {
      text += " the type of drink you desire";
    }
    if (!deliveryTime) {
      text += " the time of delivery";
    }
    sendMsg(sender, { text });
  }
};