---
title: 'Zenbo'
disqus: hackmd
---
# Zenbo Project
###### tags:`Course 2021`

## Contents
[TOC]
***
## Introduction
### Sing Up an Account
:::info
If you don’t have any ASUS account, please sign up.

Go to Zenbo Developer website: https://zenbo.asus.com/developer/
:::

#### Go to SDK&Tools -> Choose Zenbo


#### Choose Which Method You Like to Develop for Zenbo


***

## Zenbo Developer

- [ ] Zenbo APP Builder (Module 1)
- [ ] DDE Editor (Module 2)
- [ ] Zenbo SDK (Module 3)
- [ ] Zenbo Story Editor

***
## Module 1 : Main function of the App builder and its application
:::info
### Today’s Agenda:Zenbo App Builder
- Introduction to the Zenbo App Builder

:::
### Introduction Zenbo App Builder
- Zenbo APP Builder is a visual programming tool especially designed for Zenbo
- Zenbo APP Builder features graphical blocks to represent code concepts
- Zenbo APP Builder also features a 3D Simulator, which allows the user to freely create and edit APPs for Zenbo.
### Overview of the App Builder
- Toolbar and Menu Items:
- save the file, open an existing file, start a new file, zoom in/out, start simulations, connecting Zenbo to the App Builder etc.
- Note: To connect the app developer and Zenbo, you must input the IP address as displayed on Zenbo. Click connection -> connect and enter the IP address.
- Left column: Blocks Menu
- access the different block function ( 17 different blocks) which are used to build the script.
- Right column:
- displays Zenbo’s face and physical command (e.g. going forward, backwards, left, right, etc.) when the script is being simulated
### Blocks Menu: What are Block Functions?
- Blocks are the core programming component of the **Zenbo APP Builder**.
- Blocks are divided into 17 categories to represent movement control, facial expression, speech, LED control, sensors, camera module, basic logical expression, and so on
- Block functions can be describe as **“puzzle pieces”** where users piece together different blocks to create a “story”.
- You can combine and connect these blocks to create your own unique App.
- Different blocks have different command and the script will run from beginning to end base on the commands.

### Getting to know Blocks

- There are 5 different kinds of Block functions:
- Start Blocks
- Intermediate Blocks
- Independent Blocks
- Compound Blocks
- Variable Blocks
(We will focus on the first four blocks)

- **Start** – The start block is where you will start building your code/script.
Start Block – Red looking block saying start

- **Intermediate Blocks** – The main blocks you will be using to build the script. Blocks with small triangular dent on the top side and a small protruding triangle underneath.
Intermediate blocks - Blocks with small triangular dent on the top side and a small protruding triangle underneath

- **Independent Blocks** – Blocks that will run when the action happens (e.g. when face detected, when event is received, etc.)
Independent – No triangle indent, separate from the start

- **Compound Blocks** – Similar to intermediate blocks but additional blocks can be put in (e.g. Repeat 5 times, play media and dance
Compound blocks – An intermediate block can be put inside it.

### Using Blocks: Important to know!
- **Blocking vs. Non-Blocking**
- **Blocking**: Run the next block after the current block is finished -> sequentially
Example: Run the head and body movement sequentially when set to blocking.

- **Non-Blocking**: Run next block as the current block is starting ->simultaneously
Example: Run the head and body movement simultaneously when set to non-blocking.

### Main Blocks you will use
(We will not focus on Commands/ Prompts today as this is more advanced knowledge)

### Zenbos’s Physical Movements
- Neck Movement
- Control of Zenbo’s head by turning the neck vertical or horizontal as well as determining how fast it will turn.

- Body Movement
- Move Zenbo forward or backward

- Body Rotation
- Turn Zenbo left, right or any angle you want

- Face
- Here you can change Zenbo’s emotions

- Built in Movements
- They allow you to use some of the default actions that was built into Zenbo.
- Some of the built in movement ranges from nodding to more complex moves such as dancing and head bobbing to music.
- With the built in movement, you can set the duration of the movement.

### Speech, Audio, Visual Blocks
- ++Speech++
- Speech is text to speech in which you type what you want Zenbo to say. These are called “strings”.
- One of the speech block allows you to input 3 strings,. Here Zenbo will randomly say 1 of the 3 strings. This is effective for different uses such as greetings (e.g. hello, hi, how you are).

- ++Audio++
- Audio function/block allows the user to record a sound that will be saved into Zenbo.
- Function are to record, play recorded sound, stop recording and stop playing recorded sound

- ++Multimedia: divided into 2 categories++
- **“Play music with image”** block will allow you to display an image and play music at the same time. Additional feature here: you can play the song at a given start time (in second) which you can set. You can again choose Blocking or Non-Blocking (refer to slide 19/20)
- **“Play media”** block will allow you to either play a video or display an image. Only one media source will be played. Blocking or Non-Blocking available.

### Camera, Detection
- ++Camera++
- Here you can take a picture or record a video. Later, the taken picture can also be shown and the recorded video can be played.
- You can set a time duration as to how long the picture/video will be shown (in seconds).
- Pictures/ videos taken/recorded will be stored in the internal memory.
 

- ++Detection++
- Detection blocks are special as they are independent blocks (blocks that will activate and run when a certain thing happens).
- “*When face is detected*” is the most common detection block. When **Zenbo sees a person’s face, it will run the script**.
- “*When hear*” is another block that is commonly used. It runs the script when it hears a specific command.
- “*When not in charge*”, “*when in charge*”, and “*when battery is at a certain %*” are other detection feature, but rarely used.
 

***
## Module 2 : Introduction to Asus Zenbo’s DDE Editor
### What is DDE?
- Make the process of creating and integrating conversational interfaces easier.
- Use web interface to edit/verify conversational dialogue quickly.
- Ties into the Zenbo App builder in which users can develop a smart conversation app.
- The DDE is an extension of the app builder where Zenbo will reference the DDE to see if it hears that speech and once it does, it would give a fixed reply based on that.

### Example of How DDE Works

### Getting Started
- Assuming that you already have a Asus account, anything you’ll save and do on the DDE editor will be saved to your account.
- If not, you can register at https://account.asus.com/signup.aspx
- An existing Google account or Facebook account can be used to login but we do HIGHLY SUGGEST creating an Asus account using your email.
- Once your account is created, you can access all the Zenbo Developer Tools at https://zenbo.asus.com/developer/tools/

### Setting Up to use the DDE (potential browser issue) - FireFox
- If you run into the trouble where your web browser blocks your connection, here is the solution to allow you to access it (FireFox)
- If you run into the trouble where your web browser blocks your connection, here is the solution to allow you to access it (Chrome)

### Project Setup
- Note that this setup is only for the Zenbo App Builder used on the web browser and not the Zenbo SDK for Android Studio. Follow the red arrows :)
- To begin, please head to https://zenbo.asus.com/developer/console/ or click console on the Asus homepage
- Create a new app in the App Management
- You can name your App Name, Package Name, and Brand Name however you like but try to keep it consistent for simplicity. (Blue underline)
- Once done, click submit to continue.
- You should now see your TestApp. Click it to view its details for the next step.

### Info to Remember
- You will now see that an App ID and App Key has been generated.
- It is key that you note down these two information below:
- Package name (App Builder Prefix Package Name)
- App ID
- The information can be found by clicking on the app file that you’ve just created and looking for ones circled.
(for those who skipped the last slide)

### Creating a New Project (1.1)
- Opening up the DDE editor at: https://zenbo.asus.com/developer/tools/
- Follow the steps below:
1. Open DDE Editor.
2. Click “Create A New Project”.
3. Click “Edit”.
4. Type in a name for your new app and click “Save”.
5. On the top left corner of the DDE editor page click “Info”.
Note down the Domain Version #0.0.1

### Creating a New Project (1.2)
- Type in a name for your new app and click “Save”.
- On the top left corner of the DDE editor page click “Info”.

### Creating a New Project (2)
- After you clicked on “Info” as per the previous slide, you will need to create a DDE Domain. The steps are as below:

1. Where it says “DDE Domain ID”, click “Register”.
2. Register a Domain Name and Description then save.
3. Once registered, click “All” this time to view the details of the new domain.
4. Remember the UUID of your domain.
##### Step 1. Step 2.
##### Step 3. Step 4.

### Stop: Things to Remember/Note Down
**IMPORTANT!!!**
What you should have note down by now is the below:

1. Package name (App Builder Prefix Package Name) - Slide 11
2. App ID - Slide 11
3. Domain Version # - Slide 12
4. UUID of the project - Slide 14

If you don’t, go back to the previous slides to see where to find them. It's important you have these information going forward from this slide.

### Creating a New Project (3)
Before saving the Project Info, make sure the following are:

1. Developer App ID is selected
2. Register Background Context to True
3. Machine Learning to True
4. Apk or Skill set as apk
5. Save the Project Info.

### Creating a New Project (4.1)
- In the DDE Editor, click on Intents. This is where you can create phrases or sentences for Zenbo to recognize.
- The Intent ID can be anything but remember try to keep it simple :)
- For demonstration purposes the default Intent has been modified, but you can change “Intent ID” and “Sentences” to your liking.
- Keep “Languages” to All but make sure your sentences are only in either:
- English;
- Traditional/Simplified Chinese;
- Japanese
### Creating a New Project (4.2)
### Creating a New Project (5)
- Similar to the last steps, go to Plans to modify the Plan ID and Input Context to your liking (something simple/consistent).
- The “Is the plan able to launch the app?” needs to be set to FALSE before being able to put a Input Context.
- From your events, choose any of the Intents you may have created.
Note down “Plan ID” and “Input Context”
### Creating a New Project (6)
- All we have to do now is deploy it to the download server, but firstly on the top right of the DDE Editor:
1. Save the Project
2. Publish the Project
- NOTE! There may be a SLU Data Validation warning, but you can Continue to Publish
3. Click “Dev” at the top Left of the DDE Editor and “Deploy to Download Server”

### Zenbo APP Builder (1)
- Back to where we initially open the DDE Editor, this time we can now start to build our app with the Zenbo APP Builder!
- If you’ve forgotten the link:
- https://zenbo.asus.com/developer/tools/
- By now, in addition to what you have noted down from slide 15, you should also have noted down the “Plan ID” and “Input Context” from slide 19.
- We will now put incorporate those values into our app.
### Zenbo APP Builder (2)
- To create the while loop, use the blocks from:
- Loop, Logic, and Others
- To create the speech detection, use the blocks from:
- Detection, and Speech
- You can have Zenbo say what you want but the sentences/phrases you created in intent is what will trigger the speech.
- Put the Context and Plans Id that you created in slide 19
### Zenbo APP Builder (3)
- Before we can simulate this, we need to save this. On the top left, click “File” then “Save”. This is where we will use our values from slide 15.
- Click OK once you’re done which will download the ZBA file to your PC.
### Zenbo APP Builder (4)
- To run the app, press the play button then “Simulate”. On the right where it says “Talk to Zenbo” you need to say one of the following for Zenbo to react (indicated by the blue side lights):
- Hey Zenbo
- Hi Zenbo
- Hello Zenbo
  

### Zenbo APP Builder (5)
- Following Zenbo’s reaction with “Hey Zenbo”, you can input the intent sentences/phrases that you have set up previously.
- If Zenbo’s speech triggers then you have successfully completed this project.
 
***
## Module 2-1 : Zenbo DDE and AppBuilder Tutorial
### Add Intent
1. Set Intent ID
2. Set Language to “<font color=red>++**All**++</font>”
3. Add Sentences


### Add Intent (cont.)

### Add Plans
1. Set <font color=red>**++Plan ID++**</font> and <font color=red>**++Input Context++**</font> (You will use both later).
2. Choose <font color=red>**++False++**</font> in “Is the plan able to launch the app?”
3. Choose the specific <font color=red>**++Intent++**</font> in Events.


 

### Deploy Your DDE
1. Save the Project.
2. Publish the Project.**NOTE!** There may be a SLU Data Validation warning, but you can Continue to Publish.
3. Click “Dev” at the top Left of the DDE Editor and “Deploy to Download Server.”


### Zenbo App Builder
1. Set your block coding.
2. Set Context and Plans Id.


### Before Simulation
1. Before we can simulate the blocky codes, we need to save them. On the top left, click “File” then “Save.” This is where we will use our values from the previous slides.
2. Click OK once you’re done which will download the ZBA file to your PC.


### Simulation
1. To run the app, press the play button then “Simulate.” On the right where it says “Talk to Zenbo,” you need to say one of the following for Zenbo to react (indicated by the blue side lights).
2. Following Zenbo’s reaction with “Hey Zenbo,” <font color=red>**++you can input the intent sentences/phrases that you have set up previously.++**</font>
3. If Zenbo’s speech triggers then you have successfully completed this project.
  

### Simulation (cont.)
1. To run the app, press the play button then “Simulate.” On the right where it says “Talk to Zenbo” you need to say one of the following for Zenbo to react (indicated by the blue side lights):
- Hey Zenbo
- Hi Zenbo
- Hello Zenbo
- When the blue lights go on like ears, it means Zenbo is listening.

1. Following Zenbo’s reaction with “Hey Zenbo,” you can input the intent sentences/phrases that you have set up previously.
2. If Zenbo’s speech triggers, it means that you have successfully completed this project.


### Advanced Skills
- Scenario: After choose body checkups, you want to ask which type of body checkups user want to do.

### Add More Intent
1. Add a new Intent.
2. Right click on ‘body temperature,’ and click ‘<font color=red>**++create a new concept++**</font>’ and set Concept name.
3. Concept is like a category, you can add similar words into the same concept, for example, body temperature and blood pressure.

### Set Belief
1. Click the orange block and click "Set Belief Name."
2. Belief is like a variable in which you can save your selection, but your selection need to be in the concept.
***
1. Click Concepts.
2. Click New Instance and set new instance for blood pressure.
3. Blue tags refer to all the synonyms of an Instance. For example, body temperature, temperature, etc.


### Connect two Plans
1. Click Plans and choose Plan: body.
2. Click <font color=red>**++New action++**</font> and set the Output Contexts.

### Connect two Plans (cont.)
1. Add a new Plan
2. Set <font color=red>**++Input Context++**</font> be the same <font color=red>**++Output Context++**</font> in the previous slide.
3. DDE uses input context and output context to connect different plans.
4. Add New Belief, choose yes and select your variable name and classes.


### Deploy your DDE
1. Save the Project.
2. Publish the Project.**NOTE!** There may be a SLU Data Validation warning, but you can Continue to Publish.
3. Click “Dev” at the top Left of the DDE Editor and “Deploy to Download Server.”


### Zenbo App Builder
### Zenbo App Builder (cont.)
1. To create more events in the ListView UI, just move the item block into the list block.
 

### Simulation

### Simulation (cont.)

***
## Module 3 : Zenbo SDK with Android Studio
:::info
### Today’s Agenda:
#### Zenbo SDK
- Today, we will introduce Android Studio.
- We will learn how to use Android Studio in combination with the DDE Editor
- We will also discuss what Zenbo SDK is
:::
### Android Studio: Overview
- Android Studio will be the interface of your Zenbo App.
- Android Studio allows us to program Zenbo.

### DDE Editor – Android Studio
- Android Studio can get the Grammar Structure given by DDE Editor, and thus can use this information to do custom actions based on the Grammar Structure spoken by the user.

### Zenbo SDK: What is that?
- Zenbo SDK is a library and example Java source code that allows you to program Zenbo through Android Studio.
- Android Studio uses all the APIs from Zenbo SDK.
- Zenbo is based on Android M. You can use Android Studio or other IDE for android developing. This allows more flexibility in programming Zenbo compared to the Zenbo App Builder, developers can use callback within the architecture.
- General Callback
- Dialog System Callback
- Vision Callback

### So how can we use Android Studio? What do we need?
- Android Studio
- You can download Android Studio for free. Just go tho the following website https://developer.android.com/studio/?gclid=EAIaIQobChMIjtaJ4-zK5AIVFq6WCh0bXgDcEAAYASAAEgKp-vD_BwE

- Zenbo SDK (you need a ASUS Account in order to download)
- You can download Zenbo SDK from this link: https://zenbo.asus.com/developer/tools/

### After Downloading Zenbo SDK
- After you extract the downloaded zip file (2019 July) , you should have 2 new zip files inside. Extract the one that is the studio_3.4.1 version to anywhere on your computer. (Extract to desktop is recommended.)
- Now you should have a new folder which contain:

### Zenbo SDK – Android Studio
- Inside the extracted folder you will notice a folder called “ZenboSDK”, **++move++** that folder out from it’s original folder to somewhere (Desktop) so you can easily refer to it later. We use it in a bit. \*IMPORTANT*

- Run Android Studio; select “Open an existing Android Studio project” and choose the folder you extracted earlier.(ZenboSDK_and_SampleCode_v1.0.72.1826_for_studio_3.4.1)

- Next, in the top left again in “File” > “New”, import the ZenboSDK as a module in Android Studio.
 
- You are now ready to create a new module which will be the app you are creating for Zenbo. Just like the last slide, instead of importing a module, select “New Module”.
- The Minimum SDK must be API 23: Android 6.0 (Marshmallow).
- The Package Name should be the same as the one in “DDE Editor -> Project Info” and the “App Console”.
- Now that you have your new module, edit the build.gradle file in the module directory to include the ZenboSDK and RobotActivityLibrary and then re-sync the gradle files.
- On line 35 and 36 of your build.gradle, add these two lines:
- implementation project(':ZenboSDK')
- implementation project(':RobotActivityLibrary')
- Once you made changes, you must click “Sync Now” to sync all the project files according to the new changes.
- Modify these lines and add to your AndroidManifest.xml
- You need to change the "Domain UUID" and the "Version Value (found in DDE Editor -> Project Info)“

<meta-data android:name="zenbo_ds_domainuuid" android:value=“[DomainUUID]"/>
<meta-data android:name="zenbo_ds_version_[DomainUUID]" android:value=“[VersionValue]"/>


### Launching Zenbo App using Speech
- In your DDE Editor -> Project Info -> Launch Activity. Enter the Activity name of the App that you want to be launched using Speech.
Android Studio 
DDE Editor 
- In your DDE Editor, you should have one “Plan” that “Is able to launch app”.
 
- In this example, in Zenbo Mode (where you see Zenbo’s Face), saying “night market” or “Feng Chia night market” will launch the “Main Activity” of the App.

***
## ACCESSING ZENBO：A GUIDE TO CONNECTING WITH ZENBO

### Important tips before beginning

- When you connect Zenbo to the internet, **BE SURE** that your laptop is connected to the same network as Zenbo or else the app builder will not connect.
- Ensure that a steady connection can be established. Multiple devices/laptops can be connected to Zenbo at the same time.
- If in the case that you cannot connect Zenbo to the app builder, the second option is the put the ZBA file in a USB and import it into Zenbo via Zenbo AppBuilder.

### Connecting Zenbo to the Internet and app builder
- From the app menu, click on “settings” and then choose the Wi-Fi network that you’ll be connecting Zenbo to.
- After establishing a network connection, go back to the app menu and select “Zenbo AppBuilder” (shown below).


- On the AppBuilder page, this is where the IP address is located (red circle).
- Import app (on the left) is where you would import a ZBA file into Zenbo via a USB (insert on top left of head).
- Existed apps is where existing apps that were imported to Zenbo or previously connected to Zenbo will be stored.
- Note down the IP address as you will need to type it into the app builder toolkit later.

### Zenbo IP Address


### Connecting the AppBuilder to zenbo

- Referring back to the slide in lecture 2, you would have to click to the “disconnect” and input the IP address for Zenbo that you’ve noted down.
- When you do so, there will be an error that pops up and what you would do is that you would accept the risk and then refresh the page. The video that shows you what to do is on slide 9.


{%youtube siPZo3MKFSg%}

### If the app builder cannot connect

- In the case where you cannot connect the AppBuilder to Zenbo, a second alternative is to import your ZBA file into Zenbo.
- After saving the file, put it on a USB and connect it to Zenbo (image as below)


- Once inserted, click on “import app” (refer to slide 5) and upload your file from your USB into Zenbo


### Finding existing apps

- Referring back to slide 5 on the right side where it says “existing apps” click on that and your apps and previous apps that were loaded onto Zenbo will be saved there.


### All Done !

- These are the basics on how to connect to Zenbo.
- If you have any questions, feel free to ask your professor and assistant for assistance.

***

## Feng Chia Night Market Zenbo Application

### Open Zenbo Backplate
- Zenbo Backplate is connected using 4 Magnetic points. There is no other hooks or locking mechanism.


- To open the backplate, simply pull the backplate off by wedging the edge and then gently lifting backwards.


- You will find the USB Debugging Port at the side of the head.


- Before connecting to your Android Studio SDK. You need to enable USB Debugging on Zenbo.

 

- After enable USB Debugging, you should see this in your Android Studio. Turning from [OFFLINE] to [ASUS Zenbo]

 

***