Try   HackMD

All Haru Demos

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

Status Report

Demo Name HRI UPO Fix needed Feature needed
Overall
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Update routine on the robot, run lip sync separately if needed
Chatty
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
update routine id 1024
IST
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Haru Games
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
updates dialog and banner
Telemoji
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
voice notes not work on some pc

Setup the external PC

To work with the real robot,

  • update the .bashrc 
    ​​​​### Use Real Haru
​​​​export ROS_MASTER_HOSTNAME="haru-47"
​​​​export ROS_MASTER_URI=http://$ROS_MASTER_HOSTNAME:11311/
​​​​export MY_IP=$(ip route get 8.8.8.8 | awk -F"src " 'NR==1{split($2,a," ");print a[1]}')
​​​​export ROS_IP=$MY_IP
​​​​export ROS_HOSTNAME=$ROS_IP
  • Add the robot-ip and robot-id/etc/hosts (example given) 
    ​​​​192.8.8.8 haru-47

Telemoji (Telegram bot)

haru_japan

6137143631:AAHLv1Ceq3ACpm4rHqKgGZ5htWoYS9CPHxw

Setup

  • Git clone this repos in a new workspace (for example telegram_ws) on external-pc and on catkin_ws workspace on robot
    1. strawberry-ros-msgs/
    2. idmind-tabletop-ros-msgs/
    3. idmind-tabletop-ros-pkg/
    4. idmind-tabletop-lip-sync/
    5. idmind-tabletop-telegram-bot/
    6. haru-nlp-ros/
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
  • Run bash postint inside haru-nlp-ros directory
  • Run catkin_make inside telegram_ws directory

Run

source ~/telegram_ws/devel/setup.sh
rosrun idmind_tabletop_telegram_bot run_demo.sh

Run (fallback)

If the above command fails, then run all the launch separately

  • Run each command on new terminal after sourcing the telegram_ws
    • source ~/telegram_ws/devel/setup.sh
  • Run the following commands in the Robot if using Real robot otherwise run it on the external PC 
    ​​​​roslaunch idmind_tabletop_telegram_bot telegram_haru.launch
  • Run the following commands in external PC 
    



    ​​​​
​​​​roslaunch idmind_tabletop_lip_sync lip_sync.launch
​​​​roslaunch idmind_tabletop_telegram_bot telegram_haru_testbed.launch
​​​​roslaunch idmind_tabletop_telegram_bot telegram_haru_behaviour_tree.launch

IST

Setup

  • Install dependencies 
    

    ​​​​cd ~/ist_ws/src/haru_ist
​​​​bash ./scripts/install_dependencies.sh
  • change all the repos to develop branch in the workspace
  • Comment out this lines
  • Build 
    ​​​​catkin_make

Run

  • Turn on the robot or the simulator
  • Then run the below command that launches everything needed for the demo 
    ​​​​rosrun haru_ist run_ist.sh

Run (fallback)

If the above command fails, then run all the launch separately

  • Run each command on new terminal after sourcing the ist_ws
    source ~/ist_ws/devel/setup.sh
  • Run
    • If you are not running tts through testbed then run it with the following 2 commands 
      

      ​​​​​​​​roslaunch idmind_tabletop idmind_tabletop_audio_controller.launch
​​​​​​​​roslaunch idmind_tabletop idmind_tabletop_tts.launch
    • Always run the following
    

    ​​​​cd ~/ist_ws/src/haru_ist/haru_ist_model
​​​​bash start_model.sh

    ​​​​roslaunch haru_ist launchMyBT.launch

Chatty

Setup




cd ~/chatty_sdk
sudo chmod a+x ~/chatty_sdk/demos/conversation_follower/follower.py
python3 -m pipenv install --deploy

Run

  • Turn on the robot or the simulator
  • Run each command on new terminal after sourcing the chatty_ws
    • source ~/chatty_ws/devel/setup.sh
    




    ​​​​roslaunch idmind_tabletop idmind_tabletop_audio_controller.launch
​​​​roslaunch idmind_tabletop idmind_tabletop_tts.launch
​​​​roslaunch idmind_tabletop_lip_sync lip_sync.launch
​​​​rosrun behavior-tts temp.py
​​​​roslaunch idmind_tabletop_marker_sync idmind_tabletop_marker_sync.launch

    

    ​​​​cd ~/chatty_sdk
​​​​python3 -m pipenv run demos/conversation_follower/follower.py -H -d demos/conversation_follower/dialogues/harufest.xml

Haru Games

Setup

Here's how to set it up:

  1. delete the ~/games_ws folder
  2. run mkdir -p ~/haru_games_ws/src
  3. go into that folder and run git clone https://github.com/haru-project/haru-games-demo.git
  4. run cp -r ~/ist_ws/src/idmind-tabletop-ros-pkg ~/haru_games_ws/src/
  5. run cd ~/haru_games_ws/src/haru-games-demo/
  6. run bash scripts/install_dependencies.sh
  7. run bash scripts/post_clone.sh
    This demo requires additional resources so please unpack them into ~/haru_games_ws/src/idmind-tabletop-ros-pkg/resources . it should create a folder called images in the resources folder and a folder called haruscope in the projector folder.

Run

  • Turn on the robot or the simulator
  • Then run the below command that launches everything needed for the demo 
    

    ​​​​source ~/haru_games_ws/devel/setup.sh
​​​​rosrun haru_games_demo run_games.sh

Run (fallback)

If the above command fails, then run all the launch separately

  • Run each command on new terminal after sourcing the haru_games_ws
    source ~/haru_games_ws/devel/setup.sh 
    

    ​​​​roslaunch haru_games_demo backbone-advanced-noisefix-external-mics.launch
​​​​roslaunch strawberry_ros_dialog strawberry_ros_dialog.launch
  • wait until this says it's ready, if you're not sure you can run rostopic echo /strawberry/dialog_ready and wait until it publishes 
    ​​​​roslaunch haru_games_demo haru_games_bt.launch

Appendix

  1. Download haru_unity_app
  2. Controll all the repostories in a workspace using 1 command
    • Append the following in your .bashrc 
      













      ​​​​​​​​all-git-branch() {
​​​​​​​​    local base_path="${1:-.}"
​​​​​​​​    find $base_path -type d -name '.git' -exec echo {} \; -exec git -C {} branch \;
​​​​​​​​}

​​​​​​​​all-git-pull() {
​​​​​​​​    local base_path="${1:-.}"
​​​​​​​​    find $base_path \
​​​​​​​​    -maxdepth 2 -type d \
​​​​​​​​    -name ".git" \
​​​​​​​​    -execdir python -c 'import os; print(os.path.abspath("."))' \; \
​​​​​​​​    -execdir git branch \; \
​​​​​​​​    -execdir git pull \;
​​​​​​​​}
    • Usage
      • To get the branch name all the repositories in [PATH-TO-PULL], default is current directory path 
        ​​​​​​​​​​​​all-git-branch [PATH-TO-PULL]
      • To pull all the repositories name in [PATH-TO-PULL] 
        ​​​​​​​​​​​​all-git-pull [PATH-TO-PULL]
Last changed by 
 
Jitesh Gosar
0
294

Read more

Conda

Activate Conda for the Current Session: Use the command provided in the installation message, but replace YOUR_SHELL_NAME with the name of your shell (e.g., bash, zsh). For example, if you're using bash:

Mar 5, 2024
Untitled

Health and Safety

Dec 21, 2023
Docs: Handover by Jitesh

Much of the work I have completed can be found documented in the README files within the respective repositories or on this webpage. If the Person in charge for overseeing the project (excluding myself) is unavailable for clarification, please do not hesitate to reach out to me at gosar95@gmail.com. 1. Projector (for IST and Smalltalk Demo) 3 parts are used together. 1.1 Python strawberry-ros-projector Person in charge: Lithin, Matija Description: Here we are subscribing to the dialog_result or other rostopic with converation information, convert that data to a json output and post on localhost with port 1706.http://localhost:1706/ For json data http://localhost:3000/ For Frontend app that shows chat info You can show the Frontend on Unity by Press CTRL + w keys or by publishing the above URL on rostopic /unity_projector/conversationApp/set

May 18, 2023
Encouraging Mediator

Demo steps information Unity Doc spreadsheets Run Commands Setup rosrun haru_encouraging_mediator setup.sh Run the app roslaunch haru_encouraging_mediator master.launch number_of_ipads:=3

May 11, 2023
Read more from Jitesh Gosar
Published on HackMD