# Docs: Handover by Jitesh
Much of the work I have completed can be found documented in the README files within the respective repositories or on this webpage. If the Person in charge for overseeing the project (excluding myself) is unavailable for clarification, please do not hesitate to reach out to me at **[e-mail]**.

[toc]

## 1. Projector (for IST and Smalltalk Demo)
3 parts are used together.

### 1.1 Python [strawberry-ros-projector](https://github.com/haru-project/strawberry-ros-projector)
* Person in charge: Lithin, Matija
* Description: Here we are subscribing to the dialog_result or other rostopic with converation information, convert that data to a json output and post on localhost with port 1706.
* http://localhost:1706/ For json data
* http://localhost:3000/ For Frontend app that shows chat info
* You can show the Frontend on Unity by Press `CTRL` + `w` keys or by publishing the above URL on rostopic `/unity_projector/conversationApp/set`
* Remarks: Lithin already make changes on the info that needs to be extracted from rostopic and he post it with other info.

### 1.2 React - js [strawberry-react-projector](https://github.com/haru-project/strawberry-react-projector)
* Person in charge: Matija, Lithin
* Description: This is the frontend of the projector which let us start displaying text between Haru and User in a browser.
* Remarks: We don't need any changes, but if we do, make its build, rename it to FRONTEND and replace it on the Python repository `strawberry-ros-projector`

### 1.3 [Haru-Unity](https://github.com/haru-project/haru-unity)'s projector-canvas
* Person in charge: Domingo
* Description: This part opens a browser inside unity app and then go to the url created by frontend.
* Remarks: We don't need any changes

## 2. Telegram Bot (harumoji, telemoji, den-den-mushi)

* Person in charge: UPO, Jose Manuel, Javier (both)
* Description: This demo let's anyone with Telegram app send a message to one of the Haru, if the bot-server is running on Haru.
* For now, we have made few bots which can run on one server at a time. If multiple server tries to run same bot then uncertain things will happen until all servers are shutdown. Current List of bots have the following username:
* `haru_japan_bot` : For Japan use
* `haru_spain_bot` : For Spain use
* `Haru_demo_bot` : For very important Demo use
* create your own bot with the help of `BotFather`
* Remarks: Just follow the README file of [idmind-tabletop-telegram-bot](https://github.com/haru-project/idmind-tabletop-telegram-bot)

## 3. Encouraging Mediator (target scene C, talking-room, Teleconference demo, classroom)

* Person in charge: Sara Cooper, UPO, Nabil, Matthew, Jose Manuel
* Useful Links:
* [My Notes](https://hackmd.io/@jitesh/em)
* [Demo steps information](https://docs.google.com/spreadsheets/d/13tUHjGnjESW9kcTUPgptH4PUCOCwH42rmsdwFcTgOBA/edit?usp=sharing)
* [Technical Documentation](https://docs.google.com/document/d/1iK73WO-o5BN1JNPcMj8eHCm4wtiuC6LbBf2jzwwOsXc/view#)
* [Collaboration SpreadSheet](https://docs.google.com/spreadsheets/d/1OEW5JdYU7L78y0oz1V_xd_WWo-O8CqgqvvvzAyP13Xk/edit#gid=0) for sharing IP address and hostnames

## 4. Ros services in [`haru-nlp-ros`](https://github.com/haru-project/haru-nlp-ros)

### 4.1 [Keyword Matcher](https://github.com/haru-project/haru-nlp/blob/develop/src/haru_nlp/keyword/match_keyword.py)
* Person in charge: Lithin, Levko, Eric
* Description:
* [KeywordMatcher](https://github.com/haru-project/haru-nlp/blob/develop/src/haru_nlp/keyword/match_keyword.py) class uses spacy.matcher method to check if a list of keywords exist in an input sentence and outputs the matching keyword.
* It is already [used in](https://github.com/haru-project/haru-nlp-ros/blob/develop/src/haru_nlp_ros/text_to_ssml_server.py#L96) `Text to SSML service`

### 4.2 Text to SSML service [`/nlp/text_to_ssml`](https://github.com/haru-project/haru-nlp-ros/blob/develop/src/haru_nlp_ros/text_to_ssml_server.py#L107)
* Person in charge: Lithin
* Desciption:
* It creates a service which takes text as an input, analyse its emotion and map a voice-genre and routine for that emotion. if the emotion is neutral then it checks for a keyword and maps the keyword to voice-genre and routine. Finally the service outputs the text with emotion-genre and routine-id.
* The service name is `/nlp/text_to_ssml` with type `TextToSSML`
* For additional routine mapping, update [ROUTINE_MAPPING](https://github.com/haru-project/haru-nlp-ros/blob/develop/src/haru_nlp_ros/text_to_ssml_server.py#L47) here.

### 4.3 SSML service (for dialog_result) [`/nlp/create_ssml`](https://github.com/haru-project/haru-nlp-ros/blob/develop/src/haru_nlp_ros/ssml_server.py#L40)
* Person in charge: Lithin
* Desciption:
* It creates a service which takes genre-mapping as an input. As soon as the new message in dialog_result comes, the service extracts useful info like text and emotion, and map a voice-genre and routine for that emotion. Finally the service outputs the text with emotion-genre and routine-id.
* The service name is `/nlp/create_ssml` with type `GetSSML`

## 5. Haru Virtual World
* Person in charge: Rabin
* [Dialog System - Light](https://github.com/haru-project/unity-dialog/blob/main/scripts/dialogflow_api.py)
* [Dialog System - Light: Usage example](https://github.com/haru-project/unity-dialog/blob/main/scripts/dialog_server.py)