# 2021 Introduction to Machine Learning Final Project
## Topic
* Project Name : **Ream-Time Quadcopter Controled by SSVEP-BCI**
* Key functions(add/delete)
1. Set up private chatrooms.
2. Add people in your chatrooms.
3. Look for users in public room.
* Other functions (add/delete)
1. Construct your User Page
2. Send Private messages
3. Contact Service
## Introduction
Brain-computer interfaces (BCIs) build a direct communication pathway between the human brain and the external device without the involvement of the peripheral nerves and muscles . Steady-state visually evoked potential (SSVEP) is a periodic neural response to the repetitive visual stimulation, which occurs in the visual cortex at the flickering frequency and its harmonics. Currently, the SSVEP-BCI has reached a high information transfer rate (ITR) and is the fastest BCI paradigm.
The improvement of BCI has prompted the researchers to develop real-time BCI-actuated robotic systems include wheelchairs, prosthetics, and other assistant systems for both healthy and disabled individuals . Quadcopter control based on BCI has attracted the attention of many researchers. He et al. firstly proposed the idea of a brain-controlled quadcopter system and developed a two degree-of-freedom (DOF) AR quadcopter control system based on Motor-imagery-BCI[1-3]. Shi et al. investigated the semi-autonomous quadcopter control by using an MI-BCI for indoor 2-DOF target searching [4].. MI-BCI is applied for quadcopter control because it is closer to the user's real perception of motion. However, the command set of an MI-BCI is small, and long-windowed EEG data are required for accurate control. Therefore, the DOF and flexibility of the quadcopter control using an MI-BCI are limited. Duan et al.applied multi-modal BCI that combined MI and SSVEP and used eye-blink for the switch to control a quadcopter with two DOF [5]. Wang et al. proposed an SSVEP-based system for quadcopter control with a head-mounted device used for the visual interface, which again had two DOF [6]. However, the SSVEP paradigm in [5,6] is synchronous, in which the rest between trials was required to align the stimulus onset and phases. During the resting period, the users can only shift their gaze without controlling the quadcopter. That is, the users could not control the quadcopter in real-time and continuously.
To remove these limitations, this project developed an SSVEP-BCI to continuously control a quadcopter with 4-DOF actions. The user interface incorporates 6 continuous SSVEP flickers. The SSVEP stimuli flicker without rest so that users can send the flight commands detected by the BCI at any time and freely move their gaze.The commands are updated every 0.4s. Four latest commands define a control vector that determines the flight status of the
quadcopter.
## Data Collection
**What is EEG Data :**
When the human brain is in operation, the nerve cells that make up the brain will continuously discharge electricity, and scientists use brainwave instruments to measure the weak electrical or magnetic changes on the human scalp. The brain wave measurement is done by placing electrodes on the scalp and recording the weak bioelectricity generated in the human brain through the medical brain wave instrument to obtain a curve.
The electrogram is generally obtained by means of electrodes on the surface of the scalp. The mechanism of scalp potential generation is generally believed to be: at rest, the whole cell of the cone cell's apical dendrites-cytosol axis is in a polarized state; when an impulse is transmitted to one end of the cell, it causes the end to reverse polarization, and then the potential difference between the two ends of the cell can generate a bipolar electric field system, and current flows from one end to the other. Since both the cytoplasm and the extracellular fluid contain electrolytes, the current also passes outside the cell. This current activity can be recorded using the scalp electrodes. In fact, the potential changes in the scalp EEG are a combination of many such bipolar electric fields. The EEG does not reflect the electrical activity of a single nerve cell, but rather records the sum of the electrical activity of many groups of nerve cells in a region of the brain represented by the electrodes.
>To read more about EEG application:https://www.youtube.com/watch?v=DXX3qIcMdIA
>~~地表最強馬投顧上線啦~~
**Collection Way :**
Below Figure shows the overview of the process for gathering bio-potential signals in SSVEP experiments. Firstly electrodes will be put onto the scalp based on the position needed. This is to capture their relevant brainwave activities. An EEG device will then transmit the bio-signals to the PC for recording. Another PC is then used to present visual stimulus to excite the respective signal for the subject. Bio-signal data were collected over dry electrodes according to the International standards. Cz was used as the reference. The impedances of the electrodes are ensuring to be below one threshold throughout the experiment.
**Data Collection Software : Cygnus**
**Data Collection Instruments : Electrode Cap**
Electrode cap are used to assist with electrode placement, making it easier to affixing electrodes to the scalp. Caps ensure that electrodes are placed precisely and maintain sufficient contact with the scalp.
**Collection Process: **
1. Recess dry electrode which chosen for the
3. McHale
4. Parish
5.
## Basic Components
|Component|Score|Y/N|
|:-:|:-:|:-:|
|Membership Mechanism|15%|Y|
|Firebase Page|5%|Y|
|Database|15%|Y|
|RWD|15%|Y|
|Topic Key Function|20%|Y|
## Advanced Components
|Component|Score|Y/N|
|:-:|:-:|:-:|
|Third-Party Sign In|2.5%|Y|
|Chrome Notification|5%|Y|
|Use CSS Animation|2.5%|Y|
|Security Report|5%|Y|
# Website Detail Description
以下根據每項功能做說明：
## 1. Private Rooms
作法：每個使用者登入後，可以到private room的html網頁，
進行"Add Room"的動作，當使用者輸入**合法roomName**後，
按下 Add Room 的button，在
:::info
Uses/ {user.displayName}'s rooms /
:::
路徑會push一筆data，為 **{ room: roomName }**
同時會在同個paht存下這個roomName的push.key
作為 **未來要Add members時的主人**依據
我的設定為： **只有創建房間者**才能加別人。
### 如何加其他人進來?
呈上題：進入private room後，只需要按右上角的 "Add people" button
就可以輸入 **username** 來加入別人進來聊天。
在加入別人時，在**別人**的's rooms/下也會push這個房間的名稱
因此這個**別人**在private room setting的地方，會多出一個房間的按鈕可以進入。
至於database.rules的部分沒有更改，我只透過database讀寫順序來控制聊天室的存取權限。
## 2. Send Messages
為了製造**聊天泡泡**，上網查了許多範例，最後在`offcanvas.css`撰寫
`.bubble-right` 跟 `.bubble-right`兩個class，成功製造出聊天室泡泡!
>做兩種的原因是：要區別message是從**自己**發的還是**別人**發的，才會像聊天的樣子~
>
如下圖：
p.s. 當然經過無限個小時的**調整**才弄得稍微可看...只怪我太笨了。
## 3.經營自己的User Homepage
見 **Other functions - 1.**
# 作品網址：https://midterm-project-chatroom-b23ab.web.app/
# Components Description :
## Basic Components
### 1. Membership Mechanism
使用信箱登入，若初次使用，需要先創建一個帳號，按下左下按鈕`
New Account`，會進入讓你Register的頁面，創建好帳號後，會**自動登入**，進入User page。
### 2. Firebase Page
本專案由Firebase下建立的專案為底，最後用`firebase deploy`取得public的**hosting URL**
### 3. Database
Database 結構如下：
root 下面有 :
`ChatRooms` `Public_list` `Rooms` `Users` `User_advice`
因為有些是測試用的，以下用功能來分別說明：
1. `Rooms`是掌管整個聊天室的核心，每個聊天室會有自己的roomName，並被push到room下方。 接著每個room內的messages會用.push()的方式存入。
存入資料有：`message` `name` `photoURL` `time` <br>
2. `Users`也是本聊天室的核心，因為它記錄每個使用者擁有的**roomName**和他的個人資料：像是Background的Photo URL, quote, aboutme, Notifications...等功能。<br>
範例為以 `Alexandra's rooms` 為例的資料庫<br>
3. `Public_lists`是紀錄在 **Public room**的messages，做法跟lab06-Simple Forum一樣，相對簡單。
4. `User_advice` 是紀錄使用者寄給客服中心的建議，做法跟`Public_list`一樣，更簡單的不用去讀它XD我們假定是developer在後端自己看意見改進這樣。
5. 最後是`ChatRooms`，這個是當初，我本來想用這個做為區分使用者對聊天室的權限，後來發現從Users那邊就做完了，所以者個Path目前算是Redundant。任何一個使用者建立一個room的時候，這個room就會被push到 `/ChatRooms/`下。
### 4. RWD
在Responsive Webpage Design的部分，由於大部分的css我使用**bootstrap**，所以它已經內部寫好RWD了，但還是有些東西試另外手刻的，主要用兩種方式：
1. 最簡單的就是 `style = "margin-left: 50%"`
這種方式在不同網頁比例的時候，因為它是用**百分比**而非絕對pixel數，在畫面上就會有所調整了。
2. 用 `@media only screen and (max-width: 300px){.text {font-size: 11px}}` 針對不同螢幕大小，調整某些element或tag的size。
### 5. Topic Key Function
主要聊天室的作法在web details描述許多，這邊主要針對**程式碼**多做說明：
1. 從**建立房間**到**進入房間聊天**，我分成兩組`.html`跟`.js`檔寫：`PrivateRoomSettin.html` `PrivateChatroom.html` (.js以此類推)
2. 為了做到把 roomName傳入 `PrivateChatroom.js`，我將每個room的button的id設為該room的名稱，且觸發
``` javascript=
id = '"+ (childData.room)+"'
```
``` javascript=
onclick = ChangeRoomName(this.id)
```
在點擊這個房間時，
:::info
/Users/{user.displayName}'s/cur_room
:::
會被設成**this.id**，因此到`PrivateChatroom.js`後，就可以從`firebase.database().ref('PATH')`去取得cur_room的roomName囉!
3.接著要進行**傳訊息**，每個private room都會有
``` html=
<div class= "msg-container">
<!-- List of messages -->
<div id="msg_list">
</div>
</div>
```
當有user傳訊息，會push資料到path:
:::info
/Rooms/{roomName}
:::
接著在js讓ref(PATH).on()去讀取每次新增的**訊息**資料，就可以把訊息一個一個畫出囉!
## Advanced Components
### 1. Third-Party Sign In
第三方登入方式採取**Google**登入，先從專案的
:::info
**Authentication** >> **Sign-in Method**
:::
把 Google 打開，然後再在`signin.js`的程式碼加入：
``` javascript=
btnGoogle.addEventListener('click', function() {
/// TODO 3: Add google login button event
/// 1. Use popup function to login google
/// 2. Back to index.html when login success
/// 3. Show error message by "create_alert()"
var provider = new firebase.auth.GoogleAuthProvider();
// provider.addScope('https://www.googleapis.com/auth/contacts.readonly');
firebase.auth().signInWithPopup(provider).then(function(result) {
var token = result.credential.accessToken;
window.location.href = "User.html";
}).catch(function(error) {
create_alert('error', error.message);
});
})
```
就完成第三方登入了!
### 2. Chrome Notification
在使用者**得到來自別人(或自己XD)的私訓時**，會在右下角取得通知，不過記得**要先允許通知**(Firefox的話在最上方網址左邊會有個可以點的(像鎖的icon)
點開來按允許後，應會正常得到通知了。
不知為何chrome瀏覽器不行，只有Firefox可以。
成果類似下方：
實作方法如下：
``` javascript=
////////// Chrome Notification /////////
// At first, let's check if we have permission for notification
// If not, let's ask for it
if (window.Notification && Notification.permission !== "granted") {
Notification.requestPermission(function (status) {
if (Notification.permission !== status) {
Notification.permission = status;
}
});
}
// If the user agreed to get notified
if (window.Notification && Notification.permission === "granted") {
// Using an interval cause some browsers (including Firefox) are blocking notifications if there are too much in a certain time.
// var interval = window.setInterval(function () {
// var user = firebase.auth().currentUser;
console.log("Going to receive.");
firebase.database().ref('Users/'+ user.displayName +' \'s rooms/Notifications').on('child_added', function(snapshot) {
var newNotf = snapshot.val();
/// Widow Notifications!TODO
// Thanks to the tag, we should only see the "Hi! 9" notification
var n = new Notification("You got a new private message from " +newNotf.from,{
body: newNotf.message,
icon: newNotf.photo,
tag: 'newArrival' // 設定標籤
}
);
// "You got a new private message from " + newNotf.from+ ': '+ newNotf.message, {tag: 'soManyNotification'});
// alert('You got a new private message from '+ newNotf.from+ ': '+ newNotf.message);
n.onclick = function(){
window.location.href = "User.html";
n.close();
}
}); // End on.
// }, 200);
// }
}
// If the user hasn't told if he wants to be notified or not
// Note: because of Chrome, we are not sure the permission property is set, therefore it's unsafe to check for the "default" value.
else if (window.Notification && Notification.permission !== "denied") {
Notification.requestPermission(function (status) {
// If the user said okay
if (status === "granted") {
var i = 0;
// Using an interval cause some browsers (including Firefox) are blocking notifications if there are too much in a certain time.
var interval = window.setInterval(function () {
// Thanks to the tag, we should only see the "Hi! 9" notification
var n = new Notification("Hi! " + i, {tag: 'soManyNotification'});
if (i++ == 9) {
window.clearInterval(interval);
}
}, 200);
}
// Otherwise, we can fallback to a regular modal alert
else {
alert("Hi!");
}
});
}
// If the user refuses to get notified
else {
// We can fallback to a regular modal alert
alert("The user refuse to get notified!");
}
```
### 3. Use CSS Animation
我在`index.html`用css和幾張照片做了一個小動畫(參考網路資源)，原本想做更酷炫的，但問題頻頻出現，無法解決，最後終於成功用出一個看起來也不錯的XD而且我本人可以接受的。
基本原理如下：
:::success
1. 將 `index.html` 連結 `animation.css`，取得需要的**import**
2. 在`animation.css`先定義好元素，以中央翻轉的字牌為主，其實它是宣告一個flip然後產生三個div物件
3. 在三個物件內，使用`animation` `@keyframe`等關鍵字去進行動畫：一定秒數內三個物件**往下shift**固定距離
4. 三個物件規定在一個範圍內的顯示，先用`overflow:hidden`來限制要顯示的話**只能留在框框內**，再用`display:inline-block`規定排版
5. 最後用`animation: show 5s linear infinite;` 規定 **5秒為週期**，做**無限次**iterations
6. 最後用keyframes控制顯示位置
``` css =
@keyframes show {
0% {margin-top:-270px;}
5% {margin-top:-180px;}
33% {margin-top:-180px;}
38% {margin-top:-90px;}
66% {margin-top:-90px;}
71% {margin-top:0px;}
99.99% {margin-top:0px;}
100% {margin-top:-270px;}
}
```
:::
### 4. Security Report
見最下方 Security Report
# Other Functions Description :
## 1. 經營自己的User Page
每個User有自己的User Homepage，有點類似FB的主頁，可以換封面照片、自己的頭貼，可以發文(發照片)，也可以看到別人私訓自己的Notifications。
礙於時間有限，沒時間讓使用者 **"參觀別人的userpage"**
(如果多一點時間我有信心做得出來!不過我也已經鞠躬盡瘁ㄌ）
> []
## 2. 在聊天室可以點人的頭貼，私訓給他
在聊天室中，有時候你想對某個人單獨私訓，除了另創一個聊天室，你也可以**直接點他的頭像**，就會跳出一個可以讓你傳message給他的輸入欄位囉!
## 3. 在User page有客服管道可以聯絡
我有設計一個可以讓你**寫建議**給客服(就是我啦)的管道，就是在你的主頁右上角有個 `Contact us` 的Button，按下去就可以寫下你對我想說的話...(其實是我做太爛的地方，我知道瑕疵百出XD 或是對這份網頁有任何的建議)
在database的
:::info
root/Users_advice
:::
就會收到你的 Username 和advice囉!
## Security Report (Optional)
我在聊天室**加入成員**的部分這樣設計：
:::danger
只有**創建聊天室的人**可以進行**加成員**的動作，其他人不行
:::
防止聊天室內部資料亂掉。
## Reference
[1] A. J. Doud, J. P. Lucas, M. T. Pisansky, and B. He, "Continuous
three -dimensional control of a virtual helicopter using a motor imagery based brain -computer interface," PLoS One, vol. 6, no. 10, p. e26322,2011.
[2] A. S. Royer, A. J. Doud, M. L. Rose, and B. He, "EEG control of a virtual helicopter in 3 -dimensional space using intelligent
strategies," IEEE Trans Neural Syst Rehabil Eng, vol. 18, no. 6, pp.
581 -9, Dec 2010.
[3] K. LaFleur, K. Cassady, A. Doud, K. Shades, E. Rogin, and B. He,
"Quadcopter control in three -dimensional space using a noninvasive
motor imagery -based brain -computer interface," J Neural Eng, vol. 10,no. 4, p. 046003, Aug 2013.
[4] T. Shi, H. Wang, and C. Zhang, "Brain Computer Interface system
based on indoor semi-autonomous navigation and motor imagery for
Unmanned Aerial Vehicle control," Expert Systems with Applications,
vol. 42, no. 9, pp. 4196-4206, 2015.
[5] X. Duan, S. Xie, X. Xie, Y. Meng, and Z. Xu, "Quadcopter Flight
Control Using a Non -invasive Multi-Modal Brain Computer Interface," Front Neurorobot, vol. 13, p. 23, 2019.
[6] M. Wang, R. Li, R. Zhang, G. Li, and D. Zhang, "A Wearable
SSVEP-Based BCI System for Quadcopter Control Using Head-Mounted Device," IEEE Access, vol. 6, pp. 26789-26798, 2018
---
###### tags: `SSVEP` `SSVEP-BCI` `EEG` `Quadcopter`
Sign in with Wallet
Connect another wallet