# To Do & Progress
### Main [link](https://hackmd.io/@EvheMary/H1F_-2Oco)
### Changelog & Plans
- Update on Openvibe pipeline
- OpenVibe Real Time Pipeline done :heavy_check_mark:
- Move to using DL for epoch / window classification (Rest / L / R)
- Plan to train one using BCIC DS
- use (2 channels)
- **First** try EEG net for realtime simulation, check performance
1. EEGNet, 2 CH, 2 sec time window, no overlap
2. Same as above, extra channel using feature enhancement
3. **Use other dataset (?)**
- Second, will try other network (?)
- **Making the Unity interface to accomodate fine-tune and testing**
### Progress:
* [Details Spreadsheet - Parameter Tuning](https://docs.google.com/spreadsheets/d/14hq6dyMisByPU6dU6DjjVxq0ev8EEa7vObcRZi0QenI/edit?usp=sharing)
* Finished Parameter Tuning for EEGNet, SCCNet, Shallow Conv Net ()
* [Performance Record & Tests](https://docs.google.com/spreadsheets/d/1y6C1sUUvXZ7Y3zKBpr-_qj7axA2z4Z7pa5uC7ELwaNU/edit?usp=sharing)

* Summary:
* Parameter tuning is done
* 1DCNN result is bad (considering not using it)
* making the Unity interface
* Calibration
* Test
* LSL Communication can now be done both ways
* Working on the scenario for recording and testing --> [Link](#Virtual-Task)
* Designing calibration track and scenario
* Designing test objectives and scenario

* 2 Class
* Within Subject
* [x] Training Parameter Tuning
* [x] Model Parameter Tuning
* Cross Subject
* [x] Training Parameter Tuning
* [x] Model Parameter Tuning
* Include fine-tune
* 3 Class
* Within Subject
* [x] Training Parameter Tuning
* [x] Model Parameter Tuning
* Cross Subject
* [x] Training Parameter Tuning
* [x] Model Parameter Tuning
* Include fine-tune

### Try
* Timeline
* **Finish virtual task interfaces**
* Fine-tune virtual task interface

### To update:
* Unity
* v-sync? fps limited
* pause unity --> pause openvibe
* send end stim when quit to stop openvibe
* OpenVibe
* Epoch crops
---
### Versions
1. Version 1 [Link](https://hackmd.io/@EvheMary/HJbD2_JQn)
2. Version 2 [Link](https://hackmd.io/@EvheMary/r1O0Y6-Fh)

### Update
#### [09/26] - [10/02]
- [X] OpenVibe Dataset ✅ 2 Class Test Results ✅ Feature Analysis
- [X] BCIC 2b Dataset (2 Classes) ✅ 2 Class Test Results ⬜️Feature Analysis
- [ ] BCIC 2a Dataset (4 Classes) ⬜️ Test Dataset ⬜️Feature
#### [10/03] - [10/9]
- [X] BCIC 2b Dataset (2 Classes) ✅ 2 Class Test Results ⬜️Feature Analysis
- [ ] 2 Classes + Rest threshold ✅ OpenVibe XML ⬜️ Python Code
- [ ] Check other paper for subject performances
- [ ] BCIC 2a Dataset (4 Classes) ⬜️ 2 Class Test Results ⬜️Feature Analysis
##### Planned:
- [ ] OpenVibe Metabox for processes (If necessary)
#### [10/10] - [10/16]
- [X] BCIC 2b Dataset (2 Classes) ✅ 2 Class Test Results ✅Feature Analysis
- [X] Move process from OpenVibe to Python
- [X] Check other paper for subject performances ([-->](#Comments))
- [ ] 2 Classes + Rest threshold ✅ OpenVibe XML ⬜️ Python Code (:heavy_check_mark:Read, :heavy_check_mark:Rest read, :white_check_mark:Classify )
- [ ] Thesis presentation [Slides](https://docs.google.com/presentation/d/17FzFNwZU7sTNwVhuC6m9W4zWXlc7pLLK/edit?usp=sharing&ouid=110788102584718099191&rtpof=true&sd=true)
- [ ] Pardigm for the task (The metrics and the task)
#### [10/17] - [10/23]
- [X] Move process from OpenVibe to Python
-
- [ ] 2 Classes + Rest threshold ✅ OpenVibe XML ⬜️ Python Code (:heavy_check_mark:Read, :heavy_check_mark:Rest read, :white_check_mark:Classify ) --> Need to change so can easily accomodate more classes
- [ ] Thesis presentation [Slides](https://docs.google.com/presentation/d/17FzFNwZU7sTNwVhuC6m9W4zWXlc7pLLK/edit?usp=sharing&ouid=110788102584718099191&rtpof=true&sd=true)
- [ ] Pardigm for the task (The metrics and the task)
#### [10/24] - [10/30]
- [ ] Stopped testing, code optimization & feature optimization
- [ ] Checking feature that will be used (Feature analysis + graphs)
- [ ] Test again, difference between using and not using rest (Also check how openvibe calculate power spectrum)
- [ ] How to callibrate rest for real time? Callibration session?
#### [10/31] - [11/13]
- Sick
- [ ] Code Optimization
- [ ] Feature Analysis
#### [10/31] - [11/13]
- [ ] Code Optimization
- [ ] Feature Analysis (Why the same value for both var and abs? --> Code)
- [ ] Flow for control
#### [11/14] - [11/20]
- [ ] Feature Analysis
- [ ] Demo
#### [11/21] - []
- [ ] Baseline correction on all (?baseline on real time?)
- [ ] Relative power
- [ ] Demo
#### [] - [01/09]
- [ ] Demo
- [ ] Signal Extraction (Tested, but unsuccesful control because of the feature)
- [X] Preprocess (basic) (Channel select, bandpass)
- [X] Feature Extraction (basic) (Spatial feature + Spectral Analysis)
- [X] Classification (basic - binary) (LR) --> 3 class (LRN)
- [X] Inference (LSL without car & Unity car)
#### [01/10] - [01/15]
- [X] Demo Unity (Full pipeline, fail to control)
- [ ] Demo Bluetooth Car (Car BT can't connect, LSL work though)
- [ ] Feature adaptation (test using prerecorded, need to change feature extraction for real time 2 channel cap)
- [ ] Paper review of other application
- [ ] Maybe need to record signal
#### [01/16] - [01/30]
- [ ] Feature check code [link](https://colab.research.google.com/drive/1NxBo-1_r3AHz0tvUmLfRxTiXLeGWYzXq?usp=sharing)

#### [01/31] - [02/14]
- [X] Feature check code [link](https://colab.research.google.com/drive/1NxBo-1_r3AHz0tvUmLfRxTiXLeGWYzXq?usp=sharing)
- [X] Numerical results [link](https://docs.google.com/spreadsheets/d/1lY5rhM4OM3s-INEW-GC085Mfcd4Em00lrKwA8Gyk_gI/edit?usp=sharing)
- After preproc, has better and balanced overall accuracy
- Use mean, variance, PSD (alpha, beta), EDR (alpha, beta), looking for more usable feature
- Unable to check feature on short window signal, so going to assume based on time-frequency log power analysis
- [ ] To do: - Apply new feature qxtraction to OpenVibe

#### [02/15] - [02/22]
- [X] OpenVibe pipeline update (Update feature extraction) [check in main link](https://hackmd.io/@EvheMary/H1F_-2Oco)
- [ ] Classifier for features are still unclear
- To do:
- Record result & Find method to get feature weight (find which feature has higher impact on classification)

#### [02/23] - [02/28]
- [X] Baseline / Rest threshold (not automated, need to be manually inputted to control script)
- [X] Record :white_check_mark: Not yet Analysis [Link](https://docs.google.com/spreadsheets/d/1gWfB-3_5XQrsf_DvHcPSbCK_3ADMX-FcWcQ5UsxwshI/edit?usp=sharing)
- [ ] Classifier
- [ ] Combine Feature --> Classify (Need weight for each feature?)
- [X] Classify for each feature --> (has biggest distance with Baseline)
- [X] Control

#### [02/28] - [03/07]
- [ ] Update on Openvibe pipeline (there are duplicates)

#### [03/08] - [04/10]
- [ ] Create dataset for training model
- [ ] Training models (EEGNet) --> Planning to use other models
- [X] DL pipeline for OpenVibe

#### [04/10] - [06/28]
- [X] Fine tune model
- [ ] Virtual task development

#### [06/28] - [07/04]
- [ ] Virtual task development

## MI - BCI

## Version 1

### Pipeline

<span style="color:red">**Notice**</span> : Records moved to separate notes [Link](https://hackmd.io/@EvheMary/HJbD2_JQn)

## Version 2

## Virtual Task
### Unity Game
- Menu
- Test
- Test
Objective : Test the classifier in a realtime car control scenario. Will record the classification result and the succesful objective count.

Testing scenario options:
- Fixed time, count objective hit
- Fixed objective count, record time

- Calibration
Objective : Give stimulation to the recorded EEG signal. Stimulation is set depending on the situation presented in the game.
- Setting
- Setting of the game
- Setting of the LSL

### Flowchart
- (1) Send type of class is being done by the unity. For each class, will send 'label stimulation' at the start, then send 'end stimulation' at the end. Repeat multiple time.
- (2) Send classification result to be used to control the car
- (3) Send stimulation when objective is hit

### Calibration Track
- 1 Left
- 1 Right
- 2 Straight

Comment: Length of the arc and the straight is designed to be same so that the duration is supposedly also the same.

## Resources
https://ieeexplore.ieee.org/abstract/document/4408441?casa_token=UZ2aOtqwN-8AAAAA:t8owzfwPcN-D0D0c3tgy6xuh57I9keH20NJxdjSNUIqymEGMxNmo24qDof9nHTKzGVx9-xIT