---
# System prepended metadata

title: To Do & Progress

---

# To Do & Progress 
### Main [link](https://hackmd.io/@EvheMary/H1F_-2Oco)
### Changelog & Plans
- Update on Openvibe pipeline
    - OpenVibe Real Time Pipeline done :heavy_check_mark: 
- Move to using DL for epoch / window classification (Rest / L / R)
    - Plan to train one using BCIC DS
    - use (2 channels)
    - **First** try EEG net for realtime simulation, check performance
        1. EEGNet, 2 CH, 2 sec time window, no overlap
        2. Same as above, extra channel using feature enhancement
        3. **Use other dataset (?)**
    - Second, will try other network (?)
- **Making the Unity interface to accomodate fine-tune and testing**
### Progress:
* [Details Spreadsheet - Parameter Tuning](https://docs.google.com/spreadsheets/d/14hq6dyMisByPU6dU6DjjVxq0ev8EEa7vObcRZi0QenI/edit?usp=sharing)
    * Finished Parameter Tuning for EEGNet, SCCNet, Shallow Conv Net ()
* [Performance Record & Tests](https://docs.google.com/spreadsheets/d/1y6C1sUUvXZ7Y3zKBpr-_qj7axA2z4Z7pa5uC7ELwaNU/edit?usp=sharing) 
    

* Summary:
    * Parameter tuning is done
    * 1DCNN result is bad (considering not using it)
    * making the Unity interface 
        * Calibration 
        * Test
    * LSL Communication can now be done both ways
    * Working on the scenario for recording and testing --> [Link](#Virtual-Task)
        * Designing calibration track and scenario
        * Designing test objectives and scenario

* 2 Class
    * Within Subject
        * [x] Training Parameter Tuning
        * [x] Model Parameter Tuning
    * Cross Subject
        * [x] Training Parameter Tuning
        * [x] Model Parameter Tuning
    * Include fine-tune 
* 3 Class
    * Within Subject
        * [x] Training Parameter Tuning
        * [x] Model Parameter Tuning
    * Cross Subject
        * [x] Training Parameter Tuning
        * [x] Model Parameter Tuning
    * Include fine-tune

### Try
* Timeline
* **Finish virtual task interfaces**
* Fine-tune virtual task interface

### To update:
* Unity
    * v-sync? fps limited
    * pause unity --> pause openvibe
    * send end stim when quit to stop openvibe
* OpenVibe
    * Epoch crops
---
### Versions
1. Version 1 [Link](https://hackmd.io/@EvheMary/HJbD2_JQn)
2. Version 2 [Link](https://hackmd.io/@EvheMary/r1O0Y6-Fh)

### Update
#### [09/26] - [10/02]
 - [X] OpenVibe Dataset ✅ 2 Class Test Results ✅ Feature Analysis
 - [X] BCIC 2b Dataset (2 Classes) ✅ 2 Class Test Results ⬜️Feature Analysis
 - [ ] BCIC 2a Dataset (4 Classes) ⬜️ Test Dataset ⬜️Feature
#### [10/03] - [10/9]
 - [X] BCIC 2b Dataset (2 Classes) ✅ 2 Class Test Results ⬜️Feature Analysis 
 - [ ] 2 Classes + Rest threshold ✅ OpenVibe XML ⬜️ Python Code
 - [ ] Check other paper for subject performances
 - [ ] BCIC 2a Dataset (4 Classes) ⬜️ 2 Class Test Results ⬜️Feature Analysis
##### Planned:
 - [ ] OpenVibe Metabox for processes (If necessary)
#### [10/10] - [10/16]
 - [X] BCIC 2b Dataset (2 Classes) ✅ 2 Class Test Results ✅Feature Analysis
 - [X] Move process from OpenVibe to Python
 - [X] Check other paper for subject performances ([-->](#Comments))
 - [ ] 2 Classes + Rest threshold ✅ OpenVibe XML ⬜️ Python Code (:heavy_check_mark:Read, :heavy_check_mark:Rest read, :white_check_mark:Classify )
 - [ ] Thesis presentation [Slides](https://docs.google.com/presentation/d/17FzFNwZU7sTNwVhuC6m9W4zWXlc7pLLK/edit?usp=sharing&ouid=110788102584718099191&rtpof=true&sd=true)
 - [ ] Pardigm for the task (The metrics and the task)
#### [10/17] - [10/23]
 - [X] Move process from OpenVibe to Python
 - 
 - [ ] 2 Classes + Rest threshold ✅ OpenVibe XML ⬜️ Python Code (:heavy_check_mark:Read, :heavy_check_mark:Rest read, :white_check_mark:Classify ) --> Need to change so can easily accomodate more classes
 - [ ] Thesis presentation [Slides](https://docs.google.com/presentation/d/17FzFNwZU7sTNwVhuC6m9W4zWXlc7pLLK/edit?usp=sharing&ouid=110788102584718099191&rtpof=true&sd=true)
 - [ ] Pardigm for the task (The metrics and the task)
#### [10/24] - [10/30]
- [ ] Stopped testing, code optimization & feature optimization
- [ ] Checking feature that will be used (Feature analysis + graphs)
- [ ] Test again, difference between using and not using rest (Also check how openvibe calculate power spectrum)
- [ ] How to callibrate rest for real time? Callibration session?
#### [10/31] - [11/13]
- Sick
- [ ] Code Optimization
- [ ] Feature Analysis
#### [10/31] - [11/13]
- [ ] Code Optimization
- [ ] Feature Analysis (Why the same value for both var and abs? --> Code)
- [ ] Flow for control
#### [11/14] - [11/20]
- [ ] Feature Analysis
- [ ] Demo 
#### [11/21] - []
- [ ] Baseline correction on all (?baseline on real time?)
- [ ] Relative power
- [ ] Demo 
#### [] - [01/09]
- [ ] Demo
    - [ ] Signal Extraction (Tested, but unsuccesful control because of the feature)
    - [X] Preprocess (basic) (Channel select, bandpass)
    - [X] Feature Extraction (basic) (Spatial feature + Spectral Analysis)
    - [X] Classification (basic - binary) (LR) --> 3 class (LRN)
    - [X] Inference (LSL without car & Unity car)
#### [01/10] - [01/15]
- [X] Demo Unity (Full pipeline, fail to control)
- [ ] Demo Bluetooth Car (Car BT can't connect, LSL work though)
- [ ] Feature adaptation (test using prerecorded, need to change feature extraction for real time 2 channel cap)
- [ ] Paper review of other application
- [ ] Maybe need to record signal
#### [01/16] - [01/30]
- [ ] Feature check code [link](https://colab.research.google.com/drive/1NxBo-1_r3AHz0tvUmLfRxTiXLeGWYzXq?usp=sharing)

#### [01/31] - [02/14]
- [X] Feature check code [link](https://colab.research.google.com/drive/1NxBo-1_r3AHz0tvUmLfRxTiXLeGWYzXq?usp=sharing)
- [X] Numerical results [link](https://docs.google.com/spreadsheets/d/1lY5rhM4OM3s-INEW-GC085Mfcd4Em00lrKwA8Gyk_gI/edit?usp=sharing)
    - After preproc, has better and balanced overall accuracy
    - Use mean, variance, PSD (alpha, beta), EDR (alpha, beta), looking for more usable feature
    - Unable to check feature on short window signal, so going to assume based on time-frequency log power analysis
- [ ] To do: - Apply new feature qxtraction to OpenVibe 

#### [02/15] - [02/22]
- [X] OpenVibe pipeline update (Update feature extraction) [check in main link](https://hackmd.io/@EvheMary/H1F_-2Oco)
- [ ] Classifier for features are still unclear
- To do: 
    - Record result & Find method to get feature weight (find which feature has higher impact on classification)

#### [02/23] - [02/28]
- [X] Baseline / Rest threshold (not automated, need to be manually inputted to control script)
- [X] Record :white_check_mark: Not yet Analysis [Link](https://docs.google.com/spreadsheets/d/1gWfB-3_5XQrsf_DvHcPSbCK_3ADMX-FcWcQ5UsxwshI/edit?usp=sharing)
- [ ] Classifier
    - [ ] Combine Feature --> Classify (Need weight for each feature?)
    - [X] Classify for each feature --> (has biggest distance with Baseline)
- [X] Control

#### [02/28] - [03/07]
- [ ] Update on Openvibe pipeline (there are duplicates)

#### [03/08] - [04/10]
- [ ] Create dataset for training model
- [ ] Training models (EEGNet) --> Planning to use other models
- [X] DL pipeline for OpenVibe

#### [04/10] - [06/28]
- [X] Fine tune model
- [ ] Virtual task development

#### [06/28] - [07/04]
- [ ] Virtual task development


## MI - BCI

## Version 1

### Pipeline
![](https://i.imgur.com/O3zRNhl.png)

<span style="color:red">**Notice**</span> : Records moved to separate notes [Link](https://hackmd.io/@EvheMary/HJbD2_JQn)

## Version 2
![](https://i.imgur.com/gD3iYKq.png)

## Virtual Task
### Unity Game
- Menu
    - Test
        - Test
            Objective : Test the classifier in a realtime car control scenario. Will record the classification result and the succesful objective count. 

            Testing scenario options:
                - Fixed time, count objective hit
                - Fixed objective count, record time
 
        - Calibration
            Objective : Give stimulation to the recorded EEG signal. Stimulation is set depending on the situation presented in the game.
    - Setting
        - Setting of the game
        - Setting of the LSL

### Flowchart
![](https://hackmd.io/_uploads/HJMsiPfKn.png)
- (1) Send type of class is being done by the unity. For each class, will send 'label stimulation' at the start, then send 'end stimulation' at the end. Repeat multiple time.
- (2) Send classification result to be used to control the car
- (3) Send stimulation when objective is hit

### Calibration Track
![](https://hackmd.io/_uploads/By_Om_MFn.png)
- 1 Left
- 1 Right
- 2 Straight

Comment: Length of the arc and the straight is designed to be same so that the duration is supposedly also the same.


## Resources
https://ieeexplore.ieee.org/abstract/document/4408441?casa_token=UZ2aOtqwN-8AAAAA:t8owzfwPcN-D0D0c3tgy6xuh57I9keH20NJxdjSNUIqymEGMxNmo24qDof9nHTKzGVx9-xIT