---
# System prepended metadata

title: Data Processing

---

# Data Processing

## Data Processor
To process the data, I use the python module [python-chess](https://python-chess.readthedocs.io/en/latest/) to simulate the full game. I record the board configue in the target player's turn, and store to the file in the foramt introduced in last chapter.

Before the process starts ,user can choose to fetch out the game that players plays white or black side. After the process, the program will generate a folder consists of `.txt` files named by `game id`+ `_` +`turn`.

### Work Flow
```graphviz
Digraph G{
    "Fetch Game From pgn"->"Move"->"Check turn"
    "Check turn"->"Record and store"[label="Players"]
    "Check turn":w->"Move":w[label="Not"]
    "Record and store"->"Move"
}
```

### Usage
`python Dataloader pgn_filename store_foldername side`

## Data Analyzer
To check if there are mutiple choices among a single board configue, I develop this tool to check out. The process is quite simple, the program read out the data from the target folder first. Then record the board configue into a global list and record the count when collision happened.

### Work Flow
```graphviz
Digraph G{
    "Read File"->"Add to List"->"Collison ?"
    "Collison ?"->"Record"[label="Yes"]
    "Collison ?"->"Read File"[label="No"]
    "Record"->"Read File"
}
```
### Usage

`python Dataanalyzer target_foldername`

## Data Visualizer
A visaulizer is necessary for checking whether the result is expected. To implement the visualizer for our model, I use  [opencv](https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html) to draw the result on the board.

The process here can be split to 3 sub-process for each block 
1 . Draw the piece on the block
2 . Check if it needs an filter to be shown in the block 
3 . Draw the result on the block

After processing each block as above, the model result can be perfectly visualized.

**This program has been removed , there are functions that provide this service in** [visualizer]()

### Work Flow
For Each Block :
```graphviz
Digraph G{
    "Draw the piece"->"Add Filter"->"Draw Result"
    "Model Result"->"Draw Result"
}
```

### Usage
`python Visualizer.py target_file`