---
# System prepended metadata

title: Object Removal Document

---

# Object Removal Document

## Setup

- Required Modules

```
# Required modules
sudo apt-get install git zip virtualenv xvfb python3
```

- Install Project

```
git clone https://github.com/showhue/object-removal.git
virtualenv venv --python=/usr/bin/python3
cd objectRemoval
pip install -r requirements.txt
```

- Color Thief Code Change
Add color pixels to color thief module
https://github.com/fengsp/color-thief-py/issues/1#issuecomment-225681712
```
def palette(self):
    return self.vboxes.map(lambda x: x["color"] + tuple([x["vbox"].count]))
```

- Lama setup
  Lama source: https://github.com/saic-mdal/lama

```
git clone https://github.com/saic-mdal/lama.git
virtualenv venv --python=/usr/bin/python3
source venv/bin/activate
pip install torch==1.8.0 torchvision==0.9.0

cd lama
export TORCH_HOME=$(pwd) && export PYTHONPATH=.
pip3 install wldhx.yadisk-direct
curl -L $(yadisk-direct https://disk.yandex.ru/d/ouP6l8VJ0HpMZg) -o big-lama.zip
unzip big-lama.zip
```

## Object Removal Flow

- Image Collector
  - Using Selenium to crawl images from unsplash.com
- Object Property Filter
  - Using Vision API to get image objects and labels
  - Filter eliminate images not suitable
  - Create image mask that is slice bigger that origin main object area
- Lama Object Removal
  - Remove object with mask by repo Lama
- Object Vague Filter
  - Filter images too vague by clustering image color

## File Structure

    - objectRemoval
        - imageJson (json files of images' properties)
        - outputJson (json files of targeted output images)
        - unplash (images crawlled from unsplash)
        - outputs (images after objectRemoval)

### Switch to virtual environment

```
source lama/venv/bin/activate
```

### Crawl Images with keywords

```
python imageCollector.py --tag <KEYWORD> --output unsplash
```

### Object Property Filter

- Target Folder

```
pyhton objectRemovalFilter.py -k <KEYWORD> -o <OUTPUT_DIRECTORY>
```


```
- <OUTPUT_DIRECTORY>/
    <image1_name>.png
    <image1_name>_mask.png
    <image2_name>.png
    <image2_name>_mask.png
    <image3_name>.png
    <image3_name>_mask.png
    .
    .
    .
```

- Json Reproduce
  If you want to adjust filter's but you don't want to run Vision API,
  use`objectRemovalFilterJson.py` instead.

```
python objectRemovalFilterJson.py -i <TARGET_JSON> -o <OUTPUT_DIRECTORY>
```

### Lama Object Removal

```
python3 bin/predict.py model.path=$(pwd)/big-lama indir=$(pwd)/<INPUT_DIRECTORY> outdir=$(pwd)/<OUTPUTDIRECTORY>
```

```
- <OUTPUT_DIRECTORY>/
    <image1_name>_mask.png
    <image2_name>_mask.png
    <image3_name>_mask.png
    .
    .
    .
```


### Object Vague Filter
- Parameters and Variables
$$
\bf{RGB} = Vector(r,g,b)\\
\bf{RGB}_i = Cluster\;Colors\;of\;Image\;(decendent\;of\;pixels)\\
\bf{RGB}_{max} = Max\;Pixels\;of\;Color\;in\;\bf{RGB}_i\\
n\in Color\;Kinds\;of\;photo\\
*\in Inner\;Area
$$

- Main Color Difference Between Inner and Outer Frame
$$
D_{main} = |\bf{RGB}_{max}^*-\bf{RGB}_{max}|
$$
- Average Color Difference Between Inner and Outer Frame
$$
D_{avg} = \frac{\sum_{i=0}^{min(n,n^*)}|\bf{RGB}_{i}^*-\bf{RGB}_{i}|}{min(n,n^*)}
$$
- Filter
    - $D_{main}<20$
    - $D_{max}<5$

![](https://i.imgur.com/E8PfZVe.png)
```
python objectVagueFilter.py -i <INPUT_DIRECTORY> -o <OUTPUT_DIRECTORY> -f <IMAGE_JSON_FILE>
```

```
- <OUTPUT_DIRECTORY>/
    <image2_name>_mask.png
    <image6_name>_mask.png
    <image11_name>_mask.png
    .
    .
    .
```

## Automation Removal

```
chmod u+x objectRemoval.sh // (Only run in the first time)
source lama/venv/bin/activate
./objectRemoval.sh -w <KEYWORD>
```

To see more please go to `objectRemoval.sh`
## Upload to Database
Go to https://github.com/showhue/uboard-backend/blob/Thomson861106/ObjectRemovalUploader/src/tools/objectRemoval.py
```
python objectRemoval.py -i <output.json> //files in outputJson
```
