## Project title: *Manga Translator*
Students: Tu Chau and Quang Nguyen
Instructors: Huy Le, Ha V. Nguyen, Tung H. Cao, Hung Ngo
Course: Deep Learning for AI, 2022
LeHongPhong Highschool for the Gifted, HCMC Vietnam
---
## Project Description:
- Vietnamese people are among the biggest fans of manga, especially Vietnamese teenagers.
- However, most of Vietnamese readers can only access to a few famous (translated) manga due to language barrier and lack of manga translators.
- Having acknowledge the situation, our team are encouraged to build an app which can translate manga into Vietnamese.
---
## Task description $T$:
High-level, briefly with some keywords & illustrations:
- *++What our system do:++* Translate manga in English into Vietnamese.
- *++What does it take as input:++* Untranslated manga
- *++What does it give as output:++* Translated manga
---
## App pipeline:
- ***Image Segmentaton***: Extract text and words from the manga pages.
- ***Translation***: Translate text written in English into Vietnamese.
- ***Completing Output***: Replace the text in English on the manga pages.
---
## Image Segmentation:
- Using pretrained paddleOCR to recognize lines of text (bounding boxes).
- Backbone architecture: MobileNetV3
- Concatenate bounding boxes (using the distance between those bounding boxes).
- Since the size of text varies $\rightarrow$ the distance between bounding boxes varies.
----
### Image Segmentation:
----
### Image Segmentation:
---
## Translation:
- Translate the text with googletrans
- Sometimes paddleOCR ommit the spaces between words $\rightarrow$ using module wordninja to prevent the problems.
---
## Text Rendering:
- Inpainting objects (text) from the manga page with [LaMa](https://github.com/saic-mdal/lama) $\rightarrow$ replace them with new text (translated sentences).
----
### Inpainting:
----
### Text Rendering:
---
### Compare with other apps:
- Inpainting with LaMa instead of inserting white box to cover the old text.
- Auto-detection rather than cropping out.
---
### What needs to be improved:
- Real-time execution.
- The app only support 1 font $\rightarrow$ insert synthetic text.
- Sometimes PaddleOCR cannot detect text accurately $\rightarrow$ need to be retrained.
----
---
## Demo:
- Demo Video: https://youtu.be/usfyNl7KHbQ
- GitHub repo: https://github.com/tuchau1404/manga_translate
{"metaMigratedAt":"2023-06-17T02:58:04.734Z","metaMigratedFrom":"YAML","breaks":true,"slideOptions":"{\"transition\":\"slide\",\"width\":900,\"height\":900,\"margin\":0.1,\"minScale\":0.2,\"maxScale\":1.5}","contributors":"[{\"id\":\"70a8c36d-ad4f-4f48-a325-03376546e656\",\"add\":3604,\"del\":825}]","description":"Students: Tu Chau and Quang Nguyen","title":"Project Capstone"}