---
# System prepended metadata

title: DCM772
tags: [Comic Analysis]

---

# [Digital Comics Image Indexing Based on Deep Learning](https://pdfs.semanticscholar.org/9a77/84eea6bfa62bf2834ee0b87a3cdda46006f2.pdf)
Introduce a new dataset with a global indexing system
Comparison of deep learning models for comic book analysis tasks




### Proposed Method : 
![](https://hackmd.io/_uploads/rkGmH2Qqh.png)



- The model is separated into offline and online tasks and the offline tasks are formulated to capture the comic book elements
- For panel,character and face detection YOLOv2 is used with anchor boxes from Faster R-CNN.
- The anchor bounding boxes are found using Kmeans on the ground tructh

**Speech Balloon Segmentation**
- The traditional methods produce several false positives and the deep learning models can not identify the boundaries as well as the traditional methods.
- The proposed method combines deep learning and traditional method 
- The deep learning method used is DeepLabv2 and the detected balloons if they have an IOU above a threshold with detected balloons of traditional method are added to the output set.

**Text Recognition**

![](https://hackmd.io/_uploads/r1dkw3X5h.png)
- Authors propose a method to utilise unlabed data for text recigntion of certain style.
- First a pretrained OCR is used to detect text which is then evaluated with a lexicality metrics and then used to train a second OCR model with the output of the first model as pseudo ground truth.
- The pretrained OCR used is FineReader.
- The lexicality metric is L = (1 −mean Levenshtein distance per character)

**Loss:**
- The losses assesed in the paper are perceptual loss and pixel difference.
- Perceptual loss is used as pixel difference makes it difficult to reconstruct intra-subject variations of a template.

### Experiments : 
- The datasets used are eBDtheque, Fahad18 and the new DCM772 dataset.
- For character and face detection mAP is taken as the metric, for panel detection the precision and recall is considered using the IOU between ground truth and prediction.
- For ballon segmentation segmentation accuracy is used and for text recognition, character and word error rates are used.
- For character and face detection deep learning models perform well but fall behind in panel detection.
- In balloon segmentation boundaries problem persists for ML methods and the combined method performs a little better.
- The text recognition OCR gives high error rated which may be attributed to the text extraction algorithm that cuts lines.