Open OCR-D-TechCall

General information

The open OCR-D-TechCall takes place every second Wednesday, 2 to 3 p.m. CET. Feel free to suggest important issues for discussion and vote for the issues which should be dealt with in the next call. Furthermore, each week one participant can talk about his (current) experiences and challenges with OCR-D. If you want to share your experiences in one of the next calls, just put your name on the list. On the Monday before a call, we will put the agenda together and send it to you.

Agenda for the next call

(numerical prefix represents subjective urgency. 1 = low, 2 = medium, 3 = high)

• discussion: tesstrain
  • https://github.com/tesseract-ocr/tesstrain/issues/261 (true CER)
  • https://github.com/tesseract-ocr/tesstrain/issues/254 (norm_mode)
• Common metadata for OCR models
  • Which version of which engine
  • datasets
  • training (hyper)parameters (topology, epochs / early stopping)
  • normalization/regularizations
  • codec, (histogram?)
  • binarized, grayscale, RGB?
• Setting up our own Docker container infrastructure - worth it?

Agenda of the last call

• OCR-D/core https://github.com/OCR-D/core/releases/tag/v2.25.0
  • OcrdPage: etree XML bindings
  • OcrdPage: common .id attribute
  • recrop if deskewed afer cropping (https://github.com/OCR-D/core/pull/688)
• Stefan Weil/Jan Kamlah: https://github.com/UB-Mannheim/GTCheck / https://github.com/UB-Mannheim/GTMake / https://github.com/UB-Mannheim/tesseractXplore

Postponed

• Emulating Aletheia's "Shrink Polygons" method in an OCR-D processor?

Archive

• Robert Sachunsky: Distributed Ground Truth transcription with Excel

  • https://dbt2021.abstractserver.com/program/#/details/presentations/184
  • https://wrznr.github.io/bibliothekartag-2021/

• Deprecating Python 3.6

  • https://github.com/OCR-D/ocrd_calamari/issues/61#issuecomment-864462986

• Evaluation of cor-asv-ann postcorrection based on various OCR models and in-/out-domain data https://github.com/ASVLeipzig/cor-asv-ann

• New ocrd_all release

  • no more ocrd-anybaseocr-block-segmentation
  • fixes for eynollah, cor-asv-ann, dinglehopper, ocrd_anybaseocr

• Voting on the defintion of CER :-) https://github.com/impactcentre/ocrevalUAtion/issues/21 https://github.com/roy-ht/editdistance/issues/28 https://github.com/roy-ht/editdistance/issues/38

  • CER('AB', 'ABBBB') == 3 /
    • 2 (len GT)
    • 5 (len OCR also len edit operatins including copy)

• https://github.com/kba/vdhd-2021-05-12 (btw also https://github.com/kba/vdhd-2021-05-05)

• eynollah with OCR-D bindings released

  • https://github.com/OCR-D/ocrd_all/releases/tag/v2021-04-25

• Rewritten ocrd-anybaseocr-crop

  • https://github.com/OCR-D/ocrd_anybaseocr/releases/tag/v1.4.0
  • https://github.com/OCR-D/ocrd_anybaseocr/pull/84
  • https://bertsky.github.io/ocrd_anybaseocr

• https://github.com/kba/page-to-alto

  • Outputs <SP/> and <HYP/> https://github.com/kba/page-to-alto/issues/6

• https://github.com/kba/page-to-alto

  • Python implementation of PAGE-to-ALTO conversion

• eynollah with OCR-D bindings ready to test

  • https://github.com/OCR-D/ocrd_all/pull/247
  • https://github.com/qurator-spk/eynollah/pull/33

• Vahid Rezanezhad (@vahidrezanezhad) Eynollah - multi-model trainable segmentation and region classification

  • https://github.com/qurator-spk/eynollah
  • https://github.com/qurator-spk/sbb_page_extractor (just the border detection/cropping)
  • https://github.com/qurator-spk/sbb_textline_detection/ (the predecessor in a way)

• Kai Labusch (@labusch) neat - online NER entity annotation (and GT editing!) tool

  • https://github.com/qurator-spk/neat

• Merlijn B. Wajer (MerlijnWajer) OCR @ Internet archive and script/lang detection

  • https://archive.org/services/docs/api/ocr.html
  • https://git.archive.org/users/merlijn/contributed

• Dynamic model selection from metadata

  • https://github.com/OCR-D/ocrd_tesserocr/pull/175
  • https://github.com/OCR-D/ocrd_tesserocr/issues/69

• Moving away from PIL? Where to?

  • https://docs.wand-py.org/en/0.6.5/
  • OpenCV
  • skimage
  • https://github.com/OCR-D/core/pull/676
  • https://github.com/OCR-D/core/compare/exif-wand

• Workflow Server Demo

  • https://github.com/OCR-D/core/pull/652
  • https://hackmd.io/23-JzLp_Q96cb6T0ttoFIA

• Sunsetting Python 3.6

  • https://endoflife.date/python -> end-of-life in Dec 2021
  • numpy, matplotlib, pandas, scikit-image … don't support 3.6 anymore

• Mike Gerber: https://github.com/qurator-spk/ocrd-galley

  • Goal and scope of ocrd-galley
    • Provide a fast way to run tested processors in the Qurator team
    • K.I.S.S. (as far as that can be said…)
    • galley as in "Kombüse", eigenes Süppchen kochen
    • Isolated processors
      • TensorFlow versions
      • CUDA toolkit versions
  • Docker container image per module
    • e.g. Dockerfile-core
    • e.g. Dockerfile-ocrd_calamari
  • Python wrapper runs containers
    • Almost seamless shell integration
      • You just call e.g. ocrd-calamari-recognize when venv is active
      • Models are built into the container image
      • TODO: Volumes to mount user directories for models
    • setup.py
    • TODO: Test ocrd process

• Jochen Barth: https://github.com/jbarth-ubhd/blitzDrt

• Uwe Hartwig: https://github.com/ulb-sachsen-anhalt/digital-derivans

• Bernhard Liebl: https://github.com/poke1024/origami

  • OCR pipeline used in Leipzig for recognizing Berliner Börsen-Zeitung

• Resource Management and ocrd_all

  • https://github.com/OCR-D/ocrd_all/pull/240
  • https://github.com/OCR-D/core/pull/663#pullrequestreview-590316447

• Performance

  • numpy/BLAS; processing with RAM fs; Workflow-Server: Multiprocessing/Multithreading
  • https://github.com/OCR-D/ocrd_tesserocr/issues/157
  • https://github.com/OCR-D/core/pull/652

• Automatically choosing models from language metadata

  • https://github.com/OCR-D/ocrd_tesserocr/issues/69
  • https://github.com/PRImA-Research-Lab/PAGE-XML/issues/27

• OCR alignment and diff view for browse-ocrd

  • https://github.com/hnesk/browse-ocrd/issues/13

• Alternative PAGE-XML visualization CLI

  • https://github.com/GBN-DBP/page-xml-draw

• Integrating origami into OCR-D?

  • https://github.com/poke1024/origami
  • https://github.com/poke1024/origami/issues/7 (which shows some results)

• End of support for Python 3.5 in OCR-D: March 1, 2021

• New in 2.22.0

  • Resource Manager https://github.com/OCR-D/core/pull/559
  • ocrd workspace rename-group https://github.com/OCR-D/core/pull/655
  • OcrdPage.get_AllAlternativeImages https://github.com/OCR-D/core/pull/654

• Upcoming for 2.2x

  • Workflow Server (https://github.com/OCR-D/core/pull/652)

• Transcription with Excel

  • https://github.com/OCR-D/ocrd_segment/tree/excel_gt_writer
  • https://github.com/OCR-D/ocrd_segment/pull/48

• Resources (models, configurations) etc. for processor

  • https://github.com/OCR-D/core/pull/559
  • https://github.com/OCR-D/spec/pull/177

• ocrd_kraken rewrite for kraken 3.x

  • https://github.com/OCR-D/ocrd_kraken/pull/33

• 1 De-Keystoning / Page splitting (@hnesk via https://gitter.im/OCR-D/Lobby?at=5ee9ffc490cd6426c8116ab0 / @bertsky via https://gitter.im/OCR-D/Lobby?at=5ef482c2d65a3b0292ac4eb2)

• Evaluation of calamari 0.3, 1.0 and tesseract in relation to segmentations (Mike Gerber)

  • https://gist.github.com/mikegerber/7e16e814bf723f04c47c5a6e2a6ac1d1
  • https://github.com/OCR-D/ocrd_calamari/issues/8#issuecomment-733615695

• Transitioning ocrd_all to slim containers

  • https://github.com/qurator-spk/ocrd-galley/
  • https://github.com/OCR-D/ocrd_all/issues/69

• Efficiently working with preloaded models in processors

  • https://hackmd.io/23-JzLp_Q96cb6T0ttoFIA?both
  • https://www.tensorflow.org/tfx/tutorials/serving/rest_simple

• Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →
  Backwards-incompatible changes in ocrd_tesserocr and ocrd_cis

• • browse-ocrd - recent improvements

• sbb_binarize now in ocrd_all

• Wanted: Complex but realist example pages for upcoming regression tests

• OLA-HD client

• browse-ocrd - what it is and how it is used in a scan workflow

• resegment inefficiency: https://github.com/cisocrgroup/ocrd_cis/issues/73#issuecomment-708945511

• clipping inefficiency: https://github.com/cisocrgroup/ocrd_cis/issues/74#issuecomment-708708

• Transitioning ocrd_all to slim containers

• OCR-D Web API - what it is and what it isn't -> Glossary of terms

• Technical questions in context of grant proposals deadline

• (Fixing https://github.com/OCR-D/ocrd_all/issues/195)

• OLA-HD client

• Logging Refactoring

• Diagnostics for workflows

• Revisiting the OCR-D Web API draft

• benchmarking

• API / OCR-D Butler

• CUDA: https://github.com/OCR-D/core/pull/454 / https://github.com/OCR-D/ocrd_all/pull/178

• New and upcoming features

  • sem/parallel woes in ocrd_all fixed
  • single output file group and adaption of processors
  • file parameter resolution

• ocrd-wf

• evaluation of workflow and intermediate results

• API

• Evaluation tools for Layout Analysis

• New in v2.13.0: Processor.input_files can handle PAGE-XML and images in the same folder -> let's adapt all processors

• Web APIs for OCR-D: Who is planning to build them and how can we coordinate?

• New features in 2.11.0+

• ocrd-sanitize-mets: Tool zum bereinigen von OCR-D mets.xml -> https://github.com/OCR-D/core/issue/544

• Building Debian/Ubuntu packages for OCR-D -> https://github.com/OCR-D/ocrd_all/issues/130

• Solution for maintaining GT in git (LFS, git-annex, …?)

• evaluation of workflow and intermediate results

• 1 ocrd-website wiki

• 1 image preprocessing new with ocrd_wrap, including generic shell wrapper

• 1 deploying models and presets

• 2 Handling rotation in metadata

• 3 Order of cropping and rotation

• 2 PAGE API extensions, esp. automatic polygon sanitation

• 1 GPU-enabled docker base image https://github.com/OCR-D/core/pull/452 / https://github.com/OCR-D/core/pull/454

• 3 Getting rid of –mets-basename

• 2 behavior if processor is called w/o args

• 3 ocrd_tool: file parameter relative filename resolution order (Issue: #160)

• 3 parameter presets

• 3 AlternativeImage in output fileGrp, AlternativeImage in structMap

• 3 repeatable -p

• 3 global mets:structMap/mets:fptr

• 3 structMap[@TYPE=OCR-D-LOGICAL] / FULLDOWNLOAD

• 3 uniform parameters for common settings

• 1 sample-calls for ocrd-tool.json

• 1 Building deb packages for OCR-D

• 3 Managing OCR models

• Evaluation of the OCR-D-processors on representative GT

• documenting workflows (Konstantin)

• checking and producing valid coordinates (Robert)

• recursive/2nd-level regions (Robert)

• derived images under same fileGrp (Robert)

• Tensorflow 1.15 vs 2.0.0 vs 2.1.0

• ignore PrintSpace (Robert) v2.7.0

• 3 ocrd-tool.json: structure of input/output group Closed

• 3 bashlib: check fo 4.4.+ v2.8.2

• 3 processor –overwrite v2.10.0

• 3 XSD validation v2.9.0

• 3 check file exists before ocrd workspace add v2.8.2

• 3 filename convention for bulk add v2.10.0

• 2 bashlib logging v2.10.0

• 2 validate pcgtsid == mets:file/@ID v2.8.2

• 2 validate mets:file/@ID syntax v2.9.0

• 2 ocrd_all release management https://github.com/OCR-D/ocrd_all/releases

(Problem) reports from testers

Conference details

Meeting Room Name: OCR-D Open TechCall
Link: https://meet.gwdg.de/b/eli-ufa-unu
Telephone number: will be posted right before each meeting in the Gitter Chatroom