This repository contains training scripts to train a text detector based on manga-image-translator which can extract bounding-boxes, text lines and segmentation of text from manga or comics to help further comics translation procedures such as text-removal, recognition, lettering, etc.
There are some awesome projects such as Lancet.
Download the text detection model from zyddnys/manga-image-translator or Google Drive.
(source: manga109, © Yoshi Masako)
Our current model can be summarized as below.
All models were trained on around 13 thousand anime & comic style images, 1/3 from Manga109-s, 1/3 from DCM, and 1/3 are synthetic data in a weak supervision manner due to the lack of available high-quality annotations.
We used text detection model of manga-image-translator to generate text lines annotations for manga, and Manga-Text-Segmentation with some post-processing to generate masks for both manga and comics. Synthetic data were generated using around 4k text-free anime-girls pictures from https://t.me/SugarPic, text-rendering, Unet and DBNet training scripts can be found in this repo. Text block detector was trained using yolov5 official repository
We would not (don't have the right) share training sets or fonts publicly. 2/3 of the training set is not so clean anyway, so the training is reproducible only if you have enough images and fonts. You can use the models this repo provided to generate labels for comics/manga, and the comic style text rendering script to generate synthetic data. Please refer to examples.ipynb for more details.