Before ingesting material into the portal, we should have a basic test that confirms whether or not the material was reasonably well recognized.
Proposal:
Most images that do poorly will be due to improper segmentation. In such cases, the final document will have a high number of lines with zero text.
Check how often the recognizer returns nothing and if the amount is above a certain percentage threshold, reject the output.
Ideally, this could be passed as a flag.
It seems easier to do this during inference rather than parsing the hOCR files at the end and checking the output.
Before ingesting material into the portal, we should have a basic test that confirms whether or not the material was reasonably well recognized.
Proposal:
Most images that do poorly will be due to improper segmentation. In such cases, the final document will have a high number of lines with zero text.
Check how often the recognizer returns nothing and if the amount is above a certain percentage threshold, reject the output.
Ideally, this could be passed as a flag.
It seems easier to do this during inference rather than parsing the hOCR files at the end and checking the output.