Bootstrapped OCR error detection for a less-resourced language variantReport as inadecuate

Bootstrapped OCR error detection for a less-resourced language variant - Download this document for free, or read online. Document in PDF available to download.

1 Berlin-Brandenburg Academy of Sciences 2 OeAW - Austrian Academy of Sciences

Abstract : This study focuses on isolated error detection in a retro-digitized newspaper corpus published from 1946 to 1990 in the former German Democratic Republic. As there are OCR errors throughout the corpus but no clean reference for this variant of German, automatic OCR correction implies to overcome data sparseness and non-standard spelling, including compounds and inflected forms. The contributions of this paper are 1 a method to bootstrap detection of potential misspellings, 2 an assessment of several types of training data, and 3 an evaluation of several off-the-shelf candidate selection techniques. The chosen solution based on statistical affix analysis reaches an accuracy 10 points higher than existing morphological analysis systems on error detection, while a combination of fuzzy and approximate string search performs best for error correction. The criteria are met since it is possible to correct erroneous tokens without introducing too much noise.

Keywords : OCR error correction Affix trees Cultural Heritage Morphological Analysis

Author: Adrien Barbaresi -



Related documents