By the way, in some consoles, the macron in the second variant is noticeably shifted. This is the same rendering issue that forced us to adjust character frames in step 3. In our scripts, such normalization is performed for each text before rendering. After this, each letter has exactly one representation. The model sees the same codes for identical letters and doesn't learn anything extra.
Step 2. Drawing strings with different fonts
Printed publications are typeset very differently. To prevent the model from getting used to a single font, we took dozens of font families. Among them are FreeSerif, DejaVu Sans, Arial, Calibri, and Times New Roman. Each line was rendered in several sizes (12, 14, 24, and 26 pt) and styles (normal, bold, italic). This way, the model learns to recognize the letter, not a specific image of the letter.
Step 3. Marking each character
Tesseract 5 can learn from entire lines without marking individual letters. However, we additionally marked each character with a box file containing its coordinates. This allowed us to check that the letter, including the macron, was entirely within the marking, and the line above it was not lost. Usually, the coordinates are calculated automatically based on the font metrics. For letters with diacritics, such calculations often make mistakes because the macron extends above the line. Also, each font draws it differently.
We fixed this with pixel-level corrections. For this purpose, we created three dictionaries:
CHAR_OFFSETS. Character offsets.
CHAR_SIZE. Corrections depending on the font size.
RATE. Coefficients for a specific font family.
We experimentally adjusted the values, looking at the results. In the end, the boxes were precisely placed on the characters, including those with macrons (Fig. 1).
Step 4. Cutting the text into lines
A long text needs to be split into fragments that fit within the width of the generated image. For this purpose, our scripts have a function called split_by_width. It cuts the line so that no character goes beyond the image's edge. So the text on the image and the text in the markup match one-to-one.
In the end, the scripts generated 1.3 million file pairs. Each pair consists of an image of a line in .tif format and a text file .gt.txt with the correct answer.
How we retrained Tesseract
Why Tesseract
Tesseract is an open-source OCR engine that has been developing for many years. It is free, works locally, has pre-trained models for hundreds of languages, and built-in tools for retraining. For a low-resource language, this is a convenient starting point. There is no need to train the model from scratch, you can fine-tune an existing one.
How it reads a line
Inside Tesseract 5, a CRNN neural network works. Its convolutional part finds strokes, curves, and features above the letters on the image. The recurrent part (LSTM) reads these features from left to right, roughly like a human reads a line. Manual annotation of where each letter ends is not needed. This is handled by the CTC loss function.
Transfer learning
As a base, we used the official Russian model rus.traineddata from the tessdata_best repository. It already knows Cyrillic well. The Mansi alphabet is almost entirely the same as Russian, so the model needs to mainly learn the new letters with macron and ӈ. It's similar to a person reading Russian and realizing that a letter with a line on top is a separate character.
For retraining, we used tesstrain, the official set of scripts for training Tesseract. It's enough to pass the initial model and a folder with 'image + text' pairs. Starting with rus.traineddata as the base, tesstrain does three things on its own.
It extracts the LSTM part with pre-trained weights from the Russian model.
It examines the training data and collects a complete list of characters from the Mansi alphabet.
It creates a new model with the Russian weights and expands the output layer for the combined alphabet. This way, the model has "places" for а̄, э̄, ӯ, ӈ, and other new letters.
The most important parameter turned out to be the number of iterations. If trained too little, the model doesn't have time to learn the new letters. If trained too long, it overfits on synthetic data and performs worse on real texts. We selected the number of iterations based on the quality on the validation set. To prevent overfitting, we used early stopping. Training was interrupted as soon as the validation metrics stopped increasing or started to deteriorate. The result was a specialized model mns.traineddata.
What resulted
OCR quality is usually evaluated using two standard metrics. CER (Character Error Rate) shows the proportion of characters recognized with errors. WER (Word Error Rate) shows the same for words. The lower the metric values, the better. To assess the contribution of fine-tuning, we compared our model with the original rus.traineddata from tessdata_best. Model CER WER rus.traineddata (basic) 18.5% 42.1% mns.traineddata (our) 0.85% 1% Our model makes fewer than one error in every 100 characters and one error in every 100 words. The Russian model made 18 characters and 42 words errors in the same text. The character errors became about 20 times less. Where does such a difference come from, clearly visible on one line. Variant Text Original line Хӯл тэ̄нутыг ва̄рнэ ма̄гыс rus.traineddata (basic) Хул тэнутыг варнэ магьгс mns.traineddata (our) Хӯл тэ̄нутыг ва̄рнэ ма̄гыс The basic model lost all macrons, and the letter "ы" turned into "ьг". The fine-tuned model read the line without a single error. The main thing she learned was to distinguish a from ā, u from ū, and recognize the letter ӈ.
Such accuracy is already sufficient to use the model for digitizing printed Mansi texts. A person only needs to proofread the result, rather than re-entering it from scratch.
Where to find the model
The model is open and available on Hugging Face, here is its page URIIT/mns-tesseract. To connect it to your Tesseract 5, it's enough to copy the file mns.traineddata into the tessdata folder and specify the language mns when launching. You can try recognition in the browser on Этнопортале.
Take into account possible limitations. The model confidently reads clean typographic text with font sizes from 12 to 26 pt. It performs worse with very small or large fonts, with Latin script, and with old editions where the print has faded.
Summary
For OCR of a low-resource language, it's not necessary to manually annotate scans for years. We managed with three steps.
Bring all letters with macrons to a unified form in Unicode.
Generate a synthetic dataset from real Mansi texts, dozens of fonts, and precise markup of each character.
Fine-tune the pre-trained Russian Tesseract 5 model instead of training from scratch.
As a result, the percentage of erroneous characters decreased from 18.5% to 0.85%, and the percentage of erroneous words decreased from 42.1% to 1%. Printed Mansi texts can now be quickly converted into digital form. They enrich the corpus used to train neural machine translators, language models, and other project models. We think the same recipe might work for other languages with Cyrillic diacritics, for example, for the Khanty or Nenets languages.
Next step for the model is related to current archival scans. Old publications with faded print and outdated fonts are currently the hardest for it. We want to add such scans to the training so that the model can handle them more confidently.
In the following articles of the series, we will talk about other models of the project. Upcoming are the language model, speech recognition and synthesis of the Mansi language.
A detailed scientific description of the work is published in the journal «Vestnik YUGU. Series Mathematical Modeling and Programming» (DOI 10.14529/mmp260206). There you can find a literature review and details of the methodology.
Useful links:
Part 1. Neural translator
Ethnoportal with project models
Documentation on training Tesseract 5
Stay with us. Neural networks help us not only in working with language, and we will also talk about this.
