TrOCR-Sin-Printed

A Sinhala text-line recognizer for printed text, released with the SinFUND and SinOCR datasets (SriDoc/datasets). For handwriting, use SriDoc/TrOCR-Sin-Handwritten.

Model

• TrOCR-style vision encoder-decoder (VisionEncoderDecoderModel).
• The encoder was initialized from facebook/deit-base-distilled-patch16-224 (ImageNet weights). The decoder was initialized from NLPC-UOM/SinBERT-large, a RoBERTa model pre-trained on Sinhala text. The cross-attention layers started from random weights.
• No OCR-pretrained TrOCR weights were used.
• Input: one pre-cropped text-line image, resized to 224 x 224.
• Decoding: beam search with 4 beams and a maximum length of 64 tokens.

Training data

The SinOCR printed training split: 90,000 synthetic text-line images of CC100 Sinhala text, rendered in 200 Sinhala fonts. Training ran for 75,000 steps (about 27 epochs), with a batch size of 32 and a learning rate of 5e-5.

Results

Character error rate (CER; lower is better) on the SinOCR test splits:

Test split	CER
SinOCR printed (10,000 lines)	0.1445
SinOCR handwritten (227 lines)	0.9940

How the CER is computed:

• Characters are Unicode code points.
• Runs of spaces are collapsed, and leading and trailing spaces are removed.
• The CER is pooled over the whole test split, not averaged per line.

SinOCR has no separate validation split, so the test split was also the evaluation set during training. The lowest test CER occurred at the final checkpoint, so checkpoint selection did not change this result.

Limitations

• The model reads single, pre-cropped lines. It does not find text on a page.
• It was trained on synthetic printed text only.

Usage

from transformers import VisionEncoderDecoderModel, TrOCRProcessor, AutoTokenizer, ViTImageProcessor
from PIL import Image

model = VisionEncoderDecoderModel.from_pretrained("SriDoc/TrOCR-Sin-Printed")
tokenizer = AutoTokenizer.from_pretrained("NLPC-UOM/SinBERT-large")
image_processor = ViTImageProcessor.from_pretrained("google/vit-base-patch16-224")
processor = TrOCRProcessor(image_processor=image_processor, tokenizer=tokenizer)

image = Image.open("line.png").convert("RGB")   # one cropped text line
pixel_values = processor(image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

Citation

@article{gunathilaka2025sinfund,
  title   = {SinFUND and SinOCR: Benchmarks for Sinhala Handwritten OCR and Template-Free Form Understanding},
  author  = {Gunathilaka, Kavishka and Hewagama, Danusha and Pushpakumara, Supul and Ambegoda, Thanuja D.},
  journal = {Research Square preprint},
  year    = {2025},
  doi     = {10.21203/rs.3.rs-6976719/v1}
}

Training log

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 75000
• mixed_precision_training: Native AMP

Training results

The "Cer" and "Validation Loss" columns were measured on the SinOCR printed test split.

Training Loss	Epoch	Step	Cer	Validation Loss
1.3019	1.78	5000	0.6416	1.7769
0.6387	3.55	10000	0.4048	0.8457
0.3402	5.33	15000	0.2808	0.6898
0.1332	7.11	20000	0.2377	0.5765
0.1141	8.89	25000	0.2223	0.4460
0.0481	10.66	30000	0.1868	0.4128
0.0391	12.44	35000	0.1563	0.4172
0.0357	14.22	40000	0.1981	0.4756
0.0215	16.0	45000	0.1983	0.5838
0.0129	17.77	50000	0.1757	0.5511
0.0087	19.55	55000	0.1699	0.5568
0.003	21.33	60000	0.1648	0.4532
0.0042	23.11	65000	0.1582	0.4650
0.0066	24.88	70000	0.1654	0.4740
0.0014	26.66	75000	0.1448	0.4337

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1