Supervised learning for low-resource isolated glyph recognition in palm leaf manuscripts

Pattern Recognition, vol. 172, pp. 112616

Abstract

• Assess the impact of the preprocessing steps (padding, color conversion and resizing) on the performance of deep learning models. • Evaluate the effectiveness of transfer learning, focusing on different pre-trained models and fine-tuning strategies for low-resource and imbalanced datasets. • Propose a robust methodology for recognizing text in non-Latin historical palm leaf manuscripts, addressing the challenges of limited data availability and class imbalance. • Analyse the influence of image quality and class distribution on classification performance. Handwriting recognition (HWR) is currently being adapted to automatically recognize text from digitized historical archives. There has been extensive research on HWR, but studies focusing on old scripts are relatively limited. This is mainly due to script style variations, poor document quality, and a shortage of old script experts. Deep learning algorithms have made significant breakthroughs in the field of HWR. This study evaluates classification tasks with 13 different systems: VGG19, ResNet50V2, ResNet101V2, ResNet152V2, InceptionResNetV2, MobileNetV1, MobileNetV2, NASNetMobile, EfficientNetV2B0, EfficientNetV2B3, EfficientNetV2S, EfficientNetV2M and EfficientNetV2S. Three main observations regarding the classification task are conducted (a) the impact of resizing, padding, and color conversion during the preprocessing step; (b) the effectiveness of transfer learning and fine-tuning in case of limitation of resources and imbalanced classes; (c) the influence of image quality, image quantity, and class labeling accuracy. Three collections of historical palm leaf manuscripts i.e old Sundanese, old Balinese, and old Khmer were investigated. For transfer learning two handwriting datasets i.e MNIST and Omniglot were included. We propose a robust pipeline for handling old scripts by leveraging an ensemble of customized CNN models to improve isolated glyph recognition. While both color padding and resizing method showed minimal performance variation, adjusting the image size proved crucial for achieving optimal results. Additionally, selecting the appropriate pre-trained model, fine-tuning it, and applying soft voting enabled our approach to consistently outperform the baseline across all test scenarios. As a result, the isolated glyph recognition performance on the Old Sundanese, Old Balinese and Old Khmer datasets reached 97.57%, 94.96%, and 92.19%, respectively.

Authors 3

1. Erick Paulus corresponding Aachen Leiden Institute of Advanced Computer Science

   Leiden University · Padjadjaran University

   Affiliation as printed

   Department of Computer Science, Universitas Padjadajaran, Raya Bandung Sumedang KM.21, Sumedang, 45363, Indonesia

   Leiden Institute of Advanced Computer Science, Leiden University, Einsteinweg 55, Leiden, 2333 CC, The Netherlands

2. Jean-Christophe Burie

   La Rochelle Université · Laboratoire Informatique, Image et Interaction (L3i)

   Affiliation as printed

   Laboratoire Informatique Image Interaction (L3i) University of La Rochelle, Avenue Michel Crépeau, 17042, La Rochelle Cedex 1, France

3. Fons J. Verbeek Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Einsteinweg 55, Leiden, 2333 CC, The Netherlands