We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
Multilingual semantic fusion network for text recognition in the wild.
- Authors
Lou, Celi; Tong, Minglei; Xue, Liang; Kumarawadu, Sisil
- Abstract
Most current approaches in the literature of scene text recognition train the language model via a text dataset far sparser than in natural language processing, resulting in inadequate training. Therefore, we propose a simple transformer encoder–decoder model called the multilingual semantic fusion network (MSFN) that can leverage prior linguistic knowledge to learn robust language features. First, we label the text dataset with forward, backward sequences, and subwords, which are extracted by tokenization with linguistic information. Then we introduce a multilingual model to the decoder corresponding to three different channels of the labeled dataset. The final output is fused by different channels to get more accurate results. In experiments, MSFN achieves cutting-edge performance across six benchmark datasets, and extensive ablative studies have proven the effectiveness of the proposed method. Code is available at https://github.com/lclee0577/MLViT.
- Subjects
NATURAL language processing; LANGUAGE models; TEXT recognition; COMPUTER vision
- Publication
Journal of Electronic Imaging, 2023, Vol 32, Issue 2, p23015
- ISSN
1017-9909
- Publication type
Article
- DOI
10.1117/1.JEI.32.2.023015