We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Language Model Pre-training Method in Machine Translation Based on Named Entity Recognition.
- Authors
Li, Zhen; Qu, Dan; Xie, Chaojie; Zhang, Wenlin; Li, Yanxia
- Abstract
Neural Machine Translation (NMT) model has become the mainstream technology in machine translation. The supervised neural machine translation model trains with abundant of sentence-level parallel corpora. But for low-resources language or dialect with no such corpus available, it is difficult to achieve good performance. Researchers began to focus on unsupervised neural machine translation (UNMT) that monolingual corpus as training data. UNMT need to construct the language model (LM) which learns semantic information from the monolingual corpus. This paper focuses on the pre-training of LM in unsupervised machine translation and proposes a pre-training method, NER-MLM (named entity recognition masked language model). Through performing NER, the proposed method can obtain better semantic information and language model parameters with better training results. In the unsupervised machine translation task, the BLEU scores on the WMT'16 English–French, English–German, data sets are 35.30, 27.30 respectively. To the best of our knowledge, this is the highest results in the field of UNMT reported so far.
- Subjects
MACHINE translating; MODEL railroads; INFORMATION modeling; LANGUAGE &; languages
- Publication
International Journal on Artificial Intelligence Tools, 2020, Vol 29, Issue 7/8, p1
- ISSN
0218-2130
- Publication type
Article
- DOI
10.1142/S0218213020400217