Language selection

Government of Canada / Gouvernement du Canada

Search

Arabic Preprocessing Schemes for Statistical Machine Translation

From National Research Council Canada

Download
  1. (PDF, 235 KiB)
AuthorSearch for: ; Search for:
FormatText, Article
ConferenceHuman Language Technology Conference/North American Chapter of the Association for Computational Linguistics (HLT/NAACL) 2006, June 5-7, 2006, New York City, New York, USA
Abstract
In this paper, we study the effect of different word-level preprocessing decisions for Arabic on SMT quality. Our results show that given large amounts of training data, splitting off only proclitics performs best. However, for small amounts of training data, it is best to apply English-like tokenization using part-of-speech tags, and sophisticated morphological analysis and disambiguation. Moreover, choosing the appropriate preprocessing produces a significant increase in BLEU score if there is a change in genre between training and test data.
Publication date
In
  • Proceedings of Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics (HLT/NAACL) 2006.
LanguageEnglish
NRC numberNRCC 48759
NPARC number9167805
Export citationExport as RIS
Report a correctionReport a correction (opens in a new tab)
Record identifier07fcd97a-570b-45e4-a32c-5edef880e6c1
Record created2009-06-29
Record modified2020-10-09
Date modified: