Analysing Patterns of Errors in Neural and Statistical Machine Translation of Arabic and English

Abstract

This paper provides a comparative analysis of two Machine Translation (MT) engines: Google Translate (GT) and Microsoft Bing (MB). Previously, these MT engines adopted the statistical approach in their system. However, they are currently using the latest neural approach in their engines, which has become a trend in the MT field. The present data discusses the quality of the outputs by comparing the previous data using the SMT engines with the current data from the NMT engines. This paper also analyses the patterns of errors that exist in the MT outputs, which were generated using four Arabic texts and 5 English texts. Results reported a significant decrease of 72.2% and 73.1% in the number of errors found in GT and MB, respectively, with most of them are syntactic errors and incorrect terms. Missing conjunctions and determiners were also reported to be common mistakes in the analysis. Generally, the adequacy of both NMT engines has improved for English-Arabic language pairs. Even though the errors still exist, most of them can be easily corrected if thoroughly revised.