RUS  ENG
Полная версия
ЖУРНАЛЫ // Информатика и её применения

Информ. и её примен., 2021, том 15, выпуск 2, страницы 104–111 (Mi ia735)

Методы оценки качества машинного перевода: современное состояние
В. А. Нуриев, А. Ю. Егорова

Литература

1. Larsonneur C., Neural machine translation: From commodity to commons?, When translation goes digital: Case studies and critical reflections, eds. R. Desjardins, C. Larsonneur, Ph. Lacour, Palgrave Macmillan, Cham, Switzerland, 2021, 257–280  crossref
2. Davenport C., Google Translate processes 143 billion words every day, Android Police, 2018 https://www.androidpolice.com/2018/10/09/google-translate-processes-143-billion-words-every-day (accessed May 5, 2021)
3. J. Moorkens, Sh. Castilho, F. Gaspari, S. Doherty (eds.), Translation quality assessment: From principles to practice, v. 1, Machine translation: Technologies and applications ser., Springer International Publishing, Cham, Switzerland, 2018, 292 pp.  crossref
4. Specia L., Scarton C., Paetzold G. H., Quality estimation for machine translation, Synthesis lectures on human language technologies ser., Morgan & Claypool, London, 2018, 162 pp.
5. Bittner H., Evaluating the evaluator: A novel perspective on translation quality assessment, Routledge, New York, NY, USA, 2020, 282 pp.
6. Papineni K., Roukos S., Ward T., Zhu W. J., “BLEU: A method for automatic evaluation of machine translation”, 40th Annual Meeting on Association for Computational Linguistics Proceedings, Association for Computational Linguistics, Philadelphia, PA, USA, 2002, 311–318
7. Рычихин А. К., “О методах оценки качества машинного перевода”, Системы и средства информатики, 29:4 (2019), 106–118  mathnet [Rychikhin A. K., “On methods of machine translation quality assessment”, Sistemy i Sredstva Informatiki — Systems and Means of Informatics, 29:4 (2019), 106–118]
8. Козина А. В., Черепков Е. А., Белов Ю. С., “Автоматические метрики оценки качества машинного перевода”, Системный администратор, 2019, № 11, 84–87 [Kozina A. V., E. A. Cherepkov, Yu. S. Belov, “Automatic metrics for machine translation evaluation”, System Administrator, 2019, no. 11, 84–87]
9. Banerjee S., Lavie A., “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments”, Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd Annual Meeting of the Association of Computational Linguistics Proceedings, Association of Computational Linguistics, Ann Arbor, MI, USA, 2005, 65–72
10. Koehn Ph., Neural machine translation, Cambridge University Press, New York, NY, USA, 2020, 394 pp.
11. Popović M., “chrF: Character $n$-gram F-score for automatic MT evaluation”, 10th Workshop on Statistical Machine Translation Proceedings, Association for Computational Linguistics, Lisboa, Portugal, 2015, 392–395  crossref
12. Popović M., “chrF deconstructed: $\beta$ parameters and $n$-gram weights”, 1st Conference on Machine Translation Proceedings, v. 2, Association for Computational Linguistics, Berlin, Germany, 2016, 499–504
13. Chi-kiu Lo, “MEANT 2.0: Accurate semantic MT evaluation for any output language”, Conference on Machine Translation Proceedings, v. 2, Association for Computational Linguistics, Copenhagen, Denmark, 2017, 589–597
14. Stanojević M., Sima'an K., “BEER: BEtter evaluation as ranking”, 9th Workshop on Statistical Machine Translation Proceedings, Association for Computational Linguistics, Baltimore, MD, USA, 2014, 414–419  crossref
15. Stanojević M., Sima'an K., “Evaluating MT systems with BEER”, Prague Bulletin Mathematical Linguistics, 104 (2015), 17–26  crossref
16. Sellam T., Das D., Parikh A. P., BLEURT: Learning robust metrics for text generation, 9 Apr 2020, arXiv: 2004.04696 (accessed May 5, 2021)
17. Инькова О. Ю., “Надкорпусная база данных как инструмент изучения формальной вариативности коннекторов”, Компьютерная лингвистика и интеллектуальные технологии, 17(24), РГГУ, М., 2018, 240–253 [Inkova O. Yu., “Supracorpora database as an instrument of the study of the formal variability of connectives”, Computer Linguistic and Intellectual Technologies, 17(24), M., 2018, 240–253]
18. Castilho Sh., Doherty S, Gaspari F., Moorkens J., “Approaches to human and machine translation quality assessment”, Translation quality assessment: From principles to practice, eds. J. Moorkens, Sh. Castilho, F. Gaspari, S. Doherty, Springer, Cham, Switzerland, 2018, 9–38  crossref
19. Likert R., “A technique for the measurement of attitudes”, Arch. Psychol., 140 (1932), 1–55
20. Fleiss J. L., “Measuring nominal scale agreement among many raters”, Psychol. Bull., 76:5 (1971), 378–382  crossref
21. Shterionov D., Superbo R., Nagle P., et al., “Human versus automatic quality evaluation of NMT and PBSMT”, Machine Translation, 32 (2018), 217–235  crossref
22. Castilho S., Moorkens J., Gaspari F., et al., “Evaluating MT for massive open online courses. A multifaceted comparison between PBSMT and NMT systems”, Machine Translation, 32 (2018), 255–278  crossref
23. Popovic M., “Error classification and analysis for machine translation quality assessment”, Translation quality assessment: From principles to practice, 2018, 129–158, Springer, Cham, Switzerland  crossref
24. Lommel A., “Metrics for translation quality assessment: A case for standardizing error typologies”, Translation quality assessment: From principles to practice, eds. J. Moorkens, Sh. Castilho, F. Gaspari, S. Doherty, Springer, Cham, Switzerland, 2018, 109–127  crossref
25. Klubička F., Toral A., Sánchez-Cartagena V. M., “Quantitative fine-grained human evaluation of machine translation systems: A case study on English to Croatian”, Machine Translation, 32 (2018), 195–215  crossref
26. Haque R., Hasanuzzaman M., Way A., “Analysing terminology translation errors in statistical and neural machine translation”, Machine Translation, 34 (2020), 149–195  crossref
27. Vilar D., Xu J., D'Haro L., Ney H., “Error analysis of statistical machine translation output”, 5th Conference (International) on Language Resources and Evaluation Proceedings, European Language Resources Association, Genoa, Italy, 2006, 697–702
28. Гончаров А. А., Бунтман Н. В., Нуриев В. А., “Ошибки в машинном переводе: проблемы классификации”, Системы и средства информатики, 29:3 (2019), 92–103  mathnet [Goncharov A. A., N. V. Buntman, V. A. Nuriev, “Machine translation errors: Problems of classification”, Sistemy i Sredstva Informatiki — Systems and Means of Informatics, 29:3 (2019), 92–103]
29. Calixto I., Liu Q., “An error analysis for image-based multi-modal neural machine translation”, Machine Translation, 33 (2019), 155–177  crossref


© МИАН, 2025