Evaluation

NMT 133 Evaluating Gender Bias in Machine Translation

Author: Akshai Ramesh, Machine Translation Scientist @ Iconic Introduction We often tend to personify aspects of life that may vary based upon the beholder's interpretation. There are plenty of examples for this - “Mother Earth”, Doctor (Men), Cricketer (Men), Nurse(Woman), Cook(Woman), etc. The MT systems are trained with a large amount of parallel corpus which encodes this social bias. If that is the case, then to what...

Read More
NMT 119 Machine Translationese _ Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation

This week we have a guest post from Eva Vanmassenhove, Assistant Professor at Tilburg University, Dimitar Shterionov, Assistant Professor at Tilburg University, and Matt Gwilliam, from the University of Maryland. In Translation Studies, it is common to refer to a term called "translationese" that encapsulates a set of linguistic features commonly present in human translations as opposed to originally written texts. Researchers in the Machine Translation...

Read More
NMT 92 The Importance of References in Evaluating MT Output

Author: Dr. Carla Parra Escartín, Global Program Manager @ Iconic Introduction Over the years, BLEU has become the “de facto standard” for Machine Translation automatic evaluation. However, and despite being the metric being referenced in all MT research papers, it is equally criticized for not providing a reliable evaluation of the MT output. In today’s blog post we look at the work done by Freitag et al....

Read More
NMT 80 Evaluating Human-Machine Parity in Language Translation

Author: Jack Boylan, Data Scientist @ Iconic This is the first in a 2-part post addressing machine translation quality evaluation - an overarching topic regardless of the underlying algorithms. In this first part, Iconic Data Scientist Jack Boylan takes a look at a prominent study by Läubli et al. (2020) on how to carry out reliable evaluations. Next week, we’ll hear from one of the authors...

Read More