| Download | - View final version: MEANT 2.0: accurate semantic MT evaluation for any output language (PDF, 354 KiB)
|
|---|
| DOI | Resolve DOI: https://doi.org/10.18653/v1/W17-4767 |
|---|
| Author | Search for: Lo, Chi-Kiu1 |
|---|
| Name affiliation | - National Research Council of Canada. Digital Technologies
|
|---|
| Format | Text, Article |
|---|
| Conference | The 2nd Conference on Machine Translation (WMT), September 7-8, 2017, Copenhagen, Denmark |
|---|
| Abstract | We describe a new version of MEANT, which participated in the metrics task of the Second Conference on Machine Translation (WMT 2017). MEANT 2.0 uses idf-weighted distributional ngram accuracy to determine the phrasal similarity of semantic role fillers and yields better correlations with human judgments of translation quality than earlier versions. The improved phrasal similarity enables a subversion of MEANT to accurately evaluate translation adequacy for any output language, even languages without an automatic semantic parser. Our results show that MEANT, which is a non-ensemble and untrained metric, consistently performs as well as the top participants in previous years - including ensemble and trained ones - across different output languages. We also present the timing statistics for MEANT for better estimation of the evaluation cost. MEANT 2.0 is open source and publicly available. |
|---|
| Publication date | 2017-09-11 |
|---|
| Publisher | Association for Computational Linguistics (ACL) |
|---|
| In | |
|---|
| Language | English |
|---|
| Peer reviewed | Yes |
|---|
| Export citation | Export as RIS |
|---|
| Report a correction | Report a correction (opens in a new tab) |
|---|
| Record identifier | e873a590-859e-4c30-a39f-3201fd3696b0 |
|---|
| Record created | 2019-04-08 |
|---|
| Record modified | 2020-05-30 |
|---|
