| Téléchargement | - Voir la version finale : Calibration and context in human evaluation of machine translation (PDF, 4.5 Mio)
- Voir les données supplémentaires : Calibration and context in human evaluation of machine translation (PDF, 10.1 Mio)
|
|---|
| DOI | Trouver le DOI : https://doi.org/10.1017/nlp.2024.5 |
|---|
| Auteur | Rechercher : Knowles, Rebecca1Identifiant ORCID : https://orcid.org/0000-0002-1647-584X; Rechercher : Lo, Chi-kiu1Identifiant ORCID : https://orcid.org/0000-0001-8714-7846 |
|---|
| Affiliation | - Conseil national de recherches Canada. Technologies numériques
|
|---|
| Format | Texte, Article |
|---|
| Sujet | machine translation; evaluation |
|---|
| Résumé | Human evaluation of machine translation is considered the “gold standard” for evaluation, but it remains a challenging task for which to define best practices. Recent work has focused on incorporating intersentential context into human evaluation, to better distinguish between high-performing machine translation systems and human translations. In this work, we examine several ways that such context influences evaluation and evaluation protocols. We take a close look at annotator variation through the lens of calibration sets and focus on the implications for context-aware evaluation protocols. We then demonstrate one way in which degraded target-side intersentential context can influence annotator scores of individual sentences, a finding that supports the context-aware approach to evaluation and which also has implications for best practices in evaluation protocols. |
|---|
| Date de publication | 2024-06-03 |
|---|
| Maison d’édition | Cambridge University Press (CUP) |
|---|
| Licence | |
|---|
| Dans | |
|---|
| Langue | anglais |
|---|
| Publications évaluées par des pairs | Oui |
|---|
| Exporter la notice | Exporter en format RIS |
|---|
| Signaler une correction | Signaler une correction (s'ouvre dans un nouvel onglet) |
|---|
| Identificateur de l’enregistrement | 38f5f3ec-1a13-4100-bb65-f21273d1bccb |
|---|
| Enregistrement créé | 2024-06-04 |
|---|
| Enregistrement modifié | 2024-06-04 |
|---|
