Sélection de la langue

Gouvernement du Canada / Government of Canada

Recherche

Warning: statistical benchmarking is addictive, kicking the habit in machine learning

Par Conseil national de recherches du Canada

DOITrouver le DOI : https://doi.org/10.1080/09528130903010295
AuteurRechercher : 1; Rechercher :
Affiliation
  1. Conseil national de recherches du Canada. Institut de technologie de l'information du CNRC
FormatTexte, Article
Sujetmachine learning; algorithm evaluation; benchmarking; null hypothesis tests
Résumé
Algorithm performance evaluation is so entrenched in the machine learning community that one could call it an addiction. Like most addictions, it is harmful and very difficult to give up. It is harmful because it has serious limitations. Yet, we have great faith in practicing it in a ritualistic manner: we follow a fixed set of rules telling us the measure, the data sets and the statistical test to use. When we read a paper, even as reviewers, we are not sufficiently critical of results that follow these rules. Here, we will debate what are the limitations and how to best address them. This article may not cure the addiction but hopefully it will be a good first step along that road.
Date de publication
Maison d’éditionTaylor & Francis
Dans
  • Journal of Experimental and Theoretical Artificial Intelligence 22, nº 1 : 67–80.
Langueanglais
Publications évaluées par des pairsOui
Numéro NPARC23002090
Exporter la noticeExporter en format RIS
Signaler une correctionSignaler une correction (s'ouvre dans un nouvel onglet)
Identificateur de l’enregistrementa579582d-6412-4e35-b434-e89e008e276a
Enregistrement créé2017-08-10
Enregistrement modifié2020-04-16
Date de modification :