| Author | Search for: Bellinger, Colin1ORCID identifier: https://orcid.org/0000-0002-3567-7834; Search for: Corizzo, Roberto; Search for: Japkowicz, Nathalie |
|---|
| Affiliation | - National Research Council of Canada. Digital Technologies
|
|---|
| Format | Text, Article |
|---|
| Conference | 15th International Workshop on Learning with Imbalanced Domains: Theory and Applications, LIDTA 2023, September 18, 2023, LIDTA, Turin, Italy |
|---|
| Subject | bias; class imbalance; group fairness; minority subconcepts; performance evaluation |
|---|
| Abstract | Learning classifiers from imbalanced data is known to be a challenging and important problem in machine learning. As a results, the topic has been studied from a wide variety of angles. This includes the choice of evaluation measures and understanding the implications of minority class subconcepts on model learning. In this work, however, we argue that the community may not be using precise enough evaluation measures when assessing the performance of imbalanced learning pipelines on data that includes an imbalance in the minority class subconcepts. We show that the performance estimates from standard measures used in imbalance learning are biased towards the largest minority subconcepts, and that standard imbalance correction techniques can exacerbate the bias. Finally, we demonstrate that the bias can, in part, be corrected by applying instance weighting in the evaluation measures. |
|---|
| Publication date | 2023-09-18 |
|---|
| Publisher | ML Research Press |
|---|
| In | |
|---|
| Language | English |
|---|
| Peer reviewed | Yes |
|---|
| Export citation | Export as RIS |
|---|
| Report a correction | Report a correction (opens in a new tab) |
|---|
| Record identifier | 569bf8a6-ad66-45ec-b287-aa6b91a60f54 |
|---|
| Record created | 2024-09-26 |
|---|
| Record modified | 2024-09-26 |
|---|
