Efficient Exploration of Image Classifier Failures with Bayesian Optimization and Text-to-Image Models - IRT SystemX Access content directly
Conference Papers Year : 2024

Efficient Exploration of Image Classifier Failures with Bayesian Optimization and Text-to-Image Models

Houssem Ouertatani
  • Function : Author
  • PersonId : 1248955
Faouzi Adjed

Abstract

Image classifiers should be used with caution in the real world. Performance evaluated on a validation set may not reflect performance in the real world. In particular, classifiers may perform well for conditions that are frequently encountered during training, but poorly for other infrequent conditions. In this study, we hypothesize that recent advances in text-to-image generative models make them valuable for benchmarking computer vision models such as image classifiers: they can generate images conditioned by textual prompts that cause classifier failures, allowing failure conditions to be described with textual attributes. However, their generation cost becomes an issue when a large number of synthetic images need to be generated, which is the case when many different attribute combinations need to be tested. We propose an image classifier benchmarking method as an iterative process that alternates image generation, classifier evaluation, and attribute selection. This method efficiently explores the attributes that ultimately lead to poor behavior detection.
Fichier principal
Vignette du fichier
Efficient Exploration of Image Classifier Failures with Bayesian Optimization and Text-to-Image Models.pdf (2.54 Mo) Télécharger le fichier
main.pdf (2.49 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04549384 , version 1 (18-04-2024)
hal-04549384 , version 2 (22-04-2024)

Identifiers

  • HAL Id : hal-04549384 , version 2

Cite

Adrien Le Coz, Houssem Ouertatani, Stéphane Herbin, Faouzi Adjed. Efficient Exploration of Image Classifier Failures with Bayesian Optimization and Text-to-Image Models. Generative Models for Computer Vision - CVPR 2024 Workshop, Jun 2024, Seattle, United States. ⟨hal-04549384v2⟩
38 View
13 Download

Share

Gmail Mastodon Facebook X LinkedIn More