Counterfactual explanation based on gradual construction for deep networks

Hong Gyu Jung, Sin Han Kang, Hee Dong Kim, Dong Ok Won, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

To understand the black-box characteristics of deep networks, counterfactual explanation that deduces not only the important features of an input space but also how those features should be modified to classify input as a target class has gained an increasing interest. The patterns that deep networks have learned from a training dataset can be grasped by observing the feature variation among various classes. However, current approaches perform the feature modification to increase the classification probability for the target class irrespective of the internal characteristics of deep networks. This often leads to unclear explanations that deviate from real-world data distributions. To address this problem, we propose a counterfactual explanation method that exploits the statistics learned from a training dataset. Especially, we gradually construct an explanation by iterating over masking and composition steps. The masking step aims to select an important feature from the input data to be classified as a target class. Meanwhile, the composition step aims to optimize the previously selected feature by ensuring that its output score is close to the logit space of the training data that are classified as the target class. Experimental results show that our method produces human-friendly interpretations on various classification datasets and verify that such interpretations can be achieved with fewer feature modification.

Original languageEnglish
Article number108958
JournalPattern Recognition
Volume132
DOIs
Publication statusPublished - 2022 Dec

Keywords

  • Counterfactual explanation
  • Explainable AI
  • Generative model
  • Interpretability
  • Model-agnostics

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Counterfactual explanation based on gradual construction for deep networks'. Together they form a unique fingerprint.

Cite this