Diffusion Models for Counterfactual Explanations

ACCV 2022 (Oral Presentation)
Guillaume Jeanneret, Loïc Simon, Frédéric Jurie
Code (GitHub) Paper (arXiv)

TL;DR

We propose to use Diffusion Models for Counterfactual Explanations (DiME). We propose a simple trick to include the gradients of the target classifier to guide the diffusion process.

Abstract

Counterfactual explanations have shown promising results as a post-hoc framework to make image classifiers more explainable. In this paper, we propose DiME, a method allowing the generation of counterfactual images using the recent diffusion models. By leveraging the guided generative diffusion process, our proposed methodology shows how to use the gradients of the target classifier to generate counterfactual explanations of input instances. Further, we analyze current approaches to evaluate spurious correlations and extend the evaluation measurements by proposing a new metric: Correlation Difference. Our experimental validations show that the proposed algorithm surpasses previous state-of-the-art results on 5 out of 6 metrics on CelebA.

Pipeline

DIME Pipeline Overview

Citation

@inproceedings{Jeanneret_2022_ACCV,
    author    = {Jeanneret, Guillaume and Simon, Lo\"ic and Fr\'ed\'eric Jurie},
    title     = {Diffusion Models for Counterfactual Explanations},
    booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)},
    month     = {December},
    year      = {2022}
}