New CAM Method: ShapleyCAM

[cai2-huaiguang]

Hi Jacob,

My name is Huaiguang Cai. I am a master student at Institute of Automation, Chinese Academy of Sciences.
I recently developed a new CAM technique called ShapleyCAM, which is introduced in my paper, "CAMs as Shapley Value-based Explainers". The paper clarifies the theoretical foundations of GradCAM, HiResCAM, and CAM using a game-theoretic perspective and introduces ShapleyCAM, a new method uses gradients and Hessian-Vector Products to weight activation maps.
Additionally, I analyze the theoretical advantages and limitations of using pre-softmax (aka logit) and post-softmax (aka probility) scores and introduce the Residual Softmax Target-Class (ReST) utility function to address these limitations. This paper has recently been accepted by by The Visual Computer.

Please let me know if you have any questions or need additional details. I’d love to add ShapleyCAM to your wonderful GradCAM repository.

Best regards,
Huaiguang Cai

[jacobgil]

Dear Huaiguang Cai,
This is impressive work, and a very welcome contribution, greatly appreciated.
The cam method is wonderful, and the Residual Softmax Target-Class can be very useful for all users in general.

I definitely want to merge this.

The main issue for merging is re-using existing code as much as possible.
For example, shapley_cam.py implements compute_cam_per_layer which seems almost one to one the same implementation as in base_cam.py, but it accesses the activations/gradients in the new activations and gradients object, with a different name.
Also the activations and gradients without detach. Maybe we could modify the original class to receive a parameter if it should detach or not, and then re-use it ?

[cai2-huaiguang]

Dear Jacob,

Thank you so much for the quick response! I’m really happy to receive your response and recognition. I completely agree with your point about reusing existing code as much as possible. I will carefully review and refactor the code to integrate it more smoothly with the existing implementation.

Please give me a day, and I’ll submit the updated version.

[jacobgil]

Hi, is the forward function in shapely_cam.py still needed? Looks like now it's exactly the same as in the base class, no?
(got to it because of the lint error of the missing import in the build that failed above)

[cai2-huaiguang]

Hi, let me check for a while

[cai2-huaiguang]

Hi, ShapleyCAM relys on the calculation of the Hessian-vector product (HVP), which requires the computation graph to be keep, thus I need change the loss.backward(retain_graph=True) to torch.autograd.grad(loss, input_tensor, retain_graph = True, create_graph = True) in the forward function.

[jacobgil]

Merged!! Thanks so much for your contribution, this is great.

[jacobgil]

I'm testing this a bit, and the result seems to be exact to grad-cam. Is this expected ?

[jacobgil]

Is it because were are using the first layer before GAP, and the hessian is 0 because there is only 1 linear layer after it?

[cai2-huaiguang]

Sorry for the late reply; I spent some time working on this.

Is it because were are using the first layer before GAP, and the hessian is 0 because there is only 1 linear layer after it?

Yes! Suppose we use ResNet (regardless of which layer is target layer), when we use pre-softmax (i.e., targets = [ClassifierOutputTarget(281)] in cam.py), the calculated Hessian is 0 or None (and ShapleyCAM becomes equivalent to GradCAM), as shown in this example:

Actually, for most CNN networks, the Hessian matrix obtained using pre-softmax is zero because operations like ReLU and convolution are linear. It is only when a nonlinear activation function is used (such as in EfficientNet) that the Hessian becomes non-zero.

However, when we use post-softmax or ReST (i.e., targets = [ClassifierOutputReST(281)] in cam.py), the Hessian will not be all zeros:

However, I must admit that the visual difference between GradCAM and ShapleyCAM is quite small in most CNN networks, and the metrics, such as ADCC, also show minimal disparity like in my paper. This is likely because the Hessian matrix is relatively small, allowing the first-order terms to play a dominant role.

But in certain scenarios (such as using ReST in VGG-16, or when the internal structure of the network is highly non-linear like swin-transformer), ShapleyCAM and GradCAM can produce noticeably different results as in my paper. For instance, when using the last convolutional layer of VGG-16 as the target layer and applying ReST, and target class is boxer(targets = [ClassifierOutputReST(242)]), here is GradCAM :

And hereis ShapleyCAM

All my visual examples of my paper are in https://github.com/caihuaiguang/pytorch-shapley-cam/blob/master/tutorials/vgg16_pictures.ipynb or other ipynb file.

Please feel free to reach out if you have any questions or require further information. Your efforts for the GradCAM repository are greatly appreciated.

[jacobgil]

Thanks for the explanation.