francescosaveriozuppichini / a-journey-into-convolutional-neural-network-visualization- Goto Github PK

Hi @FrancescoSaverioZuppichini,
I have used the EfficientNet_B2 model from torchvision and modified the input and output layers for binary classification of grayscale images. Now, I want to visualize the areas of interest in the model using GradCam. However, it seems that your function does not provide implementation for binary classification (or I might be using it incorrectly). I encountered the following error. Can you please help me? Thanks in advance!

Package Version:
torch 2.1.0
torchvision 0.16.0

Modify Source Code:

• GradCam.py

# predictions = self.module(input_var)
predictions = torch.nn.Sigmoid()(self.module(input_var))

• utils.py

def tensor2cam(image, cam):
#  image_with_heatmap = image2cam(image.squeeze().permute(1,2,0).cpu().numpy(), cam.detach().cpu().numpy())
    image_with_heatmap = image2cam(image.unsqueeze(-1).cpu().numpy(), cam.detach().cpu().numpy())

Modify Model Layer:

from torchvision.models import efficientnet_b2
from torchvision.models import EfficientNet_B2_Weights
from torchvision.models._api import WeightsEnum
from torch.hub import load_state_dict_from_url
def get_state_dict(self, *args, **kwargs):
    kwargs.pop("check_hash")
    return load_state_dict_from_url(self.url, *args, **kwargs)
WeightsEnum.get_state_dict = get_state_dict

model = efficientnet_b2(weights=EfficientNet_B2_Weights.IMAGENET1K_V1)
model.features[0][0] = torch.nn.Conv2d(1, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
model.classifier[1] = torch.nn.Linear(in_features=1408, out_features=1, bias=True)

Implementation:

import torch
from visualisation.core import GradCam
from visualisation.core.utils import device

input_batch = torch.rand((1,1,256,256))
vis = GradCam(model, device)

outs = vis(
    input_image=input_batch.cuda(), 
    layer=None, 
    guide=False,
    target_class=None,
    regression=False)

I'm getting cam that are all zeros.

Hello, thanks for the great repo!

My problem is, when I use the googlenet or inception_v3 in Saliency Visualization, the unguided version works, but the guided saliency map seems not work at all when I added ‘guide=True ’ in the vis function, the results are just same with the unguided Saliency map. I checked the SaliencyMap.py but I have no idea about how to make the guide works for the above 2 networks.

saliency map for googlenet

guided saliency map for googlenet

What can I do to implement the guided saliency map with googlenet or inception_ v3 networks?

Thanks a lot.

hello thanks for wonderful project, i am facing an error while visualizing an image.

AttributeError: 'NoneType' object has no attribute 'data'

and that error indication line below in visualisation/core/SaliencyMap.py:

image = self.gradients.data.cpu().numpy()[0]

it is understandable because self.gradients initialized as None.

how can i fix it ?

'NoneType' object has no attribute 'register_forward_hook'