pooya-mohammadi / yolov5-gradcam Goto Github PK

Visualizing Yolov5's layers using GradCam

License: MIT License

Python 10.32% Jupyter Notebook 89.68%

pytorch deeputils deep-learning convolutional-neural-networks visualizing gradcam yolov5

yolov5-gradcam's Introduction

YOLO-V5 GRADCAM

I constantly desired to know to which part of an object the object-detection models pay more attention. So I searched for it, but I didn't find any for Yolov5. Here is my implementation of Grad-cam for YOLO-v5. To load the model I used the yolov5's main codes, and for computing GradCam I used the codes from the gradcam_plus_plus-pytorch repository. Please follow my GitHub account and star ⭐ the project if this functionality benefits your research or projects.

Update:

Repo works fine with yolov5-v6.1

Installation

pip install -r requirements.txt

Infer

python main.py --model-path yolov5s.pt --img-path images/cat-dog.jpg --output-dir outputs

NOTE: If you don't have any weights and just want to test, don't change the model-path argument. The yolov5s model will be automatically downloaded thanks to the download function from yolov5.

NOTE: For more input arguments, check out the main.py or run the following command:

python main.py -h

Custom Name

To pass in your custom model you might want to pass in your custom names as well, which be done as below:

python main.py --model-path cutom-model-path.pt --img-path img-path.jpg --output-dir outputs --names obj1,obj2,obj3

Examples

Note

I checked the code, but I couldn't find an explanation for why the truck's heatmap does not show anything. Please inform me or create a pull request if you find the reason.

This problem is solved in version 6.1

TO Do

Add GradCam++
Add ScoreCam
Add the functionality to the deep_utils library

References

Citation

Please cite yolov5-gradcam if it helps your research. You can use the following BibTeX entry:

@misc{deep_utils,
	title = {yolov5-gradcam},
	author = {Mohammadi Kazaj, Pooya},
	howpublished = {\url{github.com/pooya-mohammadi/yolov5-gradcam}},
	year = {2021}
}

yolov5-gradcam's People

Contributors

Stargazers

Watchers

yolov5-gradcam's Issues

No matter how many bbox,

No matter how many bbox, the thermal map should be the same. Why is the thermal map of the whole image different for different objects?

RuntimeError: don't know how to restore data location of torch.HalfStorage (tagged with )

D:\anaconda\envs\yolov5\lib\site-packages\deep_utils\vision\face_detection\ultralight_init_.py:3: UserWarning:
No module named 'deep_utils.vision.face_detection.ultralight.torch'. If you don't use UltralightTorchFaceDetector ignore this message.
UltralightTorchFaceDetector = import_module(
D:\anaconda\envs\yolov5\lib\site-packages\deep_utils\vision\face_detection\ultralight_init_.py:6: UserWarning:
No module named 'deep_utils.vision.face_detection.ultralight.tf'. If you don't use UltralightTFFaceDetector ignore this message.
UltralightTFFaceDetector = import_module(
D:\anaconda\envs\yolov5\lib\site-packages\deep_utils\callbacks\tf_keras_init_.py:3: UserWarning:
No module named 'tensorflow'. If you don't use LRScalar ignore this message.
LRScalar = import_module(
[INFO] Loading the model
Traceback (most recent call last):
File "main.py", line 125, in
main(args.img_path)
File "main.py", line 64, in main
model = YOLOV5TorchObjectDetector(args.model_path, device, img_size=input_size,
File "D:\AAAAAAAAA\YOLOV5-4.0\yolov5-gradcam-master\models\yolo_v5_object_detector.py", line 32, in init
self.model = attempt_load(model_weight, device=device)
File "D:\AAAAAAAAA\YOLOV5-4.0\yolov5-gradcam-master\models\experimental.py", line 80, in attempt_load
ckpt = torch.load(attempt_download(w), map_location=device)
File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 594, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 853, in _load
result = unpickler.load()
File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 845, in persistent_load
load_tensor(data_type, size, key, _maybe_decode_ascii(location))
File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 834, in load_tensor
loaded_storages[key] = restore_location(storage, location)
File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 811, in restore_location
return default_restore_location(storage, map_location)
File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 178, in default_restore_location
raise RuntimeError("don't know how to restore data location of "
RuntimeError: don't know how to restore data location of torch.HalfStorage (tagged with )

self.model(img, augment=False) does not work

Hi, Thanks for the repo. Whenever I am running it, I am getting an error on the self.model command. It seems it does not output pred and logits but a single tensor when given a tensor. Not sure how this line could be updated.

Traceback (most recent call last):
File "C:\Users\kusari\Documents\Camera_error_sources\yolov5-gradcam\main.py", line 124, in
main(args.img_path)
File "C:\Users\kusari\Documents\Camera_error_sources\yolov5-gradcam\main.py", line 67, in main
saliency_method = YOLOV5GradCAM(model=model, layer_name=args.target_layer, img_size=input_size)
File "C:\Users\kusari\Documents\Camera_error_sources\yolov5-gradcam\models\gradcam.py", line 44, in init
self.model(torch.zeros(1, 3, *img_size, device=device))
File "C:\Users\kusari\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\kusari\Documents\Camera_error_sources\yolov5-gradcam\models\yolo_v5_object_detector.py", line 167, in forward
prediction, logits, _ = self.model(img, augment=False)
ValueError: not enough values to unpack (expected 3, got 2)

ValueError: not enough values to unpack (expected 3, got 2)

Traceback (most recent call last):
File "D:\jupyter\pythonProject\yolov5-gradcam-master\main.py", line 124, in
main(args.img_path)
File "D:\jupyter\pythonProject\yolov5-gradcam-master\main.py", line 67, in main
saliency_method = YOLOV5GradCAM(model=model, layer_name=args.target_layer, img_size=input_size)
File "D:\jupyter\pythonProject\yolov5-gradcam-master\models\gradcam.py", line 44, in init
self.model(torch.zeros(1, 3, *img_size, device=device))
File "C:\ProgramData\Anaconda3\envs\CAM\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "D:\jupyter\pythonProject\yolov5-gradcam-master\models\yolo_v5_object_detector.py", line 167, in forward
prediction, logits, _ = self.model(img, augment=False)
ValueError: not enough values to unpack (expected 3, got 2)

Tensor not match

hello, I trianed it on DOTA datasets,but get error.
RuntimeError: The size of tensor a (80) must match the size of tensor b (1280) at non-singleton dimension 1

Can't get attribute 'DetectionModel' on <module 'models.yolo' from....>

Here's my code...↓

! pip install -r requirements.txt

import os
os.chdir("./yolov5-gradcam-master/")

!python main.py --model-path runs/train/hat_person4/weights/best.pt --img-path datasets/hat_person/yolo_dataset/images/train/000490.jpg --output-dir outputs --names hat,person

And then...Something wrong..

[INFO] Loading the model
Traceback (most recent call last):
File "main.py", line 124, in
main(args.img_path)
File "main.py", line 64, in main
names=None if args.names is None else args.names.strip().split(","))
File "f:\works\VSCODE\yolov5\yolov5-gradcam-master\models\yolo_v5_object_detector.py", line 32, in __init__
self.model = attempt_load(model_weight, device=device)
File "f:\works\VSCODE\yolov5\yolov5-gradcam-master\models\experimental.py", line 80, in attempt_load
ckpt = torch.load(attempt_download(w), map_location=device)
File "d:\Anaconda3\envs\yolo\lib\site-packages\torch\serialization.py", line 607, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "d:\Anaconda3\envs\yolo\lib\site-packages\torch\serialization.py", line 882, in _load
result = unpickler.load()
File "d:\Anaconda3\envs\yolo\lib\site-packages\torch\serialization.py", line 875, in find_class
return super().find_class(mod_name, name)
AttributeError: Can't get attribute 'DetectionModel' on <module 'models.yolo' from 'f:\works\VSCODE\yolov5\yolov5-gradcam-master\models\yolo.py'>

请问如何实现整个图片的热力图可视化？不只是检测框部分可视化。

When I cycled through multiple images, and for each image, I cycled through multiple layers of the model, I saw an abnormal increase in memory.

When I cycled through multiple images, and for each image, I cycled through multiple layers of the model, I saw an abnormal increase in memory.
I just increased the outer loop, and I found that every time I run self.model(images), it causes it to grow

RuntimeError: The size of tensor a (80) must match the size of tensor b (1280) at non-singleton dimension 1

Hi all,

When I run main main.py with my yolov5X model pretrained with ultralytics v5_6.1 on custom images I got this error message.

saliency_map = (weights * activations).sum(1, keepdim=True)
RuntimeError: The size of tensor a (80) must match the size of tensor b (1280) at non-singleton dimension 1

Is there any solution for this?

Best regards.

Can the code be used for target detection? thank you very much

No heatmap using a different model

Hi, thank you for your great job!

I tried to use a custom model, and even though the object is detected, the heatmap is basically empty.

Do you know a workaround?

Thanks agin.

Get ERROR when using main.py

`D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\deep_utils\vision\face_detection\ultralight_init_.py:5: UserWarning:
No module named 'deep_utils.vision.face_detection.ultralight.torch'. If you don't use UltralightTorchFaceDetector ignore this message.
'UltralightTorchFaceDetector')
D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\deep_utils\vision\face_detection\ultralight_init_.py:8: UserWarning:
No module named 'deep_utils.vision.face_detection.ultralight.tf'. If you don't use UltralightTFFaceDetector ignore this message.
'UltralightTFFaceDetector')
2022-05-19 14:08:34.287874: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2022-05-19 14:08:34.288041: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
['bus.jpg', 'cat-dog.jpg', 'dog.jpg', 'eagle.jpg']
[INFO] Loading the model
[INFO] Model is loaded
[INFO] fetching names from coco file
D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\torch\functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ..\aten\src\ATen\native\TensorShape.cpp:2157.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\torch\nn\modules\module.py:1025: UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.
warnings.warn("Using a non-full backward hook when the forward contains multiple autograd Nodes "
[INFO] saliency_map size : torch.Size([20, 20])
D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\torch\nn\modules\module.py:1025: UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.
warnings.warn("Using a non-full backward hook when the forward contains multiple autograd Nodes "
[INFO] model-forward took: 0.1995 seconds
D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\torch\nn\functional.py:3509: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
[INFO] person, model-backward took: 0.2481 seconds
[INFO] person, model-backward took: 0.2605 seconds
[INFO] person, model-backward took: 0.2442 seconds
[INFO] bus, model-backward took: 0.2368 seconds
[INFO] person, model-backward took: 0.2494 seconds
total time: 1.4811
Traceback (most recent call last):
File "D:/deep-learning/yolov5-gradcam-master/main.py", line 94, in
main(os.path.join( args.img_path ,item))
File "D:/deep-learning/yolov5-gradcam-master/main.py", line 78, in main
res_img, heat_map = get_res_img(bbox, mask, res_img)
File "D:/deep-learning/yolov5-gradcam-master/main.py", line 32, in get_res_img
n_heatmat = (Box.fill_outer_box(heatmap, bbox) / 255).astype(np.float32)
File "D:\python3.8.6\anaconda\envs\videocv\lib\site-packages\deep_utils\utils\box_utils\boxes.py", line 430, in fill_outer_box
img = cv2.multiply(img, mask)
NameError: name 'cv2' is not defined

Process finished with exit code 1
`

it did not work multiple object of same class.how can I use it for multiple object in the image (same class i.e person )

Can't check recent custom best.pt model for the compatibility issues of pytorch

Will you please update the repository for the up-to-date torch version?
[colab notebook,](https://colab.research.google.com/github/pooya-mohammadi/yolov5-gradcam/blob/master/main.ipynb?

IndexError: list index out of range

why register_backward_hook() got the zero gradient

I took the gradcam for yolov3 and revise a little.
I have no idea about why register_backward_hook() couldn't get the gradient. The register_forward_hook() is ok.
can you help me, thanks a lot.

The output image is darker than the input image

When I try to specify the img-size to 1280, the output image is darker than the input image.

Label problem

I trained the model on my own dataset, its category is not the same as the coco dataset, but the final display result seems to be the label of the coco dataset?

what about yolov5l,yolov5x,yolov5s?

hi,i tried to use the yolov5l,yolov5x,yolov5s models,but the results of heatmap were not good,although i tried other targets-layer.
please tell me how this work support yolov5l,yolov5x,yolov5s？thanks

Error when trying with custom model and dataset

Hi there,

I'm currently have issues when running the code with the model I trained on yolov5 v6.1, with a custom dataset

Do you know what's happening?

Thank you so much!!

KeyError: 'cv3'

My training weights were trained by adding the attention mechanism to YOLOv5's network,
and there was an error when I used the code to export the thermal map：

What is the problem and how to solve it，Thank you very much

get error "google/protobuf/pyext/descriptor.cc:358: bad argument to internal function"

Dears

When I try to operate "python main.py --model-path yolov5l.pt --img-path images/cat-dog.jpg --output-dir outputs"
and I get this error

Traceback (most recent call last):
File "main.py", line 6, in
from models.yolo_v5_object_detector import YOLOV5TorchObjectDetector
File "/content/yolov5-gradcam/models/yolo_v5_object_detector.py", line 2, in
from deep_utils.utils.box_utils.boxes import Box
File "/usr/local/lib/python3.7/dist-packages/deep_utils/init.py", line 10, in
from .vision import *
File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/init.py", line 1, in
from .face_detection import *
File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/face_detection/init.py", line 49, in
from deep_utils.vision.face_detection.ultralight.tf.ultralight_tf_face_detection import (
File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/face_detection/ultralight/tf/ultralight_tf_face_detection.py", line 17, in
from .utils.rfb_320 import create_rfb_net
File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/face_detection/ultralight/tf/utils/rfb_320.py", line 1, in
import tensorflow as tf
File "/usr/local/lib/python3.7/dist-packages/tensorflow/init.py", line 37, in
from tensorflow.python.tools import module_util as _module_util
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/init.py", line 37, in
from tensorflow.python.eager import context
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/context.py", line 29, in
from tensorflow.core.framework import function_pb2
File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/function_pb2.py", line 16, in
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/resource_handle_pb2.py", line 150, in
'module' : 'tensorflow.core.framework.resource_handle_pb2'
SystemError: google/protobuf/pyext/descriptor.cc:358: bad argument to internal function

Could you tell me how to solve this error?

Get RuntimeError when using `register_full_backward_hook`

hey, I encountered an error when running the register_backward_hook
firstly, its just a warning:
UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.
it's deprecated after 1.8.0 so I replace the register_backward_hook with register_full_backward_hook as told, but this time I get an error:
RuntimeError: Output 0 of BackwardHookFunctionBackward is a view and is being modified inplace. This view was created inside a custom Function (or because an input was returned as-is) and the autograd logic to handle view+inplace would override the custom backward associated with the custom Function, leading to incorrect gradients. This behavior is forbidden. You can fix this by cloning the output of the custom Function. I googled for a long time and still didnt get a good resolution.
Do you have any idea?

and I am using:
PyTorch: 1.10.0
torchvision 0.11.2
Python 3.9.7
OS: ubuntu1804