Comments (6)
For anyone who is still interested in obtaining the explained heatmap for Mask R-CNN models or YOLO-like models, we have given a tutorial showing the visualization results. Hope this can help.
from interpretdl.
Hi, @jessecanada
We have done something similar for a YOLO model of PaddleDec. Let me find the code and see if we can do the visualization directly for a Mask R-CNN model.
I'll get back to you on Monday or Tuesday ;)
from interpretdl.
Hi, @jessecanada
We've tried Grad-CAM on a Mask R-CNN model based on PaddleDetection. We are able to visualize/interpret a bounding box prediction and its confidence with respect to the ROI. One of the visualizations looks like this:
We are still trying to figure out how to incorporate object detection tasks into InterpretDL, but here is how we implemented it based on PaddleDetection:
- Modify the get_prediction function so that it also outputs cls_prob and bbox_pred.
- In mask RCNN code, outputs roi_feat in single_scale_eval, and in build, calculate and output the gradients of cls_prob or bbox_pred with respect to roi_feat.
- In tools/infer.py, comment out test mode program so that gradients can be calculated without error, and then save gradients and roi_feats for visualization.
- Run tools/infer.py by specifying architecture, weights and image.
If you have any further questions, please let us know!
from interpretdl.
Hi @XuanyuWu123
I'm searching how to visualize heatmaps on Mask R-CNN.
Could you teach me how to implement Grad-CAM on a Mask R-CNN?
thanks
from interpretdl.
Hi @nomurakeiya @XuanyuWu123
Do you figure out how to implement grad-cam on Mask R-CNN?
from interpretdl.
Hello all,
Thanks for the interests in our repo.
For the implementation of Grad-CAM on Mask R-CNN, there are several points need to be clarified:
- There are three outputs of a Mask R-CNN: the bounding box coordinates, the cls prediction of this box, and the mask.
- It is easy to get the explanation on the cls prediction of a certain box, that should be the heatmap computed by Grad-CAM or other algorithms. However, explanations on the bounding box coordinates or the mask, are not well defined. Please tell us if you desired other explanation results.
- For the heatmap, one problem is that in eval mode, a NMS bbox_post_process is done for Mask R-CNN, which stops the computation of gradients. So that the final outputs can not be explained directly.
- But we can still compute the gradients of raw outputs (where bbox_head outputs 1000 boxes). This is possible for computing Grad-CAM from here.
- For PaddleDetection, we am still thinking about how to explain the model directly with our tool. At this moment, a possible way is to modify the source code of PaddleDetection, gets the feature map and gradients of a certain layer, and then compute Grad-CAM.
Let us know if there are more questions.
Cheers
from interpretdl.
Related Issues (20)
- 运行example_grad_cam_cv.ipynb时报错 HOT 3
- 热力图 HOT 7
- 关于分割模型的可视化 HOT 5
- Infidelity metric not found, probably a new release needed HOT 1
- Interface
- 长宽不相等的图像怎么输入 HOT 5
- Gradient computation changed since paddle2.4
- 分割可视化 HOT 25
- Shapely Value HOT 7
- Grad-CAM可视化分割模型 HOT 2
- GradcamSeg是否支持paddle2.4以上版本? HOT 1
- 请问这里边有没有和自然语言处理相关的 HOT 7
- softmax(): argument 'x' (position 0) must be Tensor, but got list HOT 3
- Tutorial error HOT 3
- Failed to apply gradcam on mobileNetv3-large HOT 6
- 在语义分割中应用GradCAM时,如何对整幅图像进行热力图显示,传入的pixels大小如何确定 HOT 2
- 怎么拿出paddledetection中间的某一层进行可视化 HOT 1
- wrong url HOT 2
- UFuncTypeError: Cannot cast ufunc 'add' output from dtype('O') to dtype('float64') with casting rule 'same_kind' HOT 6
- pip install interpretdl ERROR: Failed building wheel for scikit-learn HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from interpretdl.