saliency for reconstruction task about saliency HOT 3 CLOSED

Hi Chen,
Thanks for reaching out. Do you mean tasks such as image infilling or super-resolution when you say reconstruction? I haven't looked into how saliency looks like for reconstruction models, but one idea is seeing each pixel as a separate class which would give you a heatmap of where the model is looking at when predicting that particular pixel. You can maybe sample a subset of pixels in the image on a grid and visualize heatmaps at each location separately.

from saliency.

Hi Chen,
Thanks for reaching out. Do you mean tasks such as image infilling or super-resolution when you say reconstruction? I haven't looked into how saliency looks like for reconstruction models, but one idea is seeing each pixel as a separate class which would give you a heatmap of where the model is looking at when predicting that particular pixel. You can maybe sample a subset of pixels in the image on a grid and visualize heatmaps at each location separately.

Thanks for your inspiring reply! for reconstruction I mean some auto-encoder/GAN based image to image reconstruction. Your idea to use each pixel as class is worth trying, but might still have some problem: we may treat them as continuos value instead of discrete value. So the problem became: could we change the code for regression task visualization?

from saliency.

I think it should work out of the box as long as you pass the regression neuron as the output neuron for the saliency. Please let me know if you run into any issues!

from saliency.