Can't use vgg16 for heatmap about convnets-keras HOT 9 CLOSED

Do you have the last version of Keras ? The last version of Theano ?

from convnets-keras.

yes,i use last version of Keras & Theano
pip install git
can you give me an example of vgg16+heatmap?

from convnets-keras.

Your code works fine for me.
Could you please print your versions of Theano and Keras by doing in your python interpreter : theano.__version__ and keras.__version__ ?

from convnets-keras.

theano
0.9.0dev0.dev-e3569d12ec7c247de6516025405619865c4393e1
keras
1.0.1

from convnets-keras.

Thank you.
Try to replace the loss by mse, I think it should work :

im = preprocess_image_batch(['examples/dog.jpg'],color_mode="bgr")

# Test pretrained model
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model = convnet('vgg_16', 'weights/vgg16_weights.h5', heatmap=True)
model.compile(optimizer=sgd, loss='mse')
out = model.predict(im)
heatmap = out[0,ids,:,:].sum(axis=0)


# Then, we can get the image
import matplotlib.pyplot as plt
plt.imsave("heatmap_dog.png",heatmap)

This is how it is used in convnets.py, but maybe this is coming from a recent commit you didn't have.

from convnets-keras.

when I use mse as above:

Using gpu device 0: GeForce GTX 960 (CNMeM is enabled with initial size: 30.0% of memory, cuDNN not available)
Traceback (most recent call last):
File "convnets.py", line 335, in
out = model.predict(im)
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 453, in predict
return self.model.predict(x, batch_size=batch_size, verbose=verbose)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1113, in predict
batch_size=batch_size, verbose=verbose)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 833, in _predict_loop
batch_outs = f(ins_batch)
File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 499, in call
return self.function(*inputs)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 902, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 889, in call
self.fn() if output_subset is None else
RuntimeError: BaseGpuCorrMM: Failed to allocate output of 1 x 64 x 1600 x 2560
Apply node that caused the error: GpuCorrMM{valid, (1, 1)}(GpuContiguous.0, GpuContiguous.0)
Toposort index: 85
Inputs types: [CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D)]
Inputs shapes: [(1, 3, 1602, 2562), (64, 3, 3, 3)]
Inputs strides: [(0, 4104324, 2562, 1), (27, 9, 3, 1)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[GpuElemwise{Composite{(i0 * ((i1 + i2) + Abs((i1 + i2))))}}[(0, 1)](CudaNdarrayConstant{[[[[ 0.5]]]]}, GpuCorrMM{valid, %281, 1%29}.0, GpuReshape{4}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

from convnets-keras.

I wonder if this could be a problem of size of memory, since you have this error :

 RuntimeError: BaseGpuCorrMM: Failed to allocate output of 1 x 64 x 1600 x 2560

How much memory does your GPU have ? I see that you are controlling the memory used by using CNMeM such that it use less than 30% of your memory. But the VGG16 network is very heavy.

You could try :

1. To do the same thing but with AlexNet, which is less heavy. To do this, just replace

model = convnet('vgg_16', 'weights/vgg16_weights.h5', heatmap=True)

by

model = convnet('alexnet', 'weights/alexnet_weights.h5', heatmap=True)

2. To disable CNMeM (or change 30% to 90%)
3. To try the same code, but on your CPU. It will be slower, but you won't have a memory issue. To do this, just add at the beginning of your file :

import theano
theano.config.device = "cpu"

from convnets-keras.

thank you .
it's a memory error,my gpu has 4G memory,can't run vgg16 for dog map size
cpu is too slow
is there any way use gpu on vgg16 & big picture?
now,we deal the picture as one tensor,can we split it as many batchs?

from convnets-keras.

As mentioned by @leonardblier in the issue #1.
" Let's take as an example the AlexNet, which take an input of size (227,227), and the dog picture, which is of size 2560x1600. Then we compute the output of the network for each sub-frame of size (227,227) of the entire picture. This produces for each one a score (for each label), and this is our heatmap, of size (2560 - 227, 1600 - 227) = (2333, 1373).
But just computing the output for each sub-frame would be computionally heavy. So what we do instead is that we convert the fully-connected layers into convolutional layers, only by reshaping the weights matrix, so this can be computed quite fast."

If you still want to use your GPU on VGG16 and on a big image, you could make mini-batches containing all sub-frames, make a forward pass on each sub-frame and then aggregate the results to reconstruct the heatmap. It will be very inefficient (and will not make use of the trick to convert dense layers to convolutions) but might be faster than running on CPU only.

from convnets-keras.