albertomontesg / keras-model-zoo Goto Github PK

Hi @albertomontesg,
What are the preprocessing steps? Like mean subtraction and normalization?

Hi,

I'm trying to run the c3d model from the keras model zoo, but it doesn't even initialize. When I do:

 from kerasmodelzoo.models import c3d
 M = c3d.model(weights=True, summary=True)

The model crashed in line 33 of the c3d module when it tries to add a MaxPooling3D layer. Here is the tail of the backtrace:

Using TensorFlow backend.
Traceback (most recent call last):
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py", line 594, in call_cpp_shape_fn
    status)
  File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/errors.py", line 463, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.InvalidArgumentError: Negative dimension size caused by subtracting 2 from 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "load_c3d.py", line 9, in <module>
    model = c3d.model(weights=True, summary=True)
  File "/home/ingo/code/keras-model-zoo/kerasmodelzoo/models/c3d.py", line 34, in model
    border_mode='valid', name='pool3'))
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/models.py", line 324, in add
    output_tensor = layer(self.outputs[0])
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/engine/topology.py", line 517, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/engine/topology.py", line 571, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/engine/topology.py", line 155, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/layers/pooling.py", line 305, in call
    dim_ordering=self.dim_ordering)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/layers/pooling.py", line 353, in _pooling_function
    border_mode, dim_ordering, pool_mode='max')
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1847, in pool3d
    x = tf.nn.max_pool3d(x, pool_size, strides, padding=padding)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1467, in max_pool3d
    strides=strides, padding=padding, name=name)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op
    op_def=op_def)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2382, in create_op
    set_shapes_for_outputs(ret)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1783, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py", line 596, in call_cpp_shape_fn
    raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1

The error message is pretty clear but it really happens in the depth of the keras backend and it's not quite obvious which statement actually resulted in this dimension mismatch. I went to the place where the tensorflow max-pooling operation is actually created by the keras backend and looked at the dimensions. Pooling is actually applied with a pool size of (1, 2, 2, 2 , 1) and strides of (1, 2, 2, 2, 1) to a relu-tensor of shape (?, 1, 4, 28, 256). The error probably results from the second dimensions.

I suspected that this might be due to differences in dimension ordering between tensorflow and theano and tried with theano as well. Indeed it gets further but then crashes in line 56 when the Flatten() operation is added. Given that I am not really interested in getting it to run with the theano backend, I didn't investigate any further.

Did anybody have success running the c3d model with tensorflow? Am I missing something important? Any help would be greatly appreciated.

Here can be found the structure.
It would be necessary to find the weights.

Although the code for the models and the weights files will still be compatible with Keras 1.0 (release next week), the JSON files will not. I think you should remove the JSON files to avoid compatibility problems. It is always safer to instantiate a model from its definition code than from a JSON file.

The input for the c3d model is of dimensions 3x16x112x112 (channels x timesteps x width x height), however the c3d_means file has a numpy array of size (3, 16, 128, 171).

I was wondering if this means the video has to be resized ? What should be done if I am using C3D as a feature extractor on a different dataset ?

VGG-M is a very small network but gives reasonable performance. It is helpful in the tasks where we have small dataset of our own and we want extract features from a network which is trained on ImageNet. Thus fine tuning on a small architecture helps in avoiding overfitting.

Hi @albertomontesg thanks for the great work. If i want to use Tensorflow to do video classification task with C3D model. is there an easy way to load the Sports-1M pre-trained model? Thanks.

Create test for the code and add CI and coverage.

Hi @albertomontesg , I checked online about fine tuning the last layer in Keras. and it seems that using model.layers.pop() first, and then add the my desired output layer is a simple way.

However,
i tried this,
.................................................
model.add(Dense(487, activation='softmax', name='fc8'))
model.layers.pop()
model.add(Dense(200, activation='softmax', name='fc8'))
model.compile(loss='sparse_categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])

But the error came out: Exception: The name "fc8" is used 2 times in the model. All layer names should be unique.

But I print model.summary() after i pop the last layer, the fc-8 layer indeed poped out already. Why the name is still used 2 times in the model.

thanks a lot for any suggestions.

How exactly did you convert the VGG model to Keras model? I am trying to convert the VGG-M model using the code by @MarcBS. But due to version problem, I am not able to do it. Is there any better way to it?
Did you use some different script? Did you also upgrade the .caffemodel model file before converting into keras model?

Hi @albertomontesg, I have the same problem like Pool 2D optimization #4377, which can't train C3D with large batch size. Thanks for your suggestions going with the theano module theano.sandbox.cuda.nnet.

However, I directly search theano.sandbox.cuda.nnet on google, but no useful documentation shows up. I don't which one are your refer to, theano.sandbox.cuda.fftconv.conv3d_fft? or theano.sandbox.cuda.dnn.GpuDnnConv3d, or others? Sorry that I am not familiar with Theano.

Could you advise through(like give me an example about 3d conv, 3d pool), or update the model.py using theano.sandbox.cuda.nnet? Thank you very much.

Convert this repository into a Python package where it would be possible to get a keras model on the following way:

from kerasmodelzoo.VGG16.model import VGG, VGG_weights, VGG_mean

model = VGG()
weights = VGG_weights()
mean = VGG_mean()

Thanks for your work firstly,

I'm now training my own C3D model used UCF-101 datasets, and I also fine tune your pre-trained model. But I can't get a satisfied result.

it would be appreciated if you could provide some training details? optimizers /loss function, learning rate, data preprocessing,etc.

Thanks a lot.