salman-h-khan / zsd_release Goto Github PK

When i run python detect.py on your code it is showing a type error.

Using TensorFlow backend.
/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Traceback (most recent call last):
File "detect.py", line 123, in
shared_layers = nn.nn_base(img_input, trainable=True)
File "/home/dheeraj/dheeraj/cvpro1/ZSD_Release/keras_frcnn/resnet.py", line 180, in nn_base
x = FixedBatchNormalization(axis=bn_axis, name='bn_conv1')(x)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/keras/engine/base_layer.py", line 463, in call
self.build(unpack_singleton(input_shapes))
File "/home/dheeraj/dheeraj/cvpro1/ZSD_Release/keras_frcnn/FixedBatchNormalization.py", line 30, in build
trainable=False)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/keras/engine/base_layer.py", line 279, in add_weight
weight = K.variable(initializer(shape, dtype=dtype),
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/keras/initializers.py", line 46, in call
return K.constant(1, shape=shape, dtype=dtype)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 649, in constant
value, dtype=dtype, shape=shape, name=name)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 783, in constant
return constant_op.constant(value, dtype=dtype, shape=shape, name=name)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 245, in constant
allow_broadcast=True)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py", line 283, in _constant_impl
allow_broadcast=allow_broadcast))
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/tensor_util.py", line 463, in make_tensor_proto
if shape is not None and np.prod(shape, dtype=np.int64) == 0:
File "<array_function internals>", line 6, in prod
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2962, in prod
keepdims=keepdims, initial=initial, where=where)
File "/home/dheeraj/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 90, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

How can i rectify this error?

Really cool work on zero-shot detection. Run a few experiments on the toy examples. I think the w2v choice for each cluster/class is also crucial. Maybe a dynamic representation of the cluster is more elegant. Any plan to release the training code? :)

When I know the word, I can get it's word embedding vector, but when I know it's vector, I have no idea how to get it's words. W2 is word embedding vector.T or not?
I can't find in the reference code, where calculate the cosine similiarity as the paper.

when I run detect.py got this error
ValueError: Shape must be rank 1 but is rank 0 for 'bn_conv1/Reshape_4' (op: 'Reshape') with input shapes: [1,1,1,64], [].
will you kindly suggest me how i can fix it?

I want to run it on MS coco dataset but for that, I will need to make my own config.pickle file. Can you share how do you created your config file?

When I do:
python detect.py Dataset/Sampleinput/input.png,

I get the following error:

Traceback (most recent call last):
File "detect.py", line 123, in
shared_layers = nn.nn_base(img_input, trainable=True)
File "/data2/charades2/ZSD_Release/keras_frcnn/resnet.py", line 182, in nn_base
x = FixedBatchNormalization(axis=bn_axis, name='bn_conv1')(x)
File "/home/user/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 75, in symbolic_fn_wrapper
return func(*args, **kwargs)
File "/home/user/anaconda3/lib/python3.7/site-packages/keras/engine/base_layer.py", line 463, in call
self.build(unpack_singleton(input_shapes))
File "/data2/charades2/ZSD_Release/keras_frcnn/FixedBatchNormalization.py", line 31, in build
trainable=False)
File "/home/user/anaconda3/lib/python3.7/site-packages/keras/engine/base_layer.py", line 279, in add_weight
weight = K.variable(initializer(shape, dtype=dtype),
File "/home/user/anaconda3/lib/python3.7/site-packages/keras/initializers.py", line 46, in call
return K.constant(1, shape=shape, dtype=dtype)
File "/home/user/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 649, in constant
value, dtype=dtype, shape=shape, name=name)
File "/home/user/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/backend.py", line 937, in constant
return constant_op.constant(value, dtype=dtype, shape=shape, name=name)
File "/home/user/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/constant_op.py", line 258, in constant
allow_broadcast=True)
File "/home/user/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/constant_op.py", line 287, in _constant_impl
return _eager_fill(shape.as_list(), t, ctx)
File "/home/user/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/constant_op.py", line 52, in _eager_fill
dims = convert_to_eager_tensor(dims, ctx, dtypes.int32)
File "/home/user/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/constant_op.py", line 96, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.

Could you resolve this issue? I checked the shape is (None ,). That might be causing the problem. My question is why shape is (None ,)?

Your work is very good! Would you like to share the training codes. I want to test it on my dataset.

Upon running detect.py using the pretrained model available and the following named layers

resnetlast = (model_classifier.layers[-1].output)
out_class = TimeDistributed(Dense(int(resnetlast.shape[2]), kernel_initializer='uniform'),name='time_distributed_2')(resnetlast)
out_class = Activation('relu')(out_class)
out_class = Dropout(.5)(out_class)
out_class = TimeDistributed(Dense(word.shape[0], activation='linear', kernel_initializer='uniform'),name='time_distributed_3')(out_class)
out_class = TimeDistributed(Dense(word.shape[0], activation='linear', kernel_initializer='uniform'),name='dense_class_{}'.format(nb_classes))(out_class)
out_class = MyLayer(output_dim=word_all.shape[1])(out_class)
out_regr = TimeDistributed(Dense(4 * (nb_classes - 1), activation='linear', kernel_initializer='zero'),
                           name='dense_regressor_{}'.format(nb_classes))(resnetlast)

I get an error which is

ValueError: Layer #178 (named "time_distributed_2"), weight <tf.Variable 'time_distributed_2/kernel:0' shape=(708, 708) dtype=float32, numpy=
array([[ 0.03911747, -0.02416103, -0.04996177, ..., -0.00052779,
-0.04415029, 0.03913448],
[ 0.02350913, -0.04238098, 0.04727681, ..., 0.00107334,
0.03948401, 0.0225733 ],
[ 0.00324714, 0.0267023 , 0.03767557, ..., -0.03660261,
0.02911811, -0.04166988],
...,
[ 0.00710471, -0.02927768, 0.02644283, ..., 0.0140437 ,
0.02945073, -0.01651412],
[ 0.03027074, -0.03358974, 0.03127309, ..., -0.00256994,
-0.00981454, -0.00117335],
[ 0.00760389, 0.02819859, -0.02207704, ..., -0.01791253,
-0.04420076, -0.00320693]], dtype=float32)> has shape (708, 708), but the saved weight has shape (2048, 2048).

I tried to check the weights of the pretained model to verify shape of the layers using keras-team/keras#91 but unfortunately the part about param.shape doesn't work and replacing it with param doesn't give you the shape either.

My library versions are

• Keras 2.4.3
• OpenCV 4.5.2
• Tensorflow 2.3.2

A few directions in which I would try to solve this are by:

• Downgrading to the versions on which the code is tested
• Checking if model_classifier.layers.pop() is working as expected by seeing the model summary
• Try to check the naming again with the pretrained model because I strongly suspect that this shape inconsistency is because of the naming mismatch

Hi，When I run detect.py ,some errors occurs, ModuleNotFoundError: No module named 'keras_frcnn.config\r' , I use python=3.6 the package pickle is a standard part ，can't upgraded，SO how to generate the pick file config.pickle myself，thx

The name of the time_distributed layers would be set as a default way:
time_distributed; time_distributed_1; time_distributed_2
This would cause error due to the names of these layers in hdf5 file (time_distributed_1; time_distributed_2; time_distributed_3).
I suggest to edit the detect.py as follows:
resnetlast = (model_classifier.layers[-1].output)
out_class = TimeDistributed(Dense(int(resnetlast.shape[2]), kernel_initializer='uniform'),name='time_distributed_2')(resnetlast)
out_class = Activation('relu')(out_class)
out_class = Dropout(.5)(out_class)
out_class = TimeDistributed(Dense(word.shape[0], activation='linear', kernel_initializer='uniform'),name='time_distributed_3')(out_class)