salman-h-khan / pl-zsd_release Goto Github PK

Hi @salman-h-khan
It would we great if you could help me out , while running gzsd.py file, ealier I got an Import error for "keras-retinanet ", after which I tried by installing using "pip install keras-retinanet==0.5.1", this problem resolved but a new error is coming shown below:
File "gzsd.py", line 6, in
from keras_retinanet.models import retinanet_vocab_w2v as retinanet # retinanet_vocab_glo
ImportError: cannot import name 'retinanet_vocab_w2v'
I am not getting solution for this, please help me out,also does the keras-retinanet version same or different ( I had setup the environment as stated).
Thanks,
Sushil Kumar

Can you share the training code？I want to train the model on my own dataset.

Hi, I am super interested in your paper and have some questions when using your training code. Hoping to get your reply.
1.If I want �to use a new split of coco or new dataset for training, is it the pretrain with focal loss a necessary process?
2.The pre-trained model you provide('Model/resnet50_csv_50_focal_seen_w2v.h5') ,is it a zero-shot detection model?(ours-FL-word in your paper?) or it is just FL-basic.
3.The default parameter for "step_per_epoch" in your code is 10000. Is it set intentionally？Should I set the steps equal to the len of dataset？
4.How many epochs have you trained to get your result？（100K iteration in your paper，but the len of your training datasets is 62300, 100K is only about 2 epochs）

Hi, thanks for sharing this software implementation and congratulations on this nice work. Can you give some guidelines on how to apply ZSD to a custom dataset, with different classes than MSCOCO? For example, i have object level GT for apples and oranges, but want to detect lemons. What would be a good way to initialize the training weights? Thanks in advance.

Hi, I've completed training model by following the running instructions in Readme, but in my own py36 environment. When I ran the testing code, an error occoured:

Traceback (most recent call last):
IndexError: list index out of range

Could you please tell me how to solve this issue? Thank you!

Hi @salman-h-khan
Hope, you are doing well,
I just wanted to know had you run this project on any docker-container, or created docker image for this project .In case yes, can you please provide the docker image to run this project on docker. Actually I want to run this project once on gpu server using kurbernete and for than I am not able to get docker image, which could run this project.
Thanks,
Sushil Kumar

Hi i have some question about the source of the vocabulary_w2v.txt and word_w2v.txt.
Are they come from the same source?
If the answer was yes are they come from GoogleNews-vectors-negative300.bin.gz?

Have you tried applying the politary loss function to the traditional classification problem instead of the zero shot object classification problem? I would like to know whether the effect will be better or not.

When I do:
python gzsd.py

I get the following error:

Traceback (most recent call last):
File "gzsd.py", line 59, in
model = retinanet.retinanet_bbox(inputs=inputs, num_classes=num_seen, backbone=resnet)
File "/data2/charades2/PL-ZSD_Release/keras_retinanet/models/retinanet_vocab_w2v.py", line 350, in retinanet_bbox
model = retinanet(inputs=inputs, num_classes=num_classes, **kwargs)
File "/data2/charades2/PL-ZSD_Release/keras_retinanet/models/retinanet_vocab_w2v.py", line 307, in retinanet
_, C3, C4, C5 = backbone.outputs # we ignore C2
ValueError: not enough values to unpack (expected 4, got 1)

Could you kindly resolve this issue?

Excuse me, how do I modify the cls_name of the test and predict objects with other labels that are different from the test labels used in this experiment?

Hi,
Thank you for providing the awesome repository!
I tried to change the dataset from COCO to Pascal VOC 2007; however, the link provided in "Notes on Pascal VOC experiment" section only contains attribute file.

Could you share the method to generate the vocabulary file for Pascal VOC 2007?
Thank you

Hi,I try to run the train code on my computer with gtx 1080ti, and an error occoured:
/usr/local/lib/python3.6/dist-packages/keras/callbacks.py:1065: UserWarning: epsilon argument is deprecated and will be removed, use min_delta instead.
warnings.warn('epsilon argument is deprecated and '
Epoch 1/30
2019-10-28 10:45:40.107816: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:666] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-28 10:45:40.151326: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:666] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-28 10:45:40.568029: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:666] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-28 10:45:40.597243: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:666] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
I try to replace the versions of cuda cudnn and tensorflow, and it doesn't work.
Could you please tell me how to solve this issue? Thank you!

Thank you for your work,i find the codes mainly use cpu,how should i do if i want to use gpu?

I used my own glove word2vec embedding data for my own dataset and vocabulary_glo.txt you provided. I trained 1 epoch with 10000 iterations. The loss is just about 5. But when I use gzsd.py to predict and I found all the scores are zero.

Hi! Can you show me the loss curve?
Or how many steps is better to train ?

Hi, I'm interested in your paper and try to reproduce your work by Pytorch, but I meet the problem that in your paper the unseen objects' probability is calculated by W_u, W_s and seen objects' probability($P_u = W_uW_s^Tσ(δ(W_sMD)f)$, equation 9)，how to keep the probability in the range of [0, 1]? In my work, many unseen objects probabilities are large and some are even larger than 1. Though I still try to solve this problem, I hope to get your some advice and help. Thanks！

Hi, may I know how u generate vectors for classes with two words (e.g. 'traffic light' in COCO)?

Traceback (most recent call last):
File "keras_retinanet/bin/train_vocab_w2v.py", line 407, in
main()
File "keras_retinanet/bin/train_vocab_w2v.py", line 361, in main
training_model.load_weights('Model/resnet50_csv_50_focal_seen_w2v.h5')
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/keras/engine/network.py", line 1166, in load_weights
f, self.layers, reshape=reshape)
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/keras/engine/saving.py", line 1045, in load_weights_from_hdf5_group
reshape=reshape)
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/keras/engine/saving.py", line 682, in preprocess_weights_for_loading
weights = convert_nested_model(weights)
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/keras/engine/saving.py", line 658, in convert_nested_model
original_backend=original_backend))
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/keras/engine/saving.py", line 801, in preprocess_weights_for_loading
weights[0] = np.transpose(weights[0], (3, 2, 0, 1))
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 598, in transpose
return _wrapfunc(a, 'transpose', axes)
File "/usr/local/miniconda3/envs/keras/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 51, in _wrapfunc
return getattr(obj, method)(*args, **kwds)
ValueError: axes don't match array

Your work is very great! Would you like to share the training codes. I want to test it on my dataset.

Why do I use your code to test for at most 10% accuracy, but not 12.4% of your paper?

Hello,I want to train a model on my own dataset ,but I don`t know how to generate a pre-trained model by using word vectors and focal loss.So can you please support the pre-trained codes,thank you!

How was the file word_w2v.txt generated?

For my own data set, know the tag name, how do we generate the file?

I faced a warning during training this code on coco2014.

train_vocab_w2v.py:367: UserWarning: Output "nms" missing from loss dictionary. We assume this was done on purpose, and we will not be expecting any data to be passed to "nms" during training. optimizer=keras.optimizers.adam(lr=1e-5, clipnorm=0.001) 202 209

But this warning did not affect the running process. I want to know whether this warning will affect the performance of trained models. If so, how can I avoid this warning during my training process? I appreciate that if you can help me to solve this problem. Thank you for your attention.

hello, I want to ask you some questions about the num classes. Does the num_classes equal 65 and 80 during training and testing on MSCOCO dataset?