Return y_true_out got all zeros about yolov3-tf2 HOT 13 CLOSED

I'm also getting all zeros when testing it after training with a custom dataset

from yolov3-tf2.

@andydion You are not "getting zeros when testing", it because your label after transformed were all zeros. (not sure if your data is like this or not, but mine is).

Above line code attached indicates that it returned a zeros initialized y_true_out but does not assign any values to it and returned directly.

I really wonder why author using this code can successfully train a detector........

from yolov3-tf2.

The tensor_scatter_nd_update code really works, I tested it before.
https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/tensor_scatter_nd_update

The output tensor is mostly zero because it only contains data on the center cell of a bounding box.
the transform target function returns a new tensor created by combining the update array.

Can you check if the out put is indeed all zero by calling tf.reduce_sum ?

from yolov3-tf2.

@zzh8829 Thanks for your reply. After checking the label are mostly zeros but not all. However, training the network the output is always zeros, can not see any bounding box at all.

from yolov3-tf2.

Do you find any solutions to this problem? I have this problem too when I train with my dataset.

from yolov3-tf2.

@waiinta No clue for now, at least this model can not work properly on coco with tf_eagar mode. But when using keras fit the loss seems decrease, I have not check the output yet. It looks like only keras fit can training model properly.

from yolov3-tf2.

Hi @jinfagang I can reproduce the same issue with tf.keras, however, I realize this while I am testing the code using detect.py. I get boxes, scores, classes and nms all 0 tensors. The image to be detected has no bounding box. I am wondering if I am performing training the right way, as I can see my model training with slow but decreasing loss! Will be grateful if someone could help

from yolov3-tf2.

@Pari-singh Abviously your model has not converge and all boxes generates well fail.

I think I have some data preparation error when training on coco since the author can train it on VOC. But the result definite not right

from yolov3-tf2.

@jinfagang, I run the detection only after my model has converged, i.e., when train loss = 5.97e4 and val loss = 5.83e4 (epoch 39)

I tried even with other images and epochs and still getting all output values as 0.

from yolov3-tf2.

@Pari-singh With so high loss, it's hard to predict any results. there must be somewhere wrong in codes at least in my situation, converges slow even I adjust bigger learning rate.

from yolov3-tf2.

I am not sure what should be the range/unit of the loss, for this code in general. If the author or someone who implemented this successfully could shed some light, would be very helpful

from yolov3-tf2.

@jinfagang @Pari-singh Have you guys been able to solve the training issues?

from yolov3-tf2.

https://github.com/zzh8829/yolov3-tf2/blob/master/docs/training_voc.md

from yolov3-tf2.