has anyone tried the mobileNet on KITTI dataset about mobilenet HOT 10 OPEN

Has started training successfully, following the steps:

1. download pretrained weight from the README.md
2. use the path to the pretrained weight as the CHECK_POINT in train_mobilenetdet_on_kitti.sh
3. modify the file "checkpoint" inside the pretrained weight folder: update the path to local pretrained weight file. For example:
   model_checkpoint_path: "MobileNet/data/mobilenetdet-model/model.ckpt-906808"
   all_model_checkpoint_paths: "MobileNet/data/mobilenetdet-model/model.ckpt-906808"

from mobilenet.

Thank you. This was very valuable advice to me!

from mobilenet.

i download pretrained weight from the README.md, but got loss nan. what is the problem do you know?

from mobilenet.

python2 will be fine

from mobilenet.

python2 will be fine

@lijunhong5457 do you mean, after using python2 instead of python3, the nan loss error vanished?

from mobilenet.

yes, when change python3 to python2 , there is no problem to train it.
i modify the checkpoint like below:
model_checkpoint_path: model.ckpt-906808"
all_model_checkpoint_paths: "model.ckpt-906808"
and I save the checkpoint to CHECK_POINT folder. it works well.

from mobilenet.

Thanks for letting me know that, @lijunhong5457 . But I am still facing InvalidArgumentError due to NaNs in the histograms. Have you ever faced that? Here is a snippet.

InvalidArgumentError (see above for traceback): Nan in summary histogram for: MobileNet/conv_ds_3/dw_batch_norm/moving_variance_1
         [[Node: MobileNet/conv_ds_3/dw_batch_norm/moving_variance_1 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](MobileNet/conv_ds_3/dw_batch_norm/moving_variance_1/tag, MobileNet/conv_ds_3/dw_batch_norm/moving_variance/read)]]
         [[Node: fifo_queue_Dequeue/_1 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_56_fifo_queue_Dequeue", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

from mobilenet.

@Santara I think that your checkpoint was improperly modified, resulting in incorrect initial parameter loading. For checkpoint, don't add new lines or spaces manually, just modify them on the original basis. But the premise is that you modify the path so that the program can find the model.

from mobilenet.

@Santara when you get loss nan, you should clear train_dir to avoid that program load wrong parameter.

from mobilenet.

Thank you for all the help, @lijunhon - but even after doing everything you suggested, I am still getting NaN in summary histograms. Is it because I am running on a CPU? It should not be, right?

from mobilenet.