Problem in training with custom dataset about alphaction HOT 4 CLOSED

The problem seems to be related to the data. According to the traceback, PyAV failed to decode some video and tried to raise an exception. However, PyTorch failed to construct the PyAV exception (see PyAV-Org/PyAV#485). I'd recommend you to check your data, found which video cannot be successfully decoded. Or, use try...except block to capture this video and then ignore it.

from alphaction.

Thanks for suggestions! 🙏
Finally I did successfully started training with a custom dataset.
Indeed, you were right, the problem was with my dataset.

P.S.: How much time did it take to train with AVA dataset?
In our case, the eta is showing 7 days in Colab Pro with 16GB Tesla V100.

2020-11-05 15:57:54,437 alphaction.trainer INFO: eta: 7 days, 5:56:30 iter: 7500 loss_pose_action: 0.0000 (0.0330) loss_object_interaction: 0.0000 (0.0003) loss_person_interaction: 0.0000 (0.0003) total_loss: 0.0000 (0.0624) accuracy_pose_action: 1.0000 (0.9737) accuracy_object_interaction: 1.0000 (0.9983) accuracy_person_interaction: 1.0000 (0.9982) time: 0.7186 (0.7177) data: 0.0260 (0.0255) lr: 0.000125 max mem: 5387

from alphaction.

For AVA dataset, training with 8 GPUs will cost around one day. Since you are training with single GPU, 7 days are reasonable. I think I have already solved your problem, so I will close this issue now. Feel free to reopen it if you have any more questions about this issue.

from alphaction.

Hi, I have a question about the custom dataset. I see your comment "I have annotated custom videos and gone through all the steps described in Data.md". I am confused that have you need to change 3 bbox.json in boxes in AVA directory structure when building your custom dataset?
Thanks before for your reply.

from alphaction.