Hi
The model is predicting the lane even if the nothing is there (image with all pixel with 0 / 1 value) or if I give a cat image as input, I am getting 3-4 lanes on that.

can you please explain the strategy behind this line of code?
link lanes = np.ones((self.dataset.max_lanes, 1 + 2 + 2 * self.dataset.max_points), dtype=np.float32) * -1e5
what/why we have 1, and 2,2* self.dataset.max_points?
why do we multiply it with * -1e5?

I would like to detect lanes in an image but not from the viewpoint of the moving car. For example, the lanes are horizontal in the image. Is this possible? Are there any restrictions implemented like the sorting of the lanes that prevent this task?

Hello, everyone.
As the paper said, the PolyLaneNet could run in 115 FPS with a recent GPU.
I am very interested in your work. We are also working on lane detection.
And I would like to know which kind of GPU do you used for this work?

Thanks for your answer.

@lucastabelini Hi! Great codes you are providing here! But I have a question.
Is the sharing of horizon between lanes a possible reason for the low upperbound performance (since the labels don't accurately indicate horizon line)? My colleague just tried directly fitting each lane in the TuSimple test set with 2-order/3-order polynomials and got almost 100% accuracy. But in your paper the upperbound is only ~98%.
BTW, the FP & FN in your TABLE 3 seem rather too low for a <98% Acc. I'm not sure where the problem is.

@lucastabelini Hi, PloyLaneNet is an amazing work. I find that you employ LPD in your paper, however, I cannot find the code of LPD script. I guess that LPD code is the function eval_json in utils\metric.py, but I am not quite sure about this. Could you tell me where is the LPD code?
Thank you very much!

Thanks for your contribution. when i try to reproduce the experiment results with tusimple dataset, i found that 1) the cls_loss is always 0 , is it normal? and the run test.py file with the 500th epoch .pth , it has a low accuracy,.can you give me some advices?

Congratulations on your great achievements, I am a beginner and I am very interested in your results. I want to reproduce the data in your paper. If I want to modify the polynomial degree, where should I modify it? Thank you very much.

Hi, thanks for the project, it is very helpful for me.
But I cannot get the pretrained model in the google driver, only config and log inside.

How can i get the pretrained model?

I have downloaded the dataset LLASMAS and there are two zip files: labels.zip and color_images.zip. It seems no official recommended way to organize directory. So what's your way to arrange directory?

when I executed this:

python3 test.py --exp_name tusimple --cfg config.yaml --epoch 2695

[2021-03-09 08:49:42,416] [INFO] Starting testing.
[2021-03-09 08:49:42,583] [ERROR] Uncaught exception
Traceback (most recent call last):
File "test.py", line 159, in
, mean_loss = test(model, test_loader, evaluator, exp_root, cfg, epoch=test_epoch, view=args.view)
File "test.py", line 23, in test
model.load_state_dict(torch.load(os.path.join(exp_root, "models", "model{:03d}.pt".format(epoch)))['model'])
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1224, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PolyRegression:
Missing key(s) in state_dict: "model._conv_stem.weight", "model._bn0.weight", "model._bn0.bias", "model._bn0.running_mean", "model._bn0.running_var", "model._blocks.0._depthwise_conv.weight", "model._blocks.0._bn1.weight", "model._blocks.0._bn1.bias", "model._blocks.0._bn1.running_mean", "model._blocks.0._bn1.running_var", "model._blocks.0._se_reduce.weight", "model._blocks.0._se_reduce.bias", "model._blocks.0._se_expand.weight", "model._blocks.0._se_expand.bias", "model._blocks.0._project_conv.weight", "model._blocks.0._bn2.weight", "model._blocks.0._bn2.bias", "model._blocks.0._bn2.running_mean", "model._blocks.0._bn2.running_var", "model._blocks.1._expand_conv.weight", "model._blocks.1._bn0.weight", "model._blocks.1._bn0.bias", "model._blocks.1._bn0.running_mean", "model._blocks.1._bn0.running_var", "model._blocks.1._depthwise_conv.weight", "model._blocks.1._bn1.weight", "model._blocks.1._bn1.bias", "model._blocks.1._bn1.running_mean", "model._blocks.1._bn1.running_var", "model._blocks.1._se_reduce.weight", "model._blocks.1._se_reduce.bias", "model._blocks.1._se_expand.weight", "model._blocks.1._se_expand.bias", "model._blocks.1._project_conv.weight", "model._blocks.1._bn2.weight", "model._blocks.1._bn2.bias", "model._blocks.1._bn2.running_mean", "model._blocks.1._bn2.running_var", "model._blocks.2._expand_conv.weight", "model._blocks.2._bn0.weight", "model._blocks.2._bn0.bias", "model._blocks.2._bn0.running_mean", "model._blocks.2._bn0.running_var", "model._blocks.2._depthwise_conv.weight", "model._blocks.2._bn1.weight", "model._blocks.2._bn1.bias", "model._blocks.2._bn1.running_mean", "model._blocks.2._bn1.running_var", "model._blocks.2._se_reduce.weight", "model._blocks.2._se_reduce.bias", "model._blocks.2._se_expand.weight", "model._blocks.2._se_expand.bias", "model._blocks.2._project_conv.weight", "model._blocks.2._bn2.weight", "model._blocks.2._bn2.bias", "model._blocks.2._bn2.running_mean", "model._blocks.2._bn2.running_var", "model._blocks.3._expand_conv.weight", "model._blocks.3._bn0.weight", "model._blocks.3._bn0.bias", "model._blocks.3._bn0.running_mean", "model._blocks.3._bn0.running_var", "model._blocks.3._depthwise_conv.weight", "model._blocks.3._bn1.weight", "model._blocks.3._bn1.bias", "model._blocks.3._bn1.running_mean", "model._blocks.3._bn1.running_var", "model._blocks.3._se_reduce.weight", "model._blocks.3._se_reduce.bias", "model._blocks.3._se_expand.weight", "model._blocks.3._se_expand.bias", "model._blocks.3._project_conv.weight", "model._blocks.3._bn2.weight", "model._blocks.3._bn2.bias", "model._blocks.3._bn2.running_mean", "model._blocks.3._bn2.running_var", "model._blocks.4._expand_conv.weight", "model._blocks.4._bn0.weight", "model._blocks.4._bn0.bias", "model._blocks.4._bn0.running_mean", "model._blocks.4._bn0.running_var", "model._blocks.4._depthwise_conv.weight", "model._blocks.4._bn1.weight", "model._blocks.4._bn1.bias", "model._blocks.4._bn1.running_mean", "model._blocks.4._bn1.running_var", "model._blocks.4._se_reduce.weight", "model._blocks.4._se_reduce.bias", "model._blocks.4._se_expand.weight", "model._blocks.4._se_expand.bias", "model._blocks.4._project_conv.weight", "model._blocks.4._bn2.weight", "model._blocks.4._bn2.bias", "model._blocks.4._bn2.running_mean", "model._blocks.4._bn2.running_var", "model._blocks.5._expand_conv.weight", "model._blocks.5._bn0.weight", "model._blocks.5._bn0.bias", "model._blocks.5._bn0.running_mean", "model._blocks.5._bn0.running_var", "model._blocks.5._depthwise_conv.weight", "model._blocks.5._bn1.weight", "model._blocks.5._bn1.bias", "model._blocks.5._bn1.running_mean", "model._blocks.5._bn1.running_var", "model._blocks.5._se_reduce.weight", "model._blocks.5._se_reduce.bias", "model._blocks.5._se_expand.weight", "model._blocks.5._se_expand.bias", "model._blocks.5._project_conv.weight", "model._blocks.5._bn2.weight", "model._blocks.5._bn2.bias", "model._blocks.5._bn2.running_mean", "model._blocks.5._bn2.running_var", "model._blocks.6._expand_conv.weight", "model._blocks.6._bn0.weight", "model._blocks.6._bn0.bias", "model._blocks.6._bn0.running_mean", "model._blocks.6._bn0.running_var", "model._blocks.6._depthwise_conv.weight", "model._blocks.6._bn1.weight", "model._blocks.6._bn1.bias", "model._blocks.6._bn1.running_mean", "model._blocks.6._bn1.running_var", "model._blocks.6._se_reduce.weight", "model._blocks.6._se_reduce.bias", "model._blocks.6._se_expand.weight", "model._blocks.6._se_expand.bias", "model._blocks.6._project_conv.weight", "model._blocks.6._bn2.weight", "model._blocks.6._bn2.bias", "model._blocks.6._bn2.running_mean", "model._blocks.6._bn2.running_var", "model._blocks.7._expand_conv.weight", "model._blocks.7._bn0.weight", "model._blocks.7._bn0.bias", "model._blocks.7._bn0.running_mean", "model._blocks.7._bn0.running_var", "model._blocks.7._depthwise_conv.weight", "model._blocks.7._bn1.weight", "model._blocks.7._bn1.bias", "model._blocks.7._bn1.running_mean", "model._blocks.7._bn1.running_var", "model._blocks.7._se_reduce.weight", "model._blocks.7._se_reduce.bias", "model._blocks.7._se_expand.weight", "model._blocks.7._se_expand.bias", "model._blocks.7._project_conv.weight", "model._blocks.7._bn2.weight", "model._blocks.7._bn2.bias", "model._blocks.7._bn2.running_mean", "model._blocks.7._bn2.running_var", "model._blocks.8._expand_conv.weight", "model._blocks.8._bn0.weight", "model._blocks.8._bn0.bias", "model._blocks.8._bn0.running_mean", "model._blocks.8._bn0.running_var", "model._blocks.8._depthwise_conv.weight", "model._blocks.8._bn1.weight", "model._blocks.8._bn1.bias", "model._blocks.8._bn1.running_mean", "model._blocks.8._bn1.running_var", "model._blocks.8._se_reduce.weight", "model._blocks.8._se_reduce.bias", "model._blocks.8._se_expand.weight", "model._blocks.8._se_expand.bias", "model._blocks.8._project_conv.weight", "model._blocks.8._bn2.weight", "model._blocks.8._bn2.bias", "model._blocks.8._bn2.running_mean", "model._blocks.8._bn2.running_var", "model._blocks.9._expand_conv.weight", "model._blocks.9._bn0.weight", "model._blocks.9._bn0.bias", "model._blocks.9._bn0.running_mean", "model._blocks.9._bn0.running_var", "model._blocks.9._depthwise_conv.weight", "model._blocks.9._bn1.weight", "model._blocks.9._bn1.bias", "model._blocks.9._bn1.running_mean", "model._blocks.9._bn1.running_var", "model._blocks.9._se_reduce.weight", "model._blocks.9._se_reduce.bias", "model._blocks.9._se_expand.weight", "model._blocks.9._se_expand.bias", "model._blocks.9._project_conv.weight", "model._blocks.9._bn2.weight", "model._blocks.9._bn2.bias", "model._blocks.9._bn2.running_mean", "model._blocks.9._bn2.running_var", "model._blocks.10._expand_conv.weight", "model._blocks.10._bn0.weight", "model._blocks.10._bn0.bias", "model._blocks.10._bn0.running_mean", "model._blocks.10._bn0.running_var", "model._blocks.10._depthwise_conv.weight", "model._blocks.10._bn1.weight", "model._blocks.10._bn1.bias", "model._blocks.10._bn1.running_mean", "model._blocks.10._bn1.running_var", "model._blocks.10._se_reduce.weight", "model._blocks.10._se_reduce.bias", "model._blocks.10._se_expand.weight", "model._blocks.10._se_expand.bias", "model._blocks.10._project_conv.weight", "model._blocks.10._bn2.weight", "model._blocks.10._bn2.bias", "model._blocks.10._bn2.running_mean", "model._blocks.10._bn2.running_var", "model._blocks.11._expand_conv.weight", "model._blocks.11._bn0.weight", "model._blocks.11._bn0.bias", "model._blocks.11._bn0.running_mean", "model._blocks.11._bn0.running_var", "model._blocks.11._depthwise_conv.weight", "model._blocks.11._bn1.weight", "model._blocks.11._bn1.bias", "model._blocks.11._bn1.running_mean", "model._blocks.11._bn1.running_var", "model._blocks.11._se_reduce.weight", "model._blocks.11._se_reduce.bias", "model._blocks.11._se_expand.weight", "model._blocks.11._se_expand.bias", "model._blocks.11._project_conv.weight", "model._blocks.11._bn2.weight", "model._blocks.11._bn2.bias", "model._blocks.11._bn2.running_mean", "model._blocks.11._bn2.running_var", "model._blocks.12._expand_conv.weight", "model._blocks.12._bn0.weight", "model._blocks.12._bn0.bias", "model._blocks.12._bn0.running_mean", "model._blocks.12._bn0.running_var", "model._blocks.12._depthwise_conv.weight", "model._blocks.12._bn1.weight", "model._blocks.12._bn1.bias", "model._blocks.12._bn1.running_mean", "model._blocks.12._bn1.running_var", "model._blocks.12._se_reduce.weight", "model._blocks.12._se_reduce.bias", "model._blocks.12._se_expand.weight", "model._blocks.12._se_expand.bias", "model._blocks.12._project_conv.weight", "model._blocks.12._bn2.weight", "model._blocks.12._bn2.bias", "model._blocks.12._bn2.running_mean", "model._blocks.12._bn2.running_var", "model._blocks.13._expand_conv.weight", "model._blocks.13._bn0.weight", "model._blocks.13._bn0.bias", "model._blocks.13._bn0.running_mean", "model._blocks.13._bn0.running_var", "model._blocks.13._depthwise_conv.weight", "model._blocks.13._bn1.weight", "model._blocks.13._bn1.bias", "model._blocks.13._bn1.running_mean", "model._blocks.13._bn1.running_var", "model._blocks.13._se_reduce.weight", "model._blocks.13._se_reduce.bias", "model._blocks.13._se_expand.weight", "model._blocks.13._se_expand.bias", "model._blocks.13._project_conv.weight", "model._blocks.13._bn2.weight", "model._blocks.13._bn2.bias", "model._blocks.13._bn2.running_mean", "model._blocks.13._bn2.running_var", "model._blocks.14._expand_conv.weight", "model._blocks.14._bn0.weight", "model._blocks.14._bn0.bias", "model._blocks.14._bn0.running_mean", "model._blocks.14._bn0.running_var", "model._blocks.14._depthwise_conv.weight", "model._blocks.14._bn1.weight", "model._blocks.14._bn1.bias", "model._blocks.14._bn1.running_mean", "model._blocks.14._bn1.running_var", "model._blocks.14._se_reduce.weight", "model._blocks.14._se_reduce.bias", "model._blocks.14._se_expand.weight", "model._blocks.14._se_expand.bias", "model._blocks.14._project_conv.weight", "model._blocks.14._bn2.weight", "model._blocks.14._bn2.bias", "model._blocks.14._bn2.running_mean", "model._blocks.14._bn2.running_var", "model._blocks.15._expand_conv.weight", "model._blocks.15._bn0.weight", "model._blocks.15._bn0.bias", "model._blocks.15._bn0.running_mean", "model._blocks.15._bn0.running_var", "model._blocks.15._depthwise_conv.weight", "model._blocks.15._bn1.weight", "model._blocks.15._bn1.bias", "model._blocks.15._bn1.running_mean", "model._blocks.15._bn1.running_var", "model._blocks.15._se_reduce.weight", "model._blocks.15._se_reduce.bias", "model._blocks.15._se_expand.weight", "model._blocks.15._se_expand.bias", "model._blocks.15._project_conv.weight", "model._blocks.15._bn2.weight", "model._blocks.15._bn2.bias", "model._blocks.15._bn2.running_mean", "model._blocks.15._bn2.running_var", "model._conv_head.weight", "model._bn1.weight", "model._bn1.bias", "model._bn1.running_mean", "model._bn1.running_var", "model._fc.regular_outputs_layer.weight", "model._fc.regular_outputs_layer.bias".
Unexpected key(s) in state_dict: "model.conv1.weight", "model.bn1.weight", "model.bn1.bias", "model.bn1.running_mean", "model.bn1.running_var", "model.bn1.num_batches_tracked", "model.layer1.0.conv1.weight", "model.layer1.0.bn1.weight", "model.layer1.0.bn1.bias", "model.layer1.0.bn1.running_mean", "model.layer1.0.bn1.running_var", "model.layer1.0.bn1.num_batches_tracked", "model.layer1.0.conv2.weight", "model.layer1.0.bn2.weight", "model.layer1.0.bn2.bias", "model.layer1.0.bn2.running_mean", "model.layer1.0.bn2.running_var", "model.layer1.0.bn2.num_batches_tracked", "model.layer1.1.conv1.weight", "model.layer1.1.bn1.weight", "model.layer1.1.bn1.bias", "model.layer1.1.bn1.running_mean", "model.layer1.1.bn1.running_var", "model.layer1.1.bn1.num_batches_tracked", "model.layer1.1.conv2.weight", "model.layer1.1.bn2.weight", "model.layer1.1.bn2.bias", "model.layer1.1.bn2.running_mean", "model.layer1.1.bn2.running_var", "model.layer1.1.bn2.num_batches_tracked", "model.layer1.2.conv1.weight", "model.layer1.2.bn1.weight", "model.layer1.2.bn1.bias", "model.layer1.2.bn1.running_mean", "model.layer1.2.bn1.running_var", "model.layer1.2.bn1.num_batches_tracked", "model.layer1.2.conv2.weight", "model.layer1.2.bn2.weight", "model.layer1.2.bn2.bias", "model.layer1.2.bn2.running_mean", "model.layer1.2.bn2.running_var", "model.layer1.2.bn2.num_batches_tracked", "model.layer2.0.conv1.weight", "model.layer2.0.bn1.weight", "model.layer2.0.bn1.bias", "model.layer2.0.bn1.running_mean", "model.layer2.0.bn1.running_var", "model.layer2.0.bn1.num_batches_tracked", "model.layer2.0.conv2.weight", "model.layer2.0.bn2.weight", "model.layer2.0.bn2.bias", "model.layer2.0.bn2.running_mean", "model.layer2.0.bn2.running_var", "model.layer2.0.bn2.num_batches_tracked", "model.layer2.0.downsample.0.weight", "model.layer2.0.downsample.1.weight", "model.layer2.0.downsample.1.bias", "model.layer2.0.downsample.1.running_mean", "model.layer2.0.downsample.1.running_var", "model.layer2.0.downsample.1.num_batches_tracked", "model.layer2.1.conv1.weight", "model.layer2.1.bn1.weight", "model.layer2.1.bn1.bias", "model.layer2.1.bn1.running_mean", "model.layer2.1.bn1.running_var", "model.layer2.1.bn1.num_batches_tracked", "model.layer2.1.conv2.weight", "model.layer2.1.bn2.weight", "model.layer2.1.bn2.bias", "model.layer2.1.bn2.running_mean", "model.layer2.1.bn2.running_var", "model.layer2.1.bn2.num_batches_tracked", "model.layer2.2.conv1.weight", "model.layer2.2.bn1.weight", "model.layer2.2.bn1.bias", "model.layer2.2.bn1.running_mean", "model.layer2.2.bn1.running_var", "model.layer2.2.bn1.num_batches_tracked", "model.layer2.2.conv2.weight", "model.layer2.2.bn2.weight", "model.layer2.2.bn2.bias", "model.layer2.2.bn2.running_mean", "model.layer2.2.bn2.running_var", "model.layer2.2.bn2.num_batches_tracked", "model.layer2.3.conv1.weight", "model.layer2.3.bn1.weight", "model.layer2.3.bn1.bias", "model.layer2.3.bn1.running_mean", "model.layer2.3.bn1.running_var", "model.layer2.3.bn1.num_batches_tracked", "model.layer2.3.conv2.weight", "model.layer2.3.bn2.weight", "model.layer2.3.bn2.bias", "model.layer2.3.bn2.running_mean", "model.layer2.3.bn2.running_var", "model.layer2.3.bn2.num_batches_tracked", "model.layer3.0.conv1.weight", "model.layer3.0.bn1.weight", "model.layer3.0.bn1.bias", "model.layer3.0.bn1.running_mean", "model.layer3.0.bn1.running_var", "model.layer3.0.bn1.num_batches_tracked", "model.layer3.0.conv2.weight", "model.layer3.0.bn2.weight", "model.layer3.0.bn2.bias", "model.layer3.0.bn2.running_mean", "model.layer3.0.bn2.running_var", "model.layer3.0.bn2.num_batches_tracked", "model.layer3.0.downsample.0.weight", "model.layer3.0.downsample.1.weight", "model.layer3.0.downsample.1.bias", "model.layer3.0.downsample.1.running_mean", "model.layer3.0.downsample.1.running_var", "model.layer3.0.downsample.1.num_batches_tracked", "model.layer3.1.conv1.weight", "model.layer3.1.bn1.weight", "model.layer3.1.bn1.bias", "model.layer3.1.bn1.running_mean", "model.layer3.1.bn1.running_var", "model.layer3.1.bn1.num_batches_tracked", "model.layer3.1.conv2.weight", "model.layer3.1.bn2.weight", "model.layer3.1.bn2.bias", "model.layer3.1.bn2.running_mean", "model.layer3.1.bn2.running_var", "model.layer3.1.bn2.num_batches_tracked", "model.layer3.2.conv1.weight", "model.layer3.2.bn1.weight", "model.layer3.2.bn1.bias", "model.layer3.2.bn1.running_mean", "model.layer3.2.bn1.running_var", "model.layer3.2.bn1.num_batches_tracked", "model.layer3.2.conv2.weight", "model.layer3.2.bn2.weight", "model.layer3.2.bn2.bias", "model.layer3.2.bn2.running_mean", "model.layer3.2.bn2.running_var", "model.layer3.2.bn2.num_batches_tracked", "model.layer3.3.conv1.weight", "model.layer3.3.bn1.weight", "model.layer3.3.bn1.bias", "model.layer3.3.bn1.running_mean", "model.layer3.3.bn1.running_var", "model.layer3.3.bn1.num_batches_tracked", "model.layer3.3.conv2.weight", "model.layer3.3.bn2.weight", "model.layer3.3.bn2.bias", "model.layer3.3.bn2.running_mean", "model.layer3.3.bn2.running_var", "model.layer3.3.bn2.num_batches_tracked", "model.layer3.4.conv1.weight", "model.layer3.4.bn1.weight", "model.layer3.4.bn1.bias", "model.layer3.4.bn1.running_mean", "model.layer3.4.bn1.running_var", "model.layer3.4.bn1.num_batches_tracked", "model.layer3.4.conv2.weight", "model.layer3.4.bn2.weight", "model.layer3.4.bn2.bias", "model.layer3.4.bn2.running_mean", "model.layer3.4.bn2.running_var", "model.layer3.4.bn2.num_batches_tracked", "model.layer3.5.conv1.weight", "model.layer3.5.bn1.weight", "model.layer3.5.bn1.bias", "model.layer3.5.bn1.running_mean", "model.layer3.5.bn1.running_var", "model.layer3.5.bn1.num_batches_tracked", "model.layer3.5.conv2.weight", "model.layer3.5.bn2.weight", "model.layer3.5.bn2.bias", "model.layer3.5.bn2.running_mean", "model.layer3.5.bn2.running_var", "model.layer3.5.bn2.num_batches_tracked", "model.layer4.0.conv1.weight", "model.layer4.0.bn1.weight", "model.layer4.0.bn1.bias", "model.layer4.0.bn1.running_mean", "model.layer4.0.bn1.running_var", "model.layer4.0.bn1.num_batches_tracked", "model.layer4.0.conv2.weight", "model.layer4.0.bn2.weight", "model.layer4.0.bn2.bias", "model.layer4.0.bn2.running_mean", "model.layer4.0.bn2.running_var", "model.layer4.0.bn2.num_batches_tracked", "model.layer4.0.downsample.0.weight", "model.layer4.0.downsample.1.weight", "model.layer4.0.downsample.1.bias", "model.layer4.0.downsample.1.running_mean", "model.layer4.0.downsample.1.running_var", "model.layer4.0.downsample.1.num_batches_tracked", "model.layer4.1.conv1.weight", "model.layer4.1.bn1.weight", "model.layer4.1.bn1.bias", "model.layer4.1.bn1.running_mean", "model.layer4.1.bn1.running_var", "model.layer4.1.bn1.num_batches_tracked", "model.layer4.1.conv2.weight", "model.layer4.1.bn2.weight", "model.layer4.1.bn2.bias", "model.layer4.1.bn2.running_mean", "model.layer4.1.bn2.running_var", "model.layer4.1.bn2.num_batches_tracked", "model.layer4.2.conv1.weight", "model.layer4.2.bn1.weight", "model.layer4.2.bn1.bias", "model.layer4.2.bn1.running_mean", "model.layer4.2.bn1.running_var", "model.layer4.2.bn1.num_batches_tracked", "model.layer4.2.conv2.weight", "model.layer4.2.bn2.weight", "model.layer4.2.bn2.bias", "model.layer4.2.bn2.running_mean", "model.layer4.2.bn2.running_var", "model.layer4.2.bn2.num_batches_tracked", "model.fc.regular_outputs_layer.weight", "model.fc.regular_outputs_layer.bias".

load_state_dict maybe cause problem

how can i solve this?

thank you

Hello,thanks for your great work,
when i train polyLaneNet with my dataset about 30000 images within about 50 epochs
i run the demo like this, what's the problem with it？

hello, I have trained polyLanenet with my data, but the results performance are bad.
the epochs are 250, the poly loss is around 0.1

Did you have a try to test in gray images on llamas datasets? Is there anything should be changed?

Thanks for your sharing.

Tusimple labeled 4 lanes for one image, and they have fixed order like left-left, ego-left, ego-right, right-right from 0 to 3.
Does the output of PolyLaneNet(L1 ~ L5) have the save order, or it just predict lane randomly?

If I want training my own dataset, do I need to label lanes in a fixed order?

Hi,
I'm curious why the training epoch is so high e.g. 384 ~ 2000+.
Does it take most of time for the polynomials fitting?

• What's the polynomials performance at the earlier epoch?
• What's the training loss value e.g. tusimple data set after 2000 epoch?

How does the program call the camera as input?
thank you

I think the subscript here should be k=0.

Sorry, I have one more question. Unlike LANEATT, I did not find another file in polylanenet to test the speed. I got 1008 FPS after running test.py. Should I run another file to test the real speed?

How to get lane type, such as dashed or solid?

Hello. You did a very good job.
I am trying my own implementation in Tensorflow in Google Colab but it is not working as expected. The loss kept around 78 and could not decrease anymore.
Could you have sometime for reviewing?

Dear Author:
Could you please tell me whether the code can run on multiple GPU devices during training, without any further laboring by me?

Do you have python script which can be used to test my own images?
I tried to modify test.py, but without any success.

Thanks,

Hello, I used your method to train 2695 epoch, but the test results are not very satisfactory, as shown in the figure below, is this something wrong with me?

Thanks for sharing your great job with us. Is it possible to replace the feature extraction backbone with a smaller backbone? Such as MobileNet, ShuffleNet, and SqueezeNet, etc.

def forward(self, x, epoch=None, **kwargs):
    output, extra_outputs = self.model(x, **kwargs)
    for i in range(len(self.curriculum_steps)):
        if epoch is not None and epoch < self.curriculum_steps[i]: 
            output[-len(self.curriculum_steps) + i] = 0
    return output, extra_outputs

I am confused about the implement of the code when conducting the 2order experiment. If the "self.curriculum_steps" is setted as [9000,0,0,0] and the batch size is 16. My understanding about the code is as follow, the shape of "output" is torch.Size([16, 35]), and the line 12 in "output" will be setted as 0 (i.e. [-4,35] = 0). If do this, some samples will be setted as 0 in the batch. To this end, it will not achieve the goal of setting high-order polynomial coefficients to 0. Maybe it should be applied to dimension 1 instead of 0? Could you tell me why use this way to achieve the function? Thank you!

In https://github.com/lukemelas/EfficientNet-PyTorch, its image reading package is PIL.Image, and it's RGB mode.
But in your code, it's opencv-python and it's BGR mode.
I think it doesn't impact the final performance, whatever if use the pretrained weights. And RGB reading mode will make training converge quickly.

thank you ! I can see you use cuda 10 and pytorch==1.4

when I use “train.py”,
set "tusimple.py" --> 'train': ['label_data_0313.json'], there are error as:

total annos 2858
Transforming annotations...
Done.
Loaded pretrained weights for efficientnet-b0
[2020-05-21 17:43:48,538] [INFO] Starting training.
[2020-05-21 17:43:48,538] [INFO] Beginning epoch 1
[2020-05-21 17:43:50,452] [ERROR] Uncaught exception
Traceback (most recent call last):
File "train.py", line 237, in
train_state=train_state,
File "train.py", line 54, in train
loss, loss_dict_i = criterion(outputs, labels, **criterion_parameters)
File "/home/cuda/lzc/road/PolyLaneNet/lib/models.py", line 116, in loss
lower_loss = mse(target_lowers[valid_lanes_idx], pred_lowers[valid_lanes_idx])
IndexError: The shape of the mask [64, 1] at index 0does not match the shape of the indexed tensor [80, 1] at index 0

but when I set "tusimple.py" --> 'train': ['label_data_0601.json'], then begin training.

Hi, gratefully to share your project, and your models are so much, in what case run 115FPS, which backbone? what intput size? is that tusimple_320x180?

Hi, thanks for your code!

I noticed that the performance of the model is not as good as the performance of linear regression with the same MSE loss.
Do they have any difference?
Have you ever tried to let the model learn the coefficients(from linear regression) of polynomial directly？
(Although I think they should be the same.)

best regards

Hello. Thank you for your contribution. In table 3, your method achieves a great performance(Acc: 98.03%). I am really interested in it. Can you share the details of the setting of this experiment ?
@lucastabelini

Could you upload the model up to github?

HI, I use the pretrained model_2695.pt to test my own dataset.but it shows like this:

Below is my config.yaml :

Training settings

seed: 0
exps_dir: 'experiments'
iter_log_interval: 1
iter_time_window: 100
model_save_interval: 1
backup:
model:
name: PolyRegression
parameters:
num_outputs: 35 # (5 lanes) * (1 conf + 2 (upper & lower) + 4 poly coeffs)
pretrained: true
backbone: 'resnet50'
pred_category: false
curriculum_steps: [0, 0, 0, 0]
loss_parameters:
conf_weight: 1
lower_weight: 1
upper_weight: 1
cls_weight: 0
poly_weight: 300
batch_size: 16
epochs: 2695
optimizer:
name: Adam
parameters:
lr: 3.0e-4
lr_scheduler:
name: CosineAnnealingLR
parameters:
T_max: 385

Testing settings

test_parameters:
conf_threshold: 0.5

Dataset settings

datasets:
train:
type: LaneDataset
parameters:
dataset: nolabel_dataset
split: train
img_size: [540, 960]
normalize: true
aug_chance: 0.9090909090909091 # 10/11
augmentations:
- name: Affine
parameters:
rotate: !!python/tuple [-10, 10]
- name: HorizontalFlip
parameters:
p: 0.5
- name: CropToFixedSize
parameters:
width: 540
height: 960
root: "/home/share/make/PolyLaneNet/test_image"

test: &test
type: LaneDataset
parameters:
dataset: nolabel_dataset
normalize: true # Wheter to normalize the input data. Use the same value used in the pretrained model (all pretrained models that I provided used normalization, so you should leave it as it is)
augmentations: [] # List of augmentations. You probably want to leave this empty for testing
img_h: 540 # The height of your test images (they shoud all have the same size)
img_w: 960 # The width of your test images
img_size: [540, 960] # Yeah, this parameter is duplicated for some reason, will fix this when I get time (feel free to open a pull request :))
max_lanes: 5 # Same number used in the pretrained model. If you use a model pretrained on TuSimple (most likely case), you'll use 5 here
root: "/home/share/make/PolyLaneNet/test_image" # Path to the directory containing your test images. The loader will look recursively for image files in this directory
img_ext: ".jpg" # Test images extension (e.g., .png, .jpg)"

val = test

val:
<<: *test

Could you give me some advice ? Thanks

How to prepare the label for model training?

Hello, my English level is very poor, I don't understand how to replace the parameter information in the text. Can you attach some pictures to it?

For example:

Hello. Thank you for your contribution. I want to know what is the difference between the two Acc in the following picture? What do they mean? The result of my operation is about 88%.

Hi,
I am trying to test the trained model on my custom image dataset. I modified the config.yaml file as per the suggestions in another issue but I get the following error

Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm\helpers\pydev\pydevd.py", line 2060, in
main()
File "C:\Program Files\JetBrains\PyCharm\helpers\pydev\pydevd.py", line 2054, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Program Files\JetBrains\PyCharm\helpers\pydev\pydevd.py", line 1405, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "C:\Program Files\JetBrains\PyCharm\helpers\pydev\pydevd.py", line 1412, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Xyedj/PycharmProjects/PolyLaneNet-master/test.py", line 162, in
_, mean_loss = test(model, test_loader, evaluator, exp_root, cfg, epoch=test_epoch, view=args.view)
File "C:/Users/Xyedj/PycharmProjects/PolyLaneNet-master/test.py", line 78, in test
return evaluator, loss / total_iters
ZeroDivisionError: division by zero

I believe it has something to do with the dataset I am using. Can you specify the dataset format to be used with test.py. Do I only need the test images in .jpg or I should have their associated annotations as well?

I trained tusimple_efficientnet-b0 (pretrained) for 40 epoches and loss are around 15. (I also got poor qualitative result from test.py)

if self.share_top_y:
    # inexistent lanes have -1e-5 as lower
    # i'm just setting it to a high value here so that the .min below works fine
    target_lowers[target_lowers < 0] = 1
    target_lowers[...] = target_lowers.min(dim=1, keepdim=True)[0]
    pred_lowers[...] = pred_lowers[:, 0].reshape(-1, 1).expand(pred.shape[0], pred.shape[1])

I am confused how the code implements the function of sharing top-y (h).
The following code seems to extract the lowest point , and share that point.

target_lowers[...] = target_lowers.min(dim=1, keepdim=True)[0]

And the code below seems to get the value of the first column and expand the other column. Why consider the first column as the lowest point？

    pred_lowers[...] = pred_lowers[:, 0].reshape(-1, 1).expand(pred.shape[0], pred.shape[1])

Thank you very much!

Sorry if it seems a silly question, bu I do not know what should I do with the output.
I'm trying to predict the output of frames taken from video and the results are like this

tensor([[ 1.5390e+01,  2.6866e-01,  5.9231e-01, -3.7041e-01,  5.3317e-01,
         -2.5742e-01,  4.1398e-01,  1.5413e+01, -7.1203e-03,  9.4199e-01,
         -5.6711e-01,  2.8064e-01,  8.4359e-02,  4.6005e-01, -2.2299e+01,
          1.8078e-03,  8.2544e-01,  1.4614e-01,  1.0139e-01, -2.6998e-01,
          5.8584e-01, -1.2889e+00,  1.7570e-02,  7.5274e-01,  9.5632e-02,
          7.7286e-02,  3.0168e-01,  3.6816e-01,  5.5416e-01,  7.4196e-02,
          7.2463e-01, -7.6063e-02, -1.3095e-01,  7.5693e-01,  3.2309e-01]],
       grad_fn=<AddmmBackward>)

I tried to draw these points with opencv's polylines but I couldn't.
Can you please tell me how to visualize the output

Hello.

I have my own dataset with a video. There is no clear solution for testing with different datasets. How can I test with my video? Must I convert it as .pt format?

Also When I test .pt file with --view argument it only shows the first image result and it doesn't show others.

I'm here to bother again. I would like to ask what the following thresholds mean? What is the difference and connection between this and the confidence score threshold 0.5 mentioned in the article?

lib.models: line 91

Thanks.

How can i get the outputs ?

when I try to run 'python train.py --validate', something wrong happend.May be in test.py the test function should add a parameter 'device',so that train.py with '--validate 'can work.

Hi, thanks for the wonderful implementation of PolyLaneNet. In the google drive for the saved model, which folder should I be looking for to get the model with the ~93% accuracy?
Thanks!

I just want to reproduce the exact reported metrics by testing the model;
it always shows the : AttributeError: 'NoneType' object has no attribute 'shape'
so maybe the code can't find the test image,so could tell me what's wrong with the below ;
I have download the dataset tusimple, there have two folder:test_set、train_set;
so i put the 0531 folder in the dataset folder(the 0531 file is from the test_set\clips\0531);
the detail is like that ：

├─cfgs
├─figures
├─datasets
│ └─tusimple
│ └─label_data_0531.json
│ └─0531
│ ── └─1492626253262712112
│ ── ── └─1.jpg
│ ── ── └─2.jpg
│ ── ── └─......jpg
│ ── ── └─20.jpg
│ ── └─1492626271917313999
│ ── └─......
│ ── └─1495489446318620134
├─experiments
├─lib
│ └─datasets
└─utils
└─test.py
└─train.py
└─config.yaml

and the root in config.yaml is that root: "datasets/tusimple"