yezhen17 / 3dioumatch-pvrcnn Goto Github PK
View Code? Open in Web Editor NEW[CVPR 2021] PyTorch implementation of 3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection.
[CVPR 2021] PyTorch implementation of 3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection.
Hello,
I noticed that this line does not work as intended. The self.unlabeled_index_list
is initialized incorrectly because the elements in self.sample_index_list
are string
, not int
. Therefore the set subtraction doesn't remove the labeled items. Can you confirm this?
I encountered such a problem during training and don't know how to solve it. I did not pre-train and used pv_rcnn_ssl.yaml
Traceback (most recent call last): File "train.py", line 237, in <module> main() File "train.py", line 151, in main total_epochs=args.epochs File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/__init__.py", line 53, in build_dataloader logger=logger, File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/kitti/kitti_dataset_ssl.py", line 25, in __init__ dataset_cfg=dataset_cfg, class_names=class_names, training=training, root_path=root_path, logger=logger File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/dataset.py", line 32, in __init__ ) if self.training else None File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/augmentor/data_augmentor.py", line 26, in __init__ cur_augmentor = getattr(self, cur_cfg.NAME)(config=cur_cfg) File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/augmentor/data_augmentor.py", line 37, in gt_sampling logger=self.logger File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/augmentor/database_sampler.py", line 24, in __init__ [self.db_infos[cur_class].extend(infos[cur_class]) for cur_class in class_names] File "/content/drive/MyDrive/3DIoU/3DIoUMatch-PVRCNN/pcdet/datasets/augmentor/database_sampler.py", line 24, in <listcomp> [self.db_infos[cur_class].extend(infos[cur_class]) for cur_class in class_names] KeyError: 'Car'
when I run
python -m pcdet.datasets.kitti.kitti_dataset create_part_dbinfos /tools/cfgs/dataset_configs/kitti_dataset.yaml train_0.01_1
there have been two results of splitting the data under different machines. One is only processing 37 sets of data, and the other is processing 3769 sets of data. Which is right here? I understand that the group of 37 is correct, but why does the group of 3769 appear?
Hi @THU17cyz
I have trouble when i reproduces SSL with 0.02 data.
I modified the code to use batch_size 4 in one gpu.
Then, I pre-trained and trained model with 0.02 split data that uploaded in your repository.
In conclusion, I modified the batch size(16->4)
I am not modified other setting(i.e. epochs, learning rate)
I think pre-trained model looks well, but ssl trained model does not looks good.
The first table is my pre-trained model's results, and the second one is my ssl-trained model's results.
I report the accuracy of the last epoch model.
Performance of cyclist is improved, but pedestrian's is decreased.
I only modified the code like this.
kitti_dataset_ssl.py line: 46
if self.training:
all_train = len(self.kitti_infos)
#self.unlabeled_index_list = list(set(list(range(all_train))) - set(self.sample_index_list)) # float()!!!
self.unlabeled_index_list = list(set(list(range(all_train))) - set([int(i) for i in self.sample_index_list]))
# print(self.unlabeled_index_list)
self.unlabeled_kitti_infos = []
Remove labeled data index from unlabeled_index_list.
kitti_dataset_ssl.py line: 390
if self.training:
#index_unlabeled = np.random.choice(self.unlabeled_index_list, 1)[0]
index_unlabeled = np.random.choice(len(self.unlabeled_kitti_infos), 1)[0]
info_unlabeled = copy.deepcopy(self.unlabeled_kitti_infos[index_unlabeled])
data_dict_unlabeled = self.get_item_single(info_unlabeled, no_db_sample=True)
return [data_dict_labeled, data_dict_unlabeled]
else:
return data_dict_labeled
np.random.choice(self.unlabeled_index_list,1)[0] => np.random.choice(len(self.unlabeled_kitti_infos),1)[0]
In pv_rcnn_ssl.py line: 38
if self.training:
batch_dict['mask'] = batch_dict['mask'][:, :1] # modify by chenxyyyy #modified by Honggyu
mask = batch_dict['mask'].view(-1)
I add batch_dict['mask'] = batch_dict['mask'][:, :1] according to #2 (also about pseudo_sem_scores is modified)
In pv_rcnn_ssl.py line: 215
if not self.unlabeled_supervise_cls:
loss_rpn_cls = loss_rpn_cls[labeled_mask, ...].mean() #modified by Honggyu
else:
loss_rpn_cls = loss_rpn_cls[labeled_mask, ...].mean() + loss_rpn_cls[unlabeled_mask, ...].mean() * self.unlabeled_weight #modified by Honggyu
loss_rpn_box = loss_rpn_box[labeled_mask, ...].mean() + loss_rpn_box[unlabeled_mask, ...].mean() * self.unlabeled_weight #modified by Honggyu
loss_point = loss_point[labeled_mask, ...].mean() #modified by Honggyu
loss_rcnn_cls = loss_rcnn_cls[labeled_mask, ...].mean() #modified by Honggyu
if not self.unlabeled_supervise_refine:
loss_rcnn_box = loss_rcnn_box[labeled_mask, ...].mean() #modified by Honggyu
else:
loss_rcnn_box = loss_rcnn_box[labeled_mask, ...].mean() + loss_rcnn_box[unlabeled_mask, ...].mean() * self.unlabeled_weight #modified by Honggyu
loss = loss_rpn_cls + loss_rpn_box + loss_point + loss_rcnn_cls + loss_rcnn_box
tb_dict_ = {}
for key in tb_dict.keys():
if 'loss' in key:
tb_dict_[key+"_labeled"] = tb_dict[key][labeled_mask, ...].mean() #modified by Honggyu
tb_dict_[key + "_unlabeled"] = tb_dict[key][unlabeled_mask, ...].mean() #modified by Honggyu
elif 'acc' in key:
tb_dict_[key+"_labeled"] = tb_dict[key][labeled_mask, ...].mean() #modified by Honggyu
tb_dict_[key + "_unlabeled"] = tb_dict[key][unlabeled_mask, ...].mean() #modified by Honggyu
elif 'point_pos_num' in key:
tb_dict_[key + "_labeled"] = tb_dict[key][labeled_mask, ...].mean() #modified by Honggyu
tb_dict_[key + "_unlabeled"] = tb_dict[key][unlabeled_mask, ...].mean() #modified by Honggyu
else:
tb_dict_[key] = tb_dict[key]
tb_dict_['pseudo_ious'] = torch.cat(pseudo_ious, dim=0).mean()
tb_dict_['pseudo_accs'] = torch.cat(pseudo_accs, dim=0).mean()
# tb_dict_['sem_score_fg'] = sem_score_fg.mean()
# tb_dict_['sem_score_bg'] = sem_score_bg.mean()
tb_dict_['sem_score_fg'] = torch.cat(pseudo_sem_score_fg, dim=0).mean() #modified by Honggyu
tb_dict_['sem_score_bg'] = torch.cat(pseudo_sem_score_bg, dim=0).mean() #modified by Honggyu
I modified .sum() to .mean() to compute loss.
And i modified batch_size = 2 to batch_size = 4 according to #2
And in pv_rcnn_ssl_60.yaml,
OPTIMIZATION.BATCH_SIZE_PER_GPU = 2.
Questions
Additionally, the model's performance fluctuates too much. Is it because the amount of labeled data is too small?(1%, 2%). or Is it because the learning rate is too high? or is it because my batch_size is too small? What do you think about it.
Here is my tensorboard results. ( _teacher is performance of teacher network)
SSL Trained model 0.02_1 in your repo.
SSL Trained model 0.02_2 in your repo.
Hello, thank you for sharing your codes!
According to your paper, 3DIoUMatch seems to use same augmentation strategy with SESS.
In SESS, point sub-sampling is used for weak augmentation.
I wonder which part of your code contains sub-sampling logic.
Thank you
Hello,
I have a question about the supervise_mode setting.
What's the difference when I set supervised_mode == 0, supervised_mode == 1 and supervised_mode == 2?
# only for 100% label
if **self.supervise_mode >**= 1:
filter = iou_max > 0.3
asgn = asgn[filter]
batch_dict['gt_boxes'][ind, ...][:] = torch.zeros_like(batch_dict['gt_boxes'][ind, ...][:])
batch_dict['gt_boxes'][ind, ...][:len(asgn)] = ori_unlabeled_boxes[i, :].gather(dim=0,
index=asgn.unsqueeze(-1).repeat(1, 8))
if self.supervise_mode == 2:
batch_dict['gt_boxes'][ind, ...][:len(asgn), 0:3] += 0.1 * torch.randn((len(asgn), 3), device=iou_max.device) * \
batch_dict['gt_boxes'][ind, ...][
:len(asgn), 3:6]
batch_dict['gt_boxes'][ind, ...][:len(asgn), 3:6] += 0.1 * torch.randn((len(asgn), 3), device=iou_max.device) * \
batch_dict['gt_boxes'][ind, ...][
:len(asgn), 3:6]
hi, i wonder know batch_dict['gt_boxes'][ind, ...][:, 0:7] and ori_unlabeled_boxes[i, :, 0:7] represent what.
for batch_dict['gt_boxes'][ind, ...][:, 0:7], it is a pseudo_box from teacher network.
but for ori_unlabeled_boxes[i, :, 0:7], does it mean a gt box or what?
anchor_by_gt_overlap = iou3d_nms_utils.boxes_iou3d_gpu(
batch_dict['gt_boxes'][ind, ...][:, 0:7],
ori_unlabeled_boxes[i, :, 0:7])
in pv_rcnn_ssl.py line 149-152.
/home/ji/anaconda3/envs/qiuqiu/lib/python3.6/runpy.py:125: RuntimeWarning: 'pcdet.datasets.kitti.kitti_dataset' found in sys.modules after import of package 'pcdet.datasets.kitti', but prior to execution of 'pcdet.datasets.kitti.kitti_dataset'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
Traceback (most recent call last):
File "/home/ji/anaconda3/envs/qiuqiu/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/ji/anaconda3/envs/qiuqiu/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/kitti/kitti_dataset.py", line 517, in
part_split=sys.argv[3]
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/kitti/kitti_dataset.py", line 452, in create_part_dbinfos
dataset = KittiDataset(dataset_cfg=dataset_cfg, class_names=class_names, root_path=data_path, training=True)
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/kitti/kitti_dataset.py", line 23, in init
dataset_cfg=dataset_cfg, class_names=class_names, training=training, root_path=root_path, logger=logger
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/dataset.py", line 32, in init
) if self.training else None
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/augmentor/data_augmentor.py", line 26, in init
cur_augmentor = getattr(self, cur_cfg.NAME)(config=cur_cfg)
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/augmentor/data_augmentor.py", line 37, in gt_sampling
logger=self.logger
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/augmentor/database_sampler.py", line 24, in init
[self.db_infos[cur_class].extend(infos[cur_class]) for cur_class in class_names]
File "/data/ji/code/3DIoUMatch-PVRCNN-main/pcdet/datasets/augmentor/database_sampler.py", line 24, in
[self.db_infos[cur_class].extend(infos[cur_class]) for cur_class in class_names]
KeyError: 'Car'
Hello, what is your GPU configuration? I trained with 8 GTX 1080Ti GPUs which are same as OpenPCDet, yet running this project reported the following error:
RuntimeError: CUDA out of memory. Tried to allocate 216.00 MiB (GPU 4; 10.91 GiB total capacity; 6.64 GiB already allocated; 151.38 MiB free; 6.71 GiB reserved in total by PyTorch)
By the way, my torch version is 1.5.0 and CUDA version is 9.0.176.
link said that pv_rcnn model occupied about 8700-9100M memory usage on each GPU with batch_size=2.
Hello, thank you for your codebase!
I am troubled that which part is the 3diou filter. The following code applies 3diou threshold (0.5/0.25/0.25 for car/ped/cyc), but it seems like class confidence filter.
pv_rcnn_ssl.py line 81
valid_inds = pseudo_score > conf_thresh.squeeze()
Hello, @THU17cyz
I think this part has error.
'ori_boxes[i]' should be revised to 'ori_boxes[inds]'
A loop started from 116,
diff = max_pseudo_box_num - ori_boxes[inds].shape[0]
new_box = torch.cat([ori_boxes[inds], torch.zeros((diff, 8), device=ori_boxes[inds].device)], dim=0)
new_boxes[inds] = new_box
Could you check this for me?
Hi, Thanks for your open-soured code!
I have a question about the iou threshold setting on kitti dataset. In the readme file, you said the IoU thresholds are set 0.5, 0.25, 0.25 for car, pedestrian, and cyclist, respectively. In the last revised paper, the thresholds are set 0.7, 0.5, 0.5 with the same mAP results. Which is correct?
I run the pretraining of pvrcnn on kitti 1% labeled data based on your split. However, I get much worse results than yours.
Here is the data info
INFO Database filter by min points Car: 184 => 168
INFO Database filter by min points Pedestrian: 24 => 24
INFO Database filter by min points Cyclist: 11 => 11
INFO Database filter by difficulty Car: 168 => 127
INFO Database filter by difficulty Pedestrian: 24 => 23
INFO Database filter by difficulty Cyclist: 11 => 8
INFO Loading KITTI dataset
INFO Total samples for KITTI dataset: 37
Here is the result of mAP with 40 recalls
Car [email protected], 0.70, 0.70:
bbox AP40:57.0614, 52.8482, 48.1710
bev AP40:48.6558, 46.2114, 41.6134
3d AP40:10.9533, 11.8981, 11.5772
aos AP40:52.69, 47.91, 43.78
Car [email protected], 0.50, 0.50:
bbox AP40:57.0614, 52.8482, 48.1710
bev AP40:92.2586, 84.3057, 76.9679
3d AP40:84.4601, 76.7450, 69.8524
aos AP40:52.69, 47.91, 43.78
Pedestrian [email protected], 0.50, 0.50:
bbox AP40:6.2376, 6.3730, 6.1160
bev AP40:0.3610, 0.6379, 0.6917
3d AP40:0.2123, 0.2935, 0.3298
aos AP40:3.35, 3.41, 3.21
Pedestrian [email protected], 0.25, 0.25:
bbox AP40:6.2376, 6.3730, 6.1160
bev AP40:4.9317, 5.4842, 5.4997
3d AP40:3.4472, 4.0254, 4.0970
aos AP40:3.35, 3.41, 3.21
Cyclist [email protected], 0.50, 0.50:
bbox AP40:5.6924, 3.8000, 3.9303
bev AP40:2.6481, 1.6338, 1.3922
3d AP40:1.9624, 1.0719, 1.0538
aos AP40:3.76, 2.20, 2.34
Cyclist [email protected], 0.25, 0.25:
bbox AP40:5.6924, 3.8000, 3.9303
bev AP40:5.9757, 4.1799, 3.8489
3d AP40:5.9757, 4.1799, 3.8489
aos AP40:3.76, 2.20, 2.34
Could you help me to solve this issue?
Hello, I'm confused about the performance you report in your paper of arxiv vision and README.
You indicate in your paper that the performance is based on an IoU of 0.7, 0.5, 0.5, but in README it is based on 0.5, 0.25, 0.25.
So which one is real one?
Looking forward to your reply.
Thanks! @THU17cyz
Hello, Thank you for your nice work.
I have a problem in data preprocessing.
When i ran the data preprocessing code with the below command,
python -m pcdet.datasets.kitti.kitti_dataset create_part_dbinfos tools/cfgs/dataset_configs/kitti_dataset.yaml split_0.01_1
But I got this error,
Then, I ran the data infos generator code from original "OpenPCDet" repository for generating "kitti_dbinfos_train.pkl"
python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml
But, when i ran this code, "kitti_dbinfos_train_3712.pkl" is made.
How i can solve this problem? Could you help me?
Hello! @THU17cyz Thank you for open-sourcing your codebase.
I have successfully run your pretrain phase on KITTI. But I had a problem running train phase.
when I use batchsize 1, I met a problem on the below code
# pcdet/models/detectors/pv_rcnn_ssl.py line:257
def update_global_step(self):
self.global_step += 1
alpha = 0.999
# Use the true average until the exponential average is more correct
alpha = min(1 - 1 / (self.global_step + 1), alpha)
for ema_param, param in zip(self.pv_rcnn_ema.parameters(), self.pv_rcnn.parameters()):
ema_param.data.mul_(alpha).add_(1 - alpha, param.data)
It notices that tha add_ cannot take 2 params,So I chage the code ema_param.data.mul_(alpha).add_(1 - alpha, param.data)
to ema_param.data.mul_(alpha).add_((1 - alpha) * param.data)
.
Then I run sucessful
But when I change the batchsize to 2、4 or others,I met the index error on below.
# pcdet/models/detectors/pv_rcnn_ssl.py line:67
for ind in unlabeled_mask:
pseudo_score = pred_dicts[ind]['pred_scores']
pseudo_box = pred_dicts[ind]['pred_boxes']
pseudo_label = pred_dicts[ind]['pred_labels']
pseudo_sem_score = pred_dicts[ind]['pred_sem_scores']
if len(pseudo_label) == 0:
pseudo_boxes.append(pseudo_label.new_zeros((0, 8)).float())
continue
It shows that the num in unlabeled_mask
beyond the range of pred_dicts
. for example, the unlabeled_mask is [2,3,6,7] but the pred_dicts size is 4, 6,7 is illegal.
I want to know if my change is correct, and how to solve the error when the batch size is large?
Looking forward to your reply
thanks for your amazing work! After i deploy the environment successfully, and in the terminal,i can python and import spconv and pcdet successfully.But when i want to run the code, I meet a problem"connot import name iou3d_nms_cuda",how can i solve it?
Hi! Yezhen Cong:
I tested the model which train with 0 unlabeled weight. The result indicated that the model perform better than 1 unlabeled weight.
Does 0 unlabeled weight means completely learn from labeled data ?
Thanks! I'm confused for a long time.
Hello, thank you for your codebase!
I am trouble that the final detection result number is consistent with the input number.
2022-04-14 14:04:05,520 INFO recall_roi_0.3: 0.000000
2022-04-14 14:04:05,520 INFO recall_rcnn_0.3: 0.386345
2022-04-14 14:04:05,520 INFO recall_roi_0.5: 0.000000
2022-04-14 14:04:05,520 INFO recall_rcnn_0.5: 0.223867
2022-04-14 14:04:05,520 INFO recall_roi_0.7: 0.000000
2022-04-14 14:04:05,520 INFO recall_rcnn_0.7: 0.059667
2022-04-14 14:04:05,523 INFO Average predicted number of objects(3744 samples): 7.316
Traceback (most recent call last):
File "train.py", line 237, in <module>
main()
File "train.py", line 230, in main
dist_test=dist_train
File "/tmpnfs/01_3d/02_pl_3d/3DIoUMatch-CaDDN_02/tools/test.py", line 121, in repeat_eval_ckpt
result_dir=cur_result_dir, save_to_file=args.save_to_file
File "/tmpnfs/01_3d/02_pl_3d/3DIoUMatch-CaDDN_02/tools/eval_utils/eval_utils.py", line 113, in eval_one_epoch
output_path=final_output_dir
File "/tmpnfs/01_3d/02_pl_3d/3DIoUMatch-CaDDN_02/pcdet/datasets/kitti/kitti_dataset_ssl_mono.py", line 435, in evaluation
ap_result_str, ap_dict = kitti_eval.get_official_eval_result(eval_gt_annos, eval_det_annos, class_names)
File "/tmpnfs/01_3d/02_pl_3d/3DIoUMatch-CaDDN_02/pcdet/datasets/kitti/kitti_object_eval_python/eval.py", line 675, in get_official_eval_result
gt_annos, dt_annos, current_classes, min_overlaps, compute_aos, PR_detail_dict=PR_detail_dict)
File "/tmpnfs/01_3d/02_pl_3d/3DIoUMatch-CaDDN_02/pcdet/datasets/kitti/kitti_object_eval_python/eval.py", line 588, in do_eval
min_overlaps, compute_aos)
File "/tmpnfs/01_3d/02_pl_3d/3DIoUMatch-CaDDN_02/pcdet/datasets/kitti/kitti_object_eval_python/eval.py", line 469, in eval_class
assert len(gt_annos) == len(dt_annos)
AssertionError````
Thank you for your great research about ssl for 3D detection.
As the title states, I have some questions about weak augmentation for teacher network.
My questions are what augmentations are adopted, and why do they need.
I wonder why weak augmentations are applied because I think the teacher network will be able to generate better pseudo labels if augmentation is not applied.
I know that time has passed since the publication of your paper, but I would like you to answer my question. Thank you.
Thanks for your great works!
In pv_rcnn_ssl.py,
line 53 # self.pv_rcnn_ema.eval() # Important! must be in train mode
why should pv_rcnn_ema model be train mode?
Some layer works different at train mode and eval mode(for example, dropout layer, batch norm layer)
Irun this code on 8 V100 GPUs with setting "train_0.1_1" in ./data/kitti, but a different result was produced.
Car [email protected], 0.70, 0.70:
bbox AP:90.4946, 87.8960, 85.9724
bev AP:89.1578, 84.3747, 78.1961
3d AP:86.5061, 73.5857, 67.6990
aos AP:90.14, 86.91, 84.47
Car [email protected], 0.70, 0.70:
bbox AP:95.7267, 89.3033, 86.2265
bev AP:91.8607, 85.0761, 80.2126
3d AP:87.7689, 73.7096, 68.5043
aos AP:95.30, 88.26, 84.71
Car [email protected], 0.50, 0.50:
bbox AP:90.4946, 87.8960, 85.9724
bev AP:90.5059, 88.8931, 87.6750
3d AP:90.5026, 88.7312, 87.1547
aos AP:90.14, 86.91, 84.47
Car [email protected], 0.50, 0.50:
bbox AP:95.7267, 89.3033, 86.2265
bev AP:95.8453, 91.9532, 89.1067
3d AP:95.7830, 91.7464, 88.7804
aos AP:95.30, 88.26, 84.71
Pedestrian [email protected], 0.50, 0.50:
bbox AP:29.3085, 29.3043, 30.2714
bev AP:12.1812, 9.6326, 10.1116
3d AP:7.7328, 7.7264, 7.3232
aos AP:18.38, 17.35, 17.85
Pedestrian [email protected], 0.50, 0.50:
bbox AP:28.3785, 28.7564, 28.5835
bev AP:9.0040, 8.4401, 8.6786
3d AP:6.9333, 6.4495, 6.4089
aos AP:17.72, 17.06, 16.89
Pedestrian [email protected], 0.25, 0.25:
bbox AP:29.3085, 29.3043, 30.2714
bev AP:31.7072, 32.7220, 31.6044
3d AP:31.6231, 32.6293, 31.4090
aos AP:18.38, 17.35, 17.85
Pedestrian [email protected], 0.25, 0.25:
bbox AP:28.3785, 28.7564, 28.5835
bev AP:32.5248, 31.4872, 32.0119
3d AP:32.4313, 31.3309, 31.1840
aos AP:17.72, 17.06, 16.89
Cyclist [email protected], 0.50, 0.50:
bbox AP:64.7605, 46.2948, 40.6833
bev AP:62.0956, 42.9335, 37.8255
3d AP:59.9777, 37.5589, 36.4040
aos AP:59.62, 41.62, 36.61
Cyclist [email protected], 0.50, 0.50:
bbox AP:65.6239, 44.3176, 40.4676
bev AP:62.5017, 39.4443, 35.8029
3d AP:59.0120, 36.5243, 34.0512
aos AP:60.14, 39.35, 35.69
Cyclist [email protected], 0.25, 0.25:
bbox AP:64.7605, 46.2948, 40.6833
bev AP:63.9723, 44.2477, 38.8835
3d AP:63.9723, 44.2477, 38.8835
aos AP:59.62, 41.62, 36.61
Cyclist [email protected], 0.25, 0.25:
bbox AP:65.6239, 44.3176, 40.4676
bev AP:65.8755, 41.9935, 38.2332
3d AP:65.8755, 41.9935, 38.2332
aos AP:60.14, 39.35, 35.69
2021-08-10 08:50:04,806 INFO Result is save to /mnt/lustre/qiaozhijian/3DIoUMatch-PVRCNN/output/cfgs/kitti_models/pv_rcnn/pretrain_0.01_1/eval/eval_with_train/epoch_80/val
Car [email protected], 0.70, 0.70:
bbox AP:90.5695, 88.4722, 86.0222
bev AP:89.3722, 84.2885, 78.6736
3d AP:87.3749, 75.5330, 68.4084
aos AP:89.97, 87.47, 84.73
Car [email protected], 0.70, 0.70:
bbox AP:96.1003, 91.0679, 86.6432
bev AP:92.3642, 85.4952, 82.3534
3d AP:88.2552, 75.8021, 70.9365
aos AP:95.39, 89.96, 85.32
Car [email protected], 0.50, 0.50:
bbox AP:90.5695, 88.4722, 86.0222
bev AP:90.5336, 89.1459, 87.9113
3d AP:90.5336, 89.0457, 87.6114
aos AP:89.97, 87.47, 84.73
Car [email protected], 0.50, 0.50:
bbox AP:96.1003, 91.0679, 86.6432
bev AP:96.1210, 93.6768, 90.9530
3d AP:96.0771, 91.9605, 89.0258
aos AP:95.39, 89.96, 85.32
Pedestrian [email protected], 0.50, 0.50:
bbox AP:39.2333, 38.0626, 35.9256
bev AP:21.4124, 19.1046, 17.8556
3d AP:16.4727, 15.8462, 15.4276
aos AP:22.01, 20.68, 19.36
Pedestrian [email protected], 0.50, 0.50:
bbox AP:35.6882, 33.2466, 32.1499
bev AP:19.7468, 17.3824, 16.9250
3d AP:15.5564, 14.1068, 13.2984
aos AP:21.43, 19.83, 19.06
Pedestrian [email protected], 0.25, 0.25:
bbox AP:39.2333, 38.0626, 35.9256
bev AP:39.8468, 35.2082, 34.9080
3d AP:39.8210, 35.1217, 34.8599
aos AP:22.01, 20.68, 19.36
Pedestrian [email protected], 0.25, 0.25:
bbox AP:35.6882, 33.2466, 32.1499
bev AP:38.8205, 35.0158, 34.0431
3d AP:38.7950, 34.9411, 33.9429
aos AP:21.43, 19.83, 19.06
Cyclist [email protected], 0.50, 0.50:
bbox AP:80.4026, 54.9720, 53.7440
bev AP:74.4704, 51.5182, 46.6159
3d AP:72.5318, 50.1919, 45.4282
aos AP:70.34, 47.42, 45.78
Cyclist [email protected], 0.50, 0.50:
bbox AP:80.2445, 53.6214, 51.1826
bev AP:76.9033, 49.3606, 46.4228
3d AP:74.8045, 47.7018, 43.8428
aos AP:69.73, 45.59, 42.78
Cyclist [email protected], 0.25, 0.25:
bbox AP:80.4026, 54.9720, 53.7440
bev AP:79.3710, 53.2193, 51.5633
3d AP:79.3710, 53.2193, 51.5633
aos AP:70.34, 47.42, 45.78
Cyclist [email protected], 0.25, 0.25:
bbox AP:80.2445, 53.6214, 51.1826
bev AP:79.4559, 51.8018, 49.1041
3d AP:79.4559, 51.8018, 49.1041
aos AP:69.73, 45.59, 42.78
2021-08-10 10:17:21,777 INFO Result is save to /mnt/lustre/qiaozhijian/3DIoUMatch-PVRCNN/output/cfgs/kitti_models/pv_rcnn_ssl_60/train_0.01_1/eval/eval_with_train/epoch_60/val
On validation datasets, my results are:
Car 73.7096 75.8021 +2.0925
Ped 6.4495 14.1068 +7.6573
Cyc 36.5243 47.7018 +11.1775
The order of magnitude is a little bit different.Which setting did you experiment on? Could you please provide a trained model?
Hello, i have a simple question?
in pv_rcnn_ssl.py line 55
i am not sure what this code means? Does it means i need to uncomment it when training and annotating it when testing? or just a reminder aims to state the _ema should be in eval state
Hello! Thank you for open-sourcing your codebase.
I have tried to use the repository to first do the pre-training phase on KITTI.
Actually, first, the repository would not run automatically for me, so I had to make some changes:
https://github.com/THU17cyz/3DIoUMatch-PVRCNN/blob/b50d9fd932c42e6916d29ad7153f1d3ccbd55827/pcdet/datasets/kitti/kitti_dataset.py#L452
I changed to training=False (because otherwise, it kept looking for a "kitti_dbinfos_train.pkl", which for some reason I couldn't properly generate with the full split txt - it just gave me an empty list [])
Here:
https://github.com/THU17cyz/3DIoUMatch-PVRCNN/blob/b50d9fd932c42e6916d29ad7153f1d3ccbd55827/pcdet/datasets/kitti/kitti_dataset.py#L226
I added
include_info_idxs = None
if len(self.sample_id_list[0].split(" ")) == 2: # means it's a split
# generate stuff to include
include_info_idxs = [int(line.split(" ")[1]) for line in self.sample_id_list]
for k in range(len(infos)):
if include_info_idxs is not None:
if k not in include_info_idxs:
continue
because there didn't seem to be any code built-in for filtering just the frames inside the smaller split.
With these changes, I tried training on the provided 0.01_1 and 0.01_2 splits, and got:
Split 1:
Car [email protected], 0.70, 0.70:
bbox AP:97.0166, 88.4264, 87.7184
bev AP:89.7851, 85.1410, 79.3515
3d AP:88.5321, 77.2608, 75.0451
aos AP:96.56, 87.50, 86.56
Car [email protected], 0.70, 0.70:
bbox AP:98.1557, 91.4289, 89.1516
bev AP:95.1296, 85.9902, 83.5684
3d AP:91.6326, 79.0419, 74.8542
aos AP:97.68, 90.40, 87.91
Car [email protected], 0.50, 0.50:
bbox AP:97.0166, 88.4264, 87.7184
bev AP:97.0982, 88.7778, 88.4398
3d AP:97.0132, 88.7039, 88.2761
aos AP:96.56, 87.50, 86.56
Car [email protected], 0.50, 0.50:
bbox AP:98.1557, 91.4289, 89.1516
bev AP:98.2354, 93.8260, 91.6486
3d AP:98.1979, 93.5422, 91.4403
aos AP:97.68, 90.40, 87.91
Pedestrian [email protected], 0.50, 0.50:
bbox AP:50.1803, 49.5963, 45.4304
bev AP:42.3984, 37.5393, 35.5838
3d AP:39.5851, 34.8857, 30.8286
aos AP:39.63, 38.13, 35.09
Pedestrian [email protected], 0.50, 0.50:
bbox AP:49.2451, 46.8583, 44.1228
bev AP:40.1931, 35.3439, 31.6750
3d AP:36.0652, 31.3460, 28.2745
aos AP:37.15, 34.31, 32.17
Pedestrian [email protected], 0.25, 0.25:
bbox AP:50.1803, 49.5963, 45.4304
bev AP:59.9030, 54.7150, 53.2468
3d AP:59.8010, 54.6809, 53.1366
aos AP:39.63, 38.13, 35.09
Pedestrian [email protected], 0.25, 0.25:
bbox AP:49.2451, 46.8583, 44.1228
bev AP:59.3406, 55.3669, 52.1828
3d AP:59.2904, 55.2920, 51.0803
aos AP:37.15, 34.31, 32.17
Cyclist [email protected], 0.50, 0.50:
bbox AP:55.6971, 39.5461, 38.2764
bev AP:54.8198, 38.2224, 37.1284
3d AP:53.1655, 37.4204, 36.4887
aos AP:46.97, 32.58, 31.53
Cyclist [email protected], 0.50, 0.50:
bbox AP:55.8919, 37.8161, 33.9657
bev AP:54.7186, 35.0099, 32.7953
3d AP:53.0402, 34.0532, 31.7318
aos AP:46.12, 30.03, 27.06
Cyclist [email protected], 0.25, 0.25:
bbox AP:55.6971, 39.5461, 38.2764
bev AP:55.2914, 38.8157, 37.4016
3d AP:55.2914, 38.8157, 37.4016
aos AP:46.97, 32.58, 31.53
Cyclist [email protected], 0.25, 0.25:
bbox AP:55.8919, 37.8161, 33.9657
bev AP:55.2210, 36.9877, 33.1595
3d AP:55.2210, 36.9877, 33.1595
aos AP:46.12, 30.03, 27.06
Split 2:
Car [email protected], 0.70, 0.70:
bbox AP:96.6914, 88.7559, 87.1686
bev AP:89.6414, 85.7397, 79.2499
3d AP:88.1025, 77.2806, 69.6538
aos AP:95.96, 87.87, 86.05
Car [email protected], 0.70, 0.70:
bbox AP:98.1452, 91.6966, 87.1124
bev AP:95.1446, 86.3751, 81.5480
3d AP:91.1320, 77.4718, 72.7242
aos AP:97.34, 90.72, 85.97
Car [email protected], 0.50, 0.50:
bbox AP:96.6914, 88.7559, 87.1686
bev AP:97.0062, 89.1552, 88.0703
3d AP:97.5818, 89.0250, 87.5697
aos AP:95.96, 87.87, 86.05
Car [email protected], 0.50, 0.50:
bbox AP:98.1452, 91.6966, 87.1124
bev AP:98.2979, 94.3327, 89.6986
3d AP:98.4326, 92.2598, 89.3975
aos AP:97.34, 90.72, 85.97
Pedestrian [email protected], 0.50, 0.50:
bbox AP:57.6434, 52.8282, 47.3798
bev AP:50.7659, 45.5090, 43.2965
3d AP:48.7103, 43.6313, 38.8455
aos AP:44.33, 40.31, 36.23
Pedestrian [email protected], 0.50, 0.50:
bbox AP:55.4885, 50.9032, 47.0967
bev AP:49.5267, 44.7794, 39.9836
3d AP:46.8996, 41.1147, 37.1701
aos AP:41.19, 37.13, 33.90
Pedestrian [email protected], 0.25, 0.25:
bbox AP:57.6434, 52.8282, 47.3798
bev AP:60.1065, 58.1532, 52.5542
3d AP:60.0202, 54.5100, 52.4791
aos AP:44.33, 40.31, 36.23
Pedestrian [email protected], 0.25, 0.25:
bbox AP:55.4885, 50.9032, 47.0967
bev AP:60.1045, 56.1739, 51.0589
3d AP:60.0185, 55.0687, 49.9226
aos AP:41.19, 37.13, 33.90
Cyclist [email protected], 0.50, 0.50:
bbox AP:37.7206, 34.5612, 33.7128
bev AP:33.1583, 29.8266, 29.3790
3d AP:31.5708, 28.8987, 28.5315
aos AP:35.55, 32.47, 31.58
Cyclist [email protected], 0.50, 0.50:
bbox AP:34.7546, 32.2395, 30.0869
bev AP:31.0673, 26.7222, 25.0152
3d AP:28.4158, 25.6330, 23.0792
aos AP:32.82, 29.64, 27.79
Cyclist [email protected], 0.25, 0.25:
bbox AP:37.7206, 34.5612, 33.7128
bev AP:34.2568, 30.1642, 29.7915
3d AP:34.2568, 30.1642, 29.7915
aos AP:35.55, 32.47, 31.58
Cyclist [email protected], 0.25, 0.25:
bbox AP:34.7546, 32.2395, 30.0869
bev AP:31.7888, 28.1894, 25.4120
3d AP:31.7888, 28.1894, 25.4120
aos AP:32.82, 29.64, 27.79
Which are actually quite comparable to the paper's results:
but not the revised results in the readme:
which are much lower (lower numbers, at looser IoU threshold) due to fixed ground-truth sampling
I was wondering if you had any ideas why this may be the case?
I didn't even generate the full ground truth database, and during training, I see the line:
INFO Total samples for KITTI dataset: 37
So indeed I am training on the reduced split
作者您好,我想研究的是半监督学习在单目3D目标检测上的应用(主要是kitti数据集,单目模型选择monodle,gupnet),就类似于您的pvrcnn实验。想先搭建一个半监督的框架,整体就如同您论文里面的那样,学生-教师,ema等等,具体的阈值过滤先不管,但是苦于没有合适的代码,您说您的代码只适合pvrcnn,那可以进行局部修改让他适用于单目模型啊?
期待您的回复
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.