tai-wang / depth-from-motion Goto Github PK

View Code? Open in Web Editor NEW

299.0 8.0 28.0 17.52 MB

[ECCV 2022 oral] Monocular 3D Object Detection with Depth from Motion

License: Apache License 2.0

Shell 0.33% Python 99.50% MATLAB 0.18%

3d-detection autonomous-driving monocular pytorch robotics structure-from-motion

depth-from-motion's People

Contributors

Stargazers

Watchers

depth-from-motion's Issues

关于Reszie3D

您好，训练的时候用的dict(type='Resize3D', img_scale=(640, 384), keep_ratio=True),，测试的时候应该怎么用Reszie3D啊？

dfm training error: ValueError: need at least one array to stack

你好。我在尝试在KITTI训练dfm，但是遇到了“ValueError: need at least one array to stack”报错。

我有点怀疑是data preparation步骤出了问题。根据readme的步骤需要将KITTI raw和Kinematic3D提供的raw_extra合并，而我之前训练过Kinematic3D，发现有些KITTI3D训练集的图片在raw里是没有对应的前三帧。所以我怀疑这个bug因为缺少前三帧导致的。
ImageSetst我是从mmdetection3d下载的，data preparation时是不是要删除ImageSetst中缺少数据的samples？

还有其他原因导致了这个bug吗？

配置与部分终端输出如下：

python tools/train.py configs/dfm/dfm_r34_1x8_kitti-3d-3class.py --work-dir /media/he/Disk/Depth-from-Motion-main/ex_1 --gpus 1 --gpu-id 0
/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/MinkowskiEngine/__init__.py:42: UserWarning: The environment variable `OMP_NUM_THREADS` not set. MinkowskiEngine will automatically set `OMP_NUM_THREADS=16`. If you want to set `OMP_NUM_THREADS` manually, please export it on the command line before running a python script. e.g. `export OMP_NUM_THREADS=12; python your_program.py`. It is recommended to set it below 24.
  "It is recommended to set it below 24.",
tools/train.py:150: UserWarning: `--gpus` is deprecated because we only support single GPU mode in non-distributed training. Use `gpus=1` now.
  warnings.warn('`--gpus` is deprecated because we only support '
fatal: not a git repository (or any parent up to mount point /media/he)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2022-09-08 13:03:24,967 - mmdet - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0]
CUDA available: True
GPU 0: NVIDIA GeForce GTX 1080
CUDA_HOME: /usr
NVCC: Cuda compilation tools, release 10.1, V10.1.24
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
PyTorch: 1.9.0
PyTorch compiling details: PyTorch built with:
  - GCC 7.5
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - Built with CUDA Runtime 11.2
  - NVCC architecture flags: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_50,code=compute_50
  - CuDNN 8.2.1  (built against CUDA 11.3)
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.2, CUDNN_VERSION=8.2.1, CXX_COMPILER=/home/conda/feedstock_root/build_artifacts/pytorch-recipe_1627802491209/_build_env/bin/x86_64-conda-linux-gnu-c++, CXX_FLAGS=-std=c++14 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/conda/feedstock_root/build_artifacts/pytorch-recipe_1627802491209/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/pytorch-recipe_1627802491209/work=/usr/local/src/conda/pytorch-1.9.0 -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/pytorch-recipe_1627802491209/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh=/usr/local/src/conda-prefix -I/usr/local/cuda/include -Wno-deprecated-declarations -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=1, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

TorchVision: 0.10.0
OpenCV: 4.6.0
MMCV: 1.5.0
MMCV Compiler: GCC 9.4
MMCV CUDA Compiler: 10.1
MMDetection: 2.24.0
MMSegmentation: 0.20.0
MMDetection3D: 1.0.0rc3+
spconv2.0: False

2022-09-08 13:03:29,693 - mmdet - INFO - workflow: [('train', 1)], max: 60 epochs
2022-09-08 13:03:29,693 - mmdet - INFO - Checkpoints will be saved to /media/he/Disk/Depth-from-Motion-main/ex_1 by HardDiskBackend.
Traceback (most recent call last):
  File "tools/train.py", line 263, in <module>
    main()
  File "tools/train.py", line 259, in main
    meta=meta)
  File "/media/he/Disk/Depth-from-Motion-main/mmdet3d/apis/train.py", line 351, in train_model
    meta=meta)
  File "/media/he/Disk/Depth-from-Motion-main/mmdet3d/apis/train.py", line 319, in train_detector
    runner.run(data_loaders, cfg.workflow)
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 47, in train
    for i, data_batch in enumerate(self.data_loader):
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/_utils.py", line 425, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/media/he/Disk/Depth-from-Motion-main/mmdet3d/datasets/custom_3d.py", line 434, in __getitem__
    data = self.prepare_train_data(idx)
  File "/media/he/Disk/Depth-from-Motion-main/mmdet3d/datasets/custom_3d.py", line 228, in prepare_train_data
    example = self.pipeline(input_dict)
  File "/media/he/Disk/Depth-from-Motion-main/mmdet3d/datasets/pipelines/compose.py", line 49, in __call__
    data = t(data)
  File "/media/he/Disk/Depth-from-Motion-main/mmdet3d/datasets/pipelines/loading.py", line 541, in __call__
    base_results['cur2prevs'] = np.stack(cur2prevs, axis=0)
  File "<__array_function__ internals>", line 6, in stack
  File "/home/he/anaconda3/envs/dfm/lib/python3.7/site-packages/numpy/core/shape_base.py", line 422, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

无法训练

我该如何使用你的代码和脚本在自己的服务器训练

where to get waymo/kitti_format/ImageSets/test_cam_only.txt?

Thanks for sharing camera-only pipeline for waymo dataset. where to get waymo/kitti_format/ImageSets/test_cam_only.txt?

the use of aggregated volume

Thanks for your excellent work!
The boost of performance is due in the aggregation of mono-volume and stereo-volume according to the paper, however, I found that the aggregated volume is only used for predicting depth and used as the weight of image features. And only the stereo-volume is used for generating BEV features.
I am kind of confused, have you ever tried to use the hybrid-volume to generate BEV features?

question about pose-free version

Hi!
I am curious about your pose-free design, but i can't find the source code.
will the code be released?

and which module will produce the depth estimation D_t used for the self-supervised loss?

Waymo license about pre-trained model

Thanks to the authors for your great work. In README.md, you mentioned:

'To comply the license agreement of Waymo dataset, the pre-trained models on Waymo dataset are not released'.

Does it mean Waymo prohibits the public release of the trained model? Could you please share the source of this?

关于RandomCrop3D实现的问题

请问一下这里增加一个矩阵T的作用是什么啊？我的理解T就是单位阵，这个T看起来有点多余？还是我哪里没有理解到位呢？另外，offset_cam2img和K貌似也是一样？

请问with_lidar_model在训练中的作用是什么？对应paper中的哪部分呢

some error when running mono_det_demo.py

we run mono_det_demo.py following this instruction:python demo/mono_det_demo.py demo/data/kitti/000008.png demo/data/kitti/kitti_000008_infos.pkl configs/dfm/dfm_r34_1x8_kitti-3d-3class.py checkpoints/dfm.pth
FileNotFoundError: [Errno 2] No such file or derectory: 'demo/data/kitti/00008_03.png'
the basic data for 3D detection on KITTI does not contain the image named 00008_03.png, what should I do?

PGD training Waymo Dataset, why set loss_bbox2d weight = 0?

I try to set loss_weight of loss_bbox2d and loss_consistency to 1, and set input resolution to origin 1920 x 1280, but the loss_bbox2d is much higher than KITTI Dataset, why this phenomenon? Thanks !

Table 3.中Mono Only

请问测试Mono Only的话是否只需要改动这里：https://github.com/Tai-Wang/Depth-from-Motion/blob/main/mmdet3d/models/backbones/dfm_backbone.py#L141：
改动为：
return cost -> return mono_cost1

还有其他地方需要调整吗？

Where is the code of the pseudo point cloud visualization in the demo video?

Hello, Thank you for your sharing. Where is the code of the pseudo point cloud visualization in the demo video?

关于深度离散化方式

您好，感谢您优秀的工作！
我试着把深度修改成为LID离散化，但是性能比均匀离散化(UD)的性能更差，而其他很多工作都证明LID往往会比UD更好。我在想会不会是问题出在在cost_volume上，从构建cost_volume到深度预测的过程会经过多次3D卷积，而LID的cost_volume的深度是不均匀的，前面的深度密集，后面的深度稀疏，直接3D卷积，可能网络就比较难兼顾，前后的特征都学不好。请问您是否尝试过使用LID或者SID的深度离散化方式吗？

nuScenes support

Do you have a plan to support nuScenes dataset?

kitti数据集下如何完成训练？

您好，请问现在的代码是不是不能支持kitti上的训练呢？还是只是精度无法对齐呀？

compute_detection_let_metrics_main细节

您好，如果想了解LET-AP计算的细节，该去哪里查看代码呢？您提供的这个compute_detection_let_metrics_main文件是waymo官方提供的还是您自己写的呢？

Training error using "./tools/dist_train.sh"

你好，当我用分布式训练网络的时候遇到了报错。

请问你的环境配置和使用的设备是什么？可以分享你配置好的环境吗？谢谢

报错信息：

./tools/dist_train.sh configs/dfm/dfm_r34_1x8_kitti-3d-3class.py 2  
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py:164: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  "The module torch.distributed.launch is deprecated "
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
WARNING:torch.distributed.run:--use_env is deprecated and will be removed in future releases.
 Please read local_rank from `os.environ('LOCAL_RANK')` instead.
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : ./tools/train.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 2
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : 127.0.0.1:29500
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic__vm4fs4n/none_9c40cmfx
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:53: FutureWarning: This is an experimental API and will be changed in future.
  "This is an experimental API and will be changed in future.", FutureWarning
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic__vm4fs4n/none_9c40cmfx/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic__vm4fs4n/none_9c40cmfx/attempt_0/1/error.json
^CTraceback (most recent call last):
  File "./tools/train.py", line 10, in <module>
Traceback (most recent call last):
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py", line 173, in <module>
    main()
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py", line 169, in main
    run(args)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/run.py", line 624, in run
    )(*cmd_args)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 116, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 348, in wrapper
    return f(*args, **kwargs)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 238, in launch_agent
    result = agent.run()
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/metrics/api.py", line 125, in wrapper
    result = f(*args, **kwargs)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/agent/server/api.py", line 700, in run
    result = self._invoke_run(role)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/agent/server/api.py", line 828, in _invoke_run
    time.sleep(monitor_interval)
KeyboardInterrupt
(dfm) Singularity torch17.simg:~/public_4/liuh/Depth-from-Motion-main> python -m torch.distributed.launch --use_env --nproc_per_node=1 ./tools/train.py ./configs/dfm/dfm_r34_1x8_kitti-3d-3class.py
^CTraceback (most recent call last):
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/__init__.py", line 15, in <module>
    import textwrap
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 963, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 906, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1280, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1252, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1364, in find_spec
  File "<frozen importlib._bootstrap_external>", line 81, in _path_stat
KeyboardInterrupt
(dfm) Singularity torch17.simg:~/public_4/liuh/Depth-from-Motion-main> python -m torch.distributed.launch --use_env --nproc_per_node=2 ./tools/train.py ./configs/dfm/dfm_r34_1x8_kitti-3d-3class.py
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py:164: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  "The module torch.distributed.launch is deprecated "
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : ./tools/train.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 2
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : 127.0.0.1:29500
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic_0qwckanz/none__gjfehp_
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:53: FutureWarning: This is an experimental API and will be changed in future.
  "This is an experimental API and will be changed in future.", FutureWarning
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_0/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48417) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 3/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=1
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_1/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_1/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48447) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 2/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=2
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_2/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_2/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48477) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 1/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=3
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_3/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_0qwckanz/none__gjfehp_/attempt_3/1/error.json
Traceback (most recent call last):
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
  File "./tools/train.py", line 17, in <module>
        from mmdet3d import __version__ as mmdet3d_versionfrom mmdet3d import __version__ as mmdet3d_version

  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
      File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError    : MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).f'MMCV=={mmcv.__version__} is used but incompatible. ' \

AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48529) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:Local worker group finished (FAILED). Waiting 300 seconds for other agents to finish
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:71: FutureWarning: This is an experimental API and will be changed in future.
  "This is an experimental API and will be changed in future.", FutureWarning
INFO:torch.distributed.elastic.agent.server.api:Done waiting for other agents. Elapsed: 0.0007059574127197266 seconds
{"name": "torchelastic.worker.status.FAILED", "source": "WORKER", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": 0, "group_rank": 0, "worker_id": "48529", "role": "default", "hostname": "m07", "state": "FAILED", "total_run_time": 235, "rdzv_backend": "static", "raw_error": "{\"message\": \"<NONE>\"}", "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python\", \"local_rank\": [0], \"role_rank\": [0], \"role_world_size\": [2]}", "agent_restarts": 3}}
{"name": "torchelastic.worker.status.FAILED", "source": "WORKER", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": 1, "group_rank": 0, "worker_id": "48530", "role": "default", "hostname": "m07", "state": "FAILED", "total_run_time": 235, "rdzv_backend": "static", "raw_error": "{\"message\": \"<NONE>\"}", "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python\", \"local_rank\": [1], \"role_rank\": [1], \"role_world_size\": [2]}", "agent_restarts": 3}}
{"name": "torchelastic.worker.status.SUCCEEDED", "source": "AGENT", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": null, "group_rank": 0, "worker_id": null, "role": "default", "hostname": "m07", "state": "SUCCEEDED", "total_run_time": 235, "rdzv_backend": "static", "raw_error": null, "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python\"}", "agent_restarts": 3}}
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py:354: UserWarning: 

**********************************************************************
               CHILD PROCESS FAILED WITH NO ERROR_FILE                
**********************************************************************
CHILD PROCESS FAILED WITH NO ERROR_FILE
Child process 48529 (local_rank 0) FAILED (exitcode 1)
Error msg: Process failed with exitcode 1
Without writing an error file to <N/A>.
While this DOES NOT affect the correctness of your application,
no trace information about the error will be available for inspection.
Consider decorating your top level entrypoint function with
torch.distributed.elastic.multiprocessing.errors.record. Example:

  from torch.distributed.elastic.multiprocessing.errors import record

  @record
  def trainer_main(args):
     # do train
**********************************************************************
  warnings.warn(_no_error_file_warning_msg(rank, failure))
Traceback (most recent call last):
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py", line 173, in <module>
    main()
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py", line 169, in main
    run(args)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/run.py", line 624, in run
    )(*cmd_args)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 116, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 348, in wrapper
    return f(*args, **kwargs)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 247, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
***************************************
        ./tools/train.py FAILED        
=======================================
Root Cause:
[0]:
  time: 2022-09-14_20:01:25
  rank: 0 (local_rank: 0)
  exitcode: 1 (pid: 48529)
  error_file: <N/A>
  msg: "Process failed with exitcode 1"
=======================================
Other Failures:
[1]:
  time: 2022-09-14_20:01:25
  rank: 1 (local_rank: 1)
  exitcode: 1 (pid: 48530)
  error_file: <N/A>
  msg: "Process failed with exitcode 1"
***************************************

(dfm) Singularity torch17.simg:~/public_4/liuh/Depth-from-Motion-main> CUDA_VISIBLE_DEVICES="0,1" bash ./tools/dist_train.sh configs/dfm/dfm_r34_1x8_kitti-3d-3class.py 2 
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py:164: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  "The module torch.distributed.launch is deprecated "
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
WARNING:torch.distributed.run:--use_env is deprecated and will be removed in future releases.
 Please read local_rank from `os.environ('LOCAL_RANK')` instead.
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : ./tools/train.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 2
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : 127.0.0.1:29500
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:53: FutureWarning: This is an experimental API and will be changed in future.
  "This is an experimental API and will be changed in future.", FutureWarning
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_0/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version    
from mmdet3d import __version__ as mmdet3d_version  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>

  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).    
f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48582) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 3/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=1
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_1/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_1/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48612) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 2/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=2
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_2/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_2/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48642) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 1/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=3
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_3/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_pinixgh0/none_kkfz_gu2/attempt_3/1/error.json
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
Traceback (most recent call last):
  File "./tools/train.py", line 17, in <module>
    from mmdet3d import __version__ as mmdet3d_version
  File "/home/super/public_4/liuh/Depth-from-Motion-main/mmdet3d/__init__.py", line 5, in <module>
    import mmseg
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/mmseg/__init__.py", line 59, in <module>
    f'MMCV=={mmcv.__version__} is used but incompatible. ' \
AssertionError: MMCV==1.6.0 is used but incompatible. Please install mmcv>=(1, 3, 13, 0, 0, 0), <=(1, 5, 0, 0, 0, 0).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 1 (pid: 48673) of binary: /home/super/anaconda3/envs/dfm/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:Local worker group finished (FAILED). Waiting 300 seconds for other agents to finish
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:71: FutureWarning: This is an experimental API and will be changed in future.
  "This is an experimental API and will be changed in future.", FutureWarning
INFO:torch.distributed.elastic.agent.server.api:Done waiting for other agents. Elapsed: 0.0005748271942138672 seconds
{"name": "torchelastic.worker.status.TERMINATED", "source": "WORKER", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": 0, "group_rank": 0, "worker_id": "48672", "role": "default", "hostname": "m07", "state": "TERMINATED", "total_run_time": 45, "rdzv_backend": "static", "raw_error": null, "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python\", \"local_rank\": [0], \"role_rank\": [0], \"role_world_size\": [2]}", "agent_restarts": 3}}
{"name": "torchelastic.worker.status.FAILED", "source": "WORKER", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": 1, "group_rank": 0, "worker_id": "48673", "role": "default", "hostname": "m07", "state": "FAILED", "total_run_time": 45, "rdzv_backend": "static", "raw_error": "{\"message\": \"<NONE>\"}", "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python\", \"local_rank\": [1], \"role_rank\": [1], \"role_world_size\": [2]}", "agent_restarts": 3}}
{"name": "torchelastic.worker.status.SUCCEEDED", "source": "AGENT", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": null, "group_rank": 0, "worker_id": null, "role": "default", "hostname": "m07", "state": "SUCCEEDED", "total_run_time": 45, "rdzv_backend": "static", "raw_error": null, "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python\"}", "agent_restarts": 3}}
/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py:354: UserWarning: 

**********************************************************************
               CHILD PROCESS FAILED WITH NO ERROR_FILE                
**********************************************************************
CHILD PROCESS FAILED WITH NO ERROR_FILE
Child process 48673 (local_rank 1) FAILED (exitcode 1)
Error msg: Process failed with exitcode 1
Without writing an error file to <N/A>.
While this DOES NOT affect the correctness of your application,
no trace information about the error will be available for inspection.
Consider decorating your top level entrypoint function with
torch.distributed.elastic.multiprocessing.errors.record. Example:

  from torch.distributed.elastic.multiprocessing.errors import record

  @record
  def trainer_main(args):
     # do train
**********************************************************************
  warnings.warn(_no_error_file_warning_msg(rank, failure))
Traceback (most recent call last):
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py", line 173, in <module>
    main()
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py", line 169, in main
    run(args)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/run.py", line 624, in run
    )(*cmd_args)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 116, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 348, in wrapper
    return f(*args, **kwargs)
  File "/home/super/anaconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 247, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
***************************************
        ./tools/train.py FAILED        
=======================================
Root Cause:
[0]:
  time: 2022-09-14_20:07:00
  rank: 1 (local_rank: 1)
  exitcode: 1 (pid: 48673)
  error_file: <N/A>
  msg: "Process failed with exitcode 1"
=======================================
Other Failures:
  <NO_OTHER_FAILURES>
***************************************

dfm_r34_1x8_kitti-3d-3class.py中kitti的位姿矩阵？

dfm_r34_1x8_kitti-3d-3class.py中没有 cam_sync=True这一条，请问哪一个脚本里处理了位姿矩阵呢？还是只用相机内参外参？

Can't inference with pretrained model

Hello, I'm interested in your work and I'm trying to see how your work performs on my test data.

I followed every step in the Installation part on README here.
I downloaded the pre-trained model from the link you provided here.
I tried to infer an image using the pre-trained model and provided configuration by running mono_det_demo.py, but It seems like I loaded the wrong config file cause there are some keys required in inference code that doesn't exist in the config file.
Is that a bug or did I load the wrong config file? Is there any document describing how to infer an image with a known camera matrix?

About the checkpoint of FCOS++

Hi, @Tai-Wang thanks for your work. The MV-FCOS3D++ cfg includes the pre-trained checkpoint https://github.com/Tai-Wang/Depth-from-Motion/blob/main/configs/dfm/multiview-dfm_r101_dcn_2x16_waymoD5-3d-3class_camsync_10sweeps.py#L247 as a prerequisite, but it is not provided, unfortunately.

So I would like to ask how to train the model and get this checkpoint by myself. More specifically, the tech report https://arxiv.org/abs/2207.12716 emphasized FCOS3D++ is trained with only object annotations. I'm not clear what is the difference from the original FCOS3D, by emphasizing this. All in all, would you mind explaining more about the training of FCOS3D++, in favor of the re-implementation of MV-FCOS3D++ by myself? Thanks.

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

	OpenMMLab 1.0 branch	OpenMMLab 2.0 branch
MMEngine		0.x
MMCV	1.x	2.x
MMDetection	0.x 、1.x、2.x	3.x
MMAction2	0.x	1.x
MMClassification	0.x	1.x
MMSegmentation	0.x	1.x
MMDetection3D	0.x	1.x
MMEditing	0.x	1.x
MMPose	0.x	1.x
MMDeploy	0.x	1.x
MMTracking	0.x	1.x
MMOCR	0.x	1.x
MMRazor	0.x	1.x
MMSelfSup	0.x	1.x
MMRotate	1.x	1.x
MMYOLO		0.x

Attention: please create a new virtual environment for OpenMMLab 2.0.

The question about data augmentation

We find that you perform flipping,rescaling and cropping on the input images. When lifting the 2.5 coordinate to 3D, you apply the effect of data augmentation onto the 2.5 coordinate and 3D grid without changing the intrinsic matrix. if we want to change the intrinsic matrix K to perform data augmentation (flipping,rescaling and cropping), we apply rescaling and cropping operation through changing intrinsic matrix K and flip 3D grid for flipping, Is this OK?
When flipping the input images, does the camera pose (cur2prevs) need to change? Thank you very much.

How to prepare the raw data of KITTI?

Thank you for sharing your code. Should we download the raw data for all the categories (city, residential,road campus,person)from http://www.cvlibs.net/datasets/kitti/raw_data.php? There are so many files. Can we download the raw data from http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d and click this link (3 temporally preceding frames (left colour)(36 GB))?

Indoor object detection on the way?

Hello Wang Tai,
Great work and thanks for releasing this contribution.As I can see the data dir includes indoor dataset like scannet. Have you tried this on indoor scenario?

cuda execution failed with error 2

你好，我在使用4张3090分布式训练的时候会报下面的错误，2张就正好吃满，没报错。
我不是很懂这是为什么，您能帮忙看下吗？谢谢。

(dfm) root@root:/opt/data/private/Mono3D# bash tools/dist_train.sh configs/dfm/dfm_r34_1x8_kitti-3d-3class.py 4
/home/miniconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/launch.py:164: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  "The module torch.distributed.launch is deprecated "
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
WARNING:torch.distributed.run:--use_env is deprecated and will be removed in future releases.
 Please read local_rank from `os.environ('LOCAL_RANK')` instead.
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : tools/train.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 4
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : 127.0.0.1:29500
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}
INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic_ri32p5t8/none_zqnifzq1
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
/home/miniconda3/envs/dfm/lib/python3.7/site-packages/torch/distributed/elastic/utils/store.py:53: FutureWarning: This is an experimental API and will be changed in future.
  "This is an experimental API and will be changed in future.", FutureWarning
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1, 2, 3]
  role_ranks=[0, 1, 2, 3]
  global_ranks=[0, 1, 2, 3]
  role_world_sizes=[4, 4, 4, 4]
  global_world_sizes=[4, 4, 4, 4]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_ri32p5t8/none_zqnifzq1/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_ri32p5t8/none_zqnifzq1/attempt_0/1/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker2 reply file to: /tmp/torchelastic_ri32p5t8/none_zqnifzq1/attempt_0/2/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker3 reply file to: /tmp/torchelastic_ri32p5t8/none_zqnifzq1/attempt_0/3/error.json
/opt/data/private/Mono3D/mmdet3d/models/backbones/mink_resnet.py:10: UserWarning: Please follow `getting_started.md` to install MinkowskiEngine.`
  'Please follow `getting_started.md` to install MinkowskiEngine.`')
/opt/data/private/Mono3D/mmdet3d/models/backbones/mink_resnet.py:10: UserWarning: Please follow `getting_started.md` to install MinkowskiEngine.`
  'Please follow `getting_started.md` to install MinkowskiEngine.`')
/opt/data/private/Mono3D/mmdet3d/models/backbones/mink_resnet.py:10: UserWarning: Please follow `getting_started.md` to install MinkowskiEngine.`
  'Please follow `getting_started.md` to install MinkowskiEngine.`')
/opt/data/private/Mono3D/mmdet3d/models/backbones/mink_resnet.py:10: UserWarning: Please follow `getting_started.md` to install MinkowskiEngine.`
  'Please follow `getting_started.md` to install MinkowskiEngine.`')
2022-10-20 13:38:32,539 - mmdet - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0]
CUDA available: True
GPU 0,1,2,3: GeForce RTX 3090
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.2, V11.2.152
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.9.0+cu111
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.0.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.10.0+cu111
OpenCV: 4.6.0
MMCV: 1.5.0
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.24.0
MMSegmentation: 0.20.0
MMDetection3D: 1.0.0rc3+unknown
spconv2.0: False
------------------------------------------------------------
2022-10-20 13:38:33,911 - mmdet - INFO - Distributed training: True
2022-10-20 13:38:34,943 - mmdet - INFO - Config:
point_cloud_range = [2, -30.4, -3, 59.6, 30.4, 1]
model = dict(
    type='DfM',
    depth_cfg=dict(
        mode='UD',
        num_bins=288,
        depth_min=2,
        depth_max=59.6,
        downsample_factor=4),
    voxel_cfg=dict(
        point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1],
        voxel_size=[0.2, 0.2, 0.2]),
    normalizer_clamp_value=10,
    lidar_model=dict(
        type='VoxelNet',
        voxel_layer=dict(
            max_num_points=5,
            point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1],
            voxel_size=[0.05, 0.05, 0.1],
            max_voxels=(40000, 40000)),
        voxel_encoder=dict(type='HardSimpleVFE', num_features=3),
        middle_encoder=dict(
            type='CustomSparseEncoder',
            in_channels=3,
            output_channels=32,
            encoder_strides=((1, ), (2, 1, 1), (2, 1, 1), ((2, 1, 1), 1, 1)),
            sparse_shape=[41, 1216, 1152],
            order=('conv', 'norm', 'act'),
            norm_cfg=dict(type='SyncBN', eps=0.001, momentum=0.01),
            with_final_bnrelu=False,
            output_volume_feat=True),
        backbone=dict(
            type='BEVHourglass',
            in_channels=160,
            out_channels=64,
            norm_cfg=dict(type='SyncBN'),
            output_prehg_feat=False),
        neck=None,
        bbox_head=None,
        init_cfg=dict(
            type='Pretrained',
            checkpoint='checkpoints/mmdet3d-second-teacher.pth')),
    backbone=dict(
        type='LIGAResNet',
        depth=34,
        num_stages=4,
        strides=(1, 2, 1, 1),
        dilations=(1, 1, 2, 4),
        out_indices=(0, 1, 2, 3),
        style='pytorch',
        frozen_stages=-1,
        norm_cfg=dict(type='SyncBN', requires_grad=True),
        norm_eval=False,
        with_max_pool=False,
        block_with_final_relu=False,
        num_channels_factor=(1, 2, 2, 2),
        init_cfg=dict(
            type='Pretrained',
            checkpoint='checkpoints/resnet34-333f7ec4.pth')),
    neck=dict(
        type='SPPUNetNeck',
        in_channels=[3, 64, 128, 128, 128],
        start_level=2,
        sem_channels=[128, 32],
        stereo_channels=[32, 32],
        with_upconv=True,
        cat_img_feature=True,
        norm_cfg=dict(type='GN', num_groups=32, requires_grad=True)),
    neck_2d=dict(
        type='FPN',
        in_channels=[32],
        out_channels=64,
        start_level=0,
        add_extra_convs='on_output',
        num_outs=5),
    bbox_head_2d=dict(
        type='LIGAATSSHead',
        reg_class_agnostic=False,
        seperate_extra_reg_branch=False,
        num_classes=3,
        in_channels=64,
        stacked_convs=4,
        feat_channels=64,
        anchor_generator=dict(
            type='AnchorGenerator',
            ratios=[1.0],
            octave_base_scale=16,
            scales_per_octave=1,
            strides=[4, 8, 16, 32, 64]),
        num_extra_reg_channel=0,
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[0.0, 0.0, 0.0, 0.0],
            target_stds=[0.1, 0.1, 0.2, 0.2]),
        loss_cls=dict(
            type='FocalLoss',
            use_sigmoid=True,
            gamma=2.0,
            alpha=0.25,
            loss_weight=1.0),
        loss_bbox=dict(type='GIoULoss', loss_weight=2.0),
        loss_centerness=dict(
            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
        train_cfg=dict(
            assigner=dict(type='ATSS3DCenterAssigner', topk=9),
            allowed_border=-1,
            pos_weight=-1,
            append_3d_centers=True),
        test_cfg=dict(
            nms_pre=1000,
            min_bbox_size=0,
            score_thr=0.05,
            nms=dict(type='nms', iou_threshold=0.6),
            max_per_img=100)),
    backbone_stereo=dict(
        type='DfMBackbone',
        in_channels=32,
        cv_channels=32,
        num_hg=1,
        cost_sample_factor=4,
        norm_cfg=dict(type='GN', num_groups=32, requires_grad=True)),
    depth_head=dict(
        type='DepthHead',
        with_convs=False,
        depth_cfg=dict(mode='UD', num_bins=288, min_depth=2, max_depth=59.6),
        depth_loss=dict(
            type='balanced_focal',
            loss_weight=1.0,
            fg_weight=5,
            bg_weight=1,
            alpha=1,
            gamma=2),
        downsample_factor=4,
        num_views=1),
    feature_transformation=dict(
        type='FrustumToVoxel',
        sem_atten_feat=True,
        stereo_atten_feat=False,
        num_3dconvs=1,
        cv_channels=32,
        out_channels=32,
        norm_cfg=dict(type='GN', num_groups=32, requires_grad=True)),
    backbone_3d=dict(
        type='BEVHourglass',
        in_channels=160,
        out_channels=64,
        norm_cfg=dict(type='GN', num_groups=32, requires_grad=True)),
    bbox_head_3d=dict(
        type='LIGAAnchor3DHead',
        num_classes=3,
        in_channels=64,
        feat_channels=64,
        num_convs=2,
        use_direction_classifier=True,
        diff_rad_by_sin=True,
        dir_offset=0.7854,
        anchor_generator=dict(
            type='Anchor3DRangeGenerator',
            ranges=[[2, -30.4, -1.78, 59.6, 30.4, -1.78],
                    [2, -30.4, -0.6, 59.6, 30.4, -0.6],
                    [2, -30.4, -0.6, 59.6, 30.4, -0.6]],
            sizes=[[3.9, 1.6, 1.56], [0.8, 0.6, 1.73], [1.76, 0.6, 1.73]],
            rotations=[0, 1.57],
            reshape_out=False),
        assign_per_class=True,
        bbox_coder=dict(type='DeltaXYZWLHRBBoxCoder'),
        loss_cls=dict(
            type='FocalLoss',
            use_sigmoid=True,
            gamma=2.0,
            alpha=0.25,
            loss_weight=1.0),
        loss_bbox=dict(
            type='SmoothL1Loss', beta=0.1111111111111111, loss_weight=0.5),
        loss_dir=dict(
            type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.2),
        loss_iou=dict(type='IOU3DLoss', loss_weight=1.0),
        norm_cfg=dict(type='GN', num_groups=32, requires_grad=True)),
    imitation_cfgs=[
        dict(
            lidar_feature_layer='spatial_features_2d',
            stereo_feature_layer='spatial_features_2d',
            normalize='cw_scale',
            layer='conv2d',
            channel=64,
            kernel_size=1,
            use_relu=False,
            mode='inbox',
            loss_weight=1.0),
        dict(
            lidar_feature_layer='volume_features',
            stereo_feature_layer='volume_features',
            normalize='cw_scale',
            layer='conv3d',
            channel=32,
            kernel_size=1,
            use_relu=False,
            mode='inbox',
            loss_weight=1.0)
    ],
    train_cfg=dict(
        assigner=[
            dict(
                type='MaxIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.6,
                neg_iou_thr=0.45,
                min_pos_iou=0.45,
                ignore_iof_thr=-1),
            dict(
                type='MaxIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.5,
                neg_iou_thr=0.35,
                min_pos_iou=0.35,
                ignore_iof_thr=-1),
            dict(
                type='MaxIoUAssigner',
                iou_calculator=dict(type='BboxOverlapsNearest3D'),
                pos_iou_thr=0.5,
                neg_iou_thr=0.35,
                min_pos_iou=0.35,
                ignore_iof_thr=-1)
        ],
        allowed_border=0,
        pos_weight=-1,
        debug=False),
    test_cfg=dict(
        use_rotate_nms=True,
        nms_across_levels=False,
        nms_thr=0.25,
        score_thr=0.1,
        min_bbox_size=0,
        nms_pre=4096,
        max_num=500))
dataset_type = 'KittiDataset'
data_root = '../all_data/kitti_mono3d_mmdet3d/'
class_names = ['Car', 'Pedestrian', 'Cyclist']
input_modality = dict(use_lidar=False, use_camera=True)
img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=False)
file_client_args = dict(backend='disk')
train_pipeline = [
    dict(type='TruncatedObjectFilter', truncated_threshold=0.98),
    dict(type='IgnoredObjectFilter'),
    dict(type='LoadAnnotations3D'),
    dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=dict(backend='disk'),
        pseudo_lidar=True),
    dict(
        type='VideoPipeline',
        num_ref_imgs=1,
        random=True,
        transforms=[
            dict(
                type='LoadImageFromFileMono3D',
                file_client_args=dict(backend='disk')),
            dict(
                type='RandomFlip3D',
                flip_ratio_bev_horizontal=0.5,
                with_baseline_offset=True),
            dict(
                type='Resize3D',
                ratio_range=(0.95, 1.05),
                keep_ratio=True,
                multiscale_mode='range',
                cam2img_keep_ratio=True),
            dict(
                type='RandomCrop3D',
                crop_size=(320, 1280),
                rel_offset_h=(1, 1),
                rel_offset_w=(0.5, 0.5)),
            dict(type='PhotoMetricDistortion'),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=False),
            dict(type='Pad', size_divisor=32)
        ]),
    dict(type='PointsFoVFilter'),
    dict(
        type='PointsRangeFilter',
        point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1],
        height_filter=False),
    dict(type='GenerateDepthMap', generate_fgmask=True),
    dict(
        type='ObjectRangeFilter',
        point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1],
        filter_scheme='corner',
        min_num_corners=1),
    dict(type='GenerateAmodal2DBoxes'),
    dict(
        type='DefaultFormatBundle3D',
        class_names=['Car', 'Pedestrian', 'Cyclist']),
    dict(
        type='Collect3D',
        keys=[
            'img', 'gt_bboxes_3d', 'gt_labels_3d', 'depth_img',
            'depth_fgmask_img', 'points', 'gt_bboxes', 'centers2d'
        ])
]
test_pipeline = [
    dict(
        type='VideoPipeline',
        num_ref_imgs=1,
        random=False,
        transforms=[
            dict(
                type='LoadImageFromFileMono3D',
                file_client_args=dict(backend='disk')),
            dict(
                type='RandomCrop3D',
                crop_size=(320, 1280),
                rel_offset_h=(1, 1),
                rel_offset_w=(0.5, 0.5)),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=False),
            dict(type='Pad', size_divisor=32)
        ]),
    dict(
        type='DefaultFormatBundle3D',
        class_names=['Car', 'Pedestrian', 'Cyclist'],
        with_label=False),
    dict(type='Collect3D', keys=['img'])
]
data = dict(
    samples_per_gpu=1,
    workers_per_gpu=1,
    train=dict(
        type='KittiDataset',
        data_root='../all_data/kitti_mono3d_mmdet3d/',
        ann_file='../all_data/kitti_mono3d_mmdet3d/kitti_infos_train.pkl',
        split='training',
        pts_prefix='velodyne',
        pipeline=[
            dict(type='TruncatedObjectFilter', truncated_threshold=0.98),
            dict(type='IgnoredObjectFilter'),
            dict(type='LoadAnnotations3D'),
            dict(
                type='LoadPointsFromFile',
                coord_type='LIDAR',
                load_dim=4,
                use_dim=4,
                file_client_args=dict(backend='disk'),
                pseudo_lidar=True),
            dict(
                type='VideoPipeline',
                num_ref_imgs=1,
                random=True,
                transforms=[
                    dict(
                        type='LoadImageFromFileMono3D',
                        file_client_args=dict(backend='disk')),
                    dict(
                        type='RandomFlip3D',
                        flip_ratio_bev_horizontal=0.5,
                        with_baseline_offset=True),
                    dict(
                        type='Resize3D',
                        ratio_range=(0.95, 1.05),
                        keep_ratio=True,
                        multiscale_mode='range',
                        cam2img_keep_ratio=True),
                    dict(
                        type='RandomCrop3D',
                        crop_size=(320, 1280),
                        rel_offset_h=(1, 1),
                        rel_offset_w=(0.5, 0.5)),
                    dict(type='PhotoMetricDistortion'),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=False),
                    dict(type='Pad', size_divisor=32)
                ]),
            dict(type='PointsFoVFilter'),
            dict(
                type='PointsRangeFilter',
                point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1],
                height_filter=False),
            dict(type='GenerateDepthMap', generate_fgmask=True),
            dict(
                type='ObjectRangeFilter',
                point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1],
                filter_scheme='corner',
                min_num_corners=1),
            dict(type='GenerateAmodal2DBoxes'),
            dict(
                type='DefaultFormatBundle3D',
                class_names=['Car', 'Pedestrian', 'Cyclist']),
            dict(
                type='Collect3D',
                keys=[
                    'img', 'gt_bboxes_3d', 'gt_labels_3d', 'depth_img',
                    'depth_fgmask_img', 'points', 'gt_bboxes', 'centers2d'
                ])
        ],
        modality=dict(use_lidar=False, use_camera=True),
        classes=['Car', 'Pedestrian', 'Cyclist'],
        test_mode=False,
        pseudo_lidar=True,
        use_similar_cls=True,
        box_type_3d='LiDAR'),
    val=dict(
        type='KittiDataset',
        data_root='../all_data/kitti_mono3d_mmdet3d/',
        ann_file='../all_data/kitti_mono3d_mmdet3d/kitti_infos_val.pkl',
        split='training',
        pts_prefix='velodyne',
        pipeline=[
            dict(
                type='VideoPipeline',
                num_ref_imgs=1,
                random=False,
                transforms=[
                    dict(
                        type='LoadImageFromFileMono3D',
                        file_client_args=dict(backend='disk')),
                    dict(
                        type='RandomCrop3D',
                        crop_size=(320, 1280),
                        rel_offset_h=(1, 1),
                        rel_offset_w=(0.5, 0.5)),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=False),
                    dict(type='Pad', size_divisor=32)
                ]),
            dict(
                type='DefaultFormatBundle3D',
                class_names=['Car', 'Pedestrian', 'Cyclist'],
                with_label=False),
            dict(type='Collect3D', keys=['img'])
        ],
        modality=dict(use_lidar=False, use_camera=True),
        classes=['Car', 'Pedestrian', 'Cyclist'],
        test_mode=True,
        pseudo_lidar=True,
        box_type_3d='LiDAR'),
    test=dict(
        type='KittiDataset',
        data_root='../all_data/kitti_mono3d_mmdet3d/',
        ann_file='../all_data/kitti_mono3d_mmdet3d/kitti_infos_val.pkl',
        split='training',
        pts_prefix='velodyne',
        pipeline=[
            dict(
                type='VideoPipeline',
                num_ref_imgs=1,
                random=False,
                transforms=[
                    dict(
                        type='LoadImageFromFileMono3D',
                        file_client_args=dict(backend='disk')),
                    dict(
                        type='RandomCrop3D',
                        crop_size=(320, 1280),
                        rel_offset_h=(1, 1),
                        rel_offset_w=(0.5, 0.5)),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=False),
                    dict(type='Pad', size_divisor=32)
                ]),
            dict(
                type='DefaultFormatBundle3D',
                class_names=['Car', 'Pedestrian', 'Cyclist'],
                with_label=False),
            dict(type='Collect3D', keys=['img'])
        ],
        modality=dict(use_lidar=False, use_camera=True),
        classes=['Car', 'Pedestrian', 'Cyclist'],
        test_mode=True,
        pseudo_lidar=True,
        box_type_3d='LiDAR'))
optimizer = dict(type='AdamW', lr=0.001, weight_decay=0.0001)
optimizer_config = dict(grad_clip=dict(max_norm=35.0, norm_type=2))
lr_config = dict(
    policy='LIGA',
    warmup='cosine',
    warmup_iters=464,
    warmup_ratio=0.1,
    step=[50])
total_epochs = 60
runner = dict(type='EpochBasedRunner', max_epochs=60)
checkpoint_config = dict(interval=1, max_keep_ckpts=10)
log_config = dict(
    interval=50,
    hooks=[dict(type='TextLoggerHook'),
           dict(type='TensorboardLoggerHook')])
evaluation = dict(interval=1)
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
work_dir = './work_dirs/dfm_r34_1x8_kitti-3d-3class'
gpu_ids = range(0, 4)

2022-10-20 13:38:34,944 - mmdet - INFO - Set random seed to 0, deterministic: False
2022-10-20 13:38:35,525 - mmcv - WARNING - The model and loaded state dict do not match exactly
/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/base_module.py:126: UserWarning: init_weights of DfMBackbone has been called more than once.
  warnings.warn(f'init_weights of {self.__class__.__name__} has '
2022-10-20 13:38:35,538 - mmdet - INFO - initialize FPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/base_module.py:126: UserWarning: init_weights of FrustumToVoxel has been called more than once.
  warnings.warn(f'init_weights of {self.__class__.__name__} has '
2022-10-20 13:38:35,540 - mmdet - INFO - initialize LIGAATSSHead with init_cfg {'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01, 'override': {'type': 'Normal', 'name': 'atss_cls', 'std': 0.01, 'bias_prob': 0.01}}
2022-10-20 13:38:35,544 - mmdet - INFO - initialize VoxelNet with init_cfg {'type': 'Pretrained', 'checkpoint': 'checkpoints/mmdet3d-second-teacher.pth'}
2022-10-20 13:38:35,545 - mmcv - INFO - load model from: checkpoints/mmdet3d-second-teacher.pth
2022-10-20 13:38:35,545 - mmcv - INFO - load checkpoint from local path: checkpoints/mmdet3d-second-teacher.pth
2022-10-20 13:38:46,441 - mmcv - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: bbox_head.cls_convs.0.0.0.weight, bbox_head.cls_convs.0.0.1.weight, bbox_head.cls_convs.0.0.1.bias, bbox_head.cls_convs.0.0.1.running_mean, bbox_head.cls_convs.0.0.1.running_var, bbox_head.cls_convs.0.0.1.num_batches_tracked, bbox_head.cls_convs.1.0.0.weight, bbox_head.cls_convs.1.0.1.weight, bbox_head.cls_convs.1.0.1.bias, bbox_head.cls_convs.1.0.1.running_mean, bbox_head.cls_convs.1.0.1.running_var, bbox_head.cls_convs.1.0.1.num_batches_tracked, bbox_head.reg_convs.0.0.0.weight, bbox_head.reg_convs.0.0.1.weight, bbox_head.reg_convs.0.0.1.bias, bbox_head.reg_convs.0.0.1.running_mean, bbox_head.reg_convs.0.0.1.running_var, bbox_head.reg_convs.0.0.1.num_batches_tracked, bbox_head.reg_convs.1.0.0.weight, bbox_head.reg_convs.1.0.1.weight, bbox_head.reg_convs.1.0.1.bias, bbox_head.reg_convs.1.0.1.running_mean, bbox_head.reg_convs.1.0.1.running_var, bbox_head.reg_convs.1.0.1.num_batches_tracked, bbox_head.conv_cls.weight, bbox_head.conv_cls.bias, bbox_head.conv_reg.weight, bbox_head.conv_reg.bias, bbox_head.conv_dir_cls.weight, bbox_head.conv_dir_cls.bias, backbone.compress_conv.0.0.weight, backbone.compress_conv.0.1.weight, backbone.compress_conv.0.1.bias, backbone.compress_conv.0.1.running_mean, backbone.compress_conv.0.1.running_var, backbone.compress_conv.0.1.num_batches_tracked

missing keys in source state_dict: backbone.compress_conv.conv.weight, backbone.compress_conv.bn.weight, backbone.compress_conv.bn.bias, backbone.compress_conv.bn.running_mean, backbone.compress_conv.bn.running_var

2022-10-20 13:38:46,447 - mmdet - INFO - initialize LIGAAnchor3DHead with init_cfg {'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01, 'override': {'type': 'Normal', 'name': 'conv_cls', 'std': 0.01, 'bias_prob': 0.01}}
2022-10-20 13:38:46,461 - mmdet - INFO - Model:
DfM(
  (backbone): LIGAResNet(
    (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (bn1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu): ReLU(inplace=True)
    (layer1): ResLayer(
      (0): LigaBasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (1): LigaBasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (2): LigaBasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
    )
    (layer2): ResLayer(
      (0): LigaBasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (2): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (3): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
    )
    (layer3): ResLayer(
      (0): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (1): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (2): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (3): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (4): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (5): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
    )
    (layer4): ResLayer(
      (0): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (1): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (2): LigaBasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)
        (bn1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
    )
  )
  init_cfg={'type': 'Pretrained', 'checkpoint': 'checkpoints/resnet34-333f7ec4.pth'}
  (neck): SPPUNetNeck(
    (spp_branches): ModuleList(
      (0): Sequential(
        (0): AvgPool2d(kernel_size=(64, 64), stride=(64, 64), padding=0)
        (1): ConvModule(
          (conv): Conv2d(128, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
      )
      (1): Sequential(
        (0): AvgPool2d(kernel_size=(32, 32), stride=(32, 32), padding=0)
        (1): ConvModule(
          (conv): Conv2d(128, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
      )
      (2): Sequential(
        (0): AvgPool2d(kernel_size=(16, 16), stride=(16, 16), padding=0)
        (1): ConvModule(
          (conv): Conv2d(128, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
      )
      (3): Sequential(
        (0): AvgPool2d(kernel_size=(8, 8), stride=(8, 8), padding=0)
        (1): ConvModule(
          (conv): Conv2d(128, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
      )
    )
    (upconv_module): upconv_module(
      (conv): ModuleList(
        (0): Sequential(
          (0): Conv2d(512, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (1): Sequential(
          (0): Conv2d(64, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (1): SyncBatchNorm(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (redir): ModuleList(
        (0): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (1): Sequential(
          (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (1): SyncBatchNorm(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (up): Upsample(scale_factor=2.0, mode=bilinear)
    )
    (lastconv): Sequential(
      (0): ConvModule(
        (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (1): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (rpnconv): Sequential(
      (0): ConvModule(
        (conv): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 128, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (1): ConvModule(
        (conv): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
    )
  )
  (backbone_stereo): DfMBackbone(
    (dres0): ConvModule(
      (conv): Conv3d(64, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
      (activate): ReLU(inplace=True)
    )
    (dres1): ConvModule(
      (conv): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
    )
    (hg_stereo): ModuleList(
      (0): hourglass(
        (conv1): Sequential(
          (0): Sequential(
            (0): Conv3d(32, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), bias=False)
            (1): GroupNorm(32, 64, eps=1e-05, affine=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv2): Sequential(
          (0): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          (1): GroupNorm(32, 64, eps=1e-05, affine=True)
        )
        (conv3): Sequential(
          (0): Sequential(
            (0): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), bias=False)
            (1): GroupNorm(32, 64, eps=1e-05, affine=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv4): Sequential(
          (0): Sequential(
            (0): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
            (1): GroupNorm(32, 64, eps=1e-05, affine=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv5): Sequential(
          (0): ConvTranspose3d(64, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), output_padding=(1, 1, 1), bias=False)
          (1): GroupNorm(32, 64, eps=1e-05, affine=True)
        )
        (conv6): Sequential(
          (0): ConvTranspose3d(64, 32, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), output_padding=(1, 1, 1), bias=False)
          (1): GroupNorm(32, 32, eps=1e-05, affine=True)
        )
      )
    )
    (pred_stereo): ModuleList(
      (0): Sequential(
        (0): ConvModule(
          (conv): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
        (1): Conv3d(32, 1, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      )
    )
    (dres0_mono): ConvModule(
      (conv): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
      (activate): ReLU(inplace=True)
    )
    (dres1_mono): ConvModule(
      (conv): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
    )
    (hg_mono): ModuleList(
      (0): hourglass(
        (conv1): Sequential(
          (0): Sequential(
            (0): Conv3d(32, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), bias=False)
            (1): GroupNorm(32, 64, eps=1e-05, affine=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv2): Sequential(
          (0): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          (1): GroupNorm(32, 64, eps=1e-05, affine=True)
        )
        (conv3): Sequential(
          (0): Sequential(
            (0): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), bias=False)
            (1): GroupNorm(32, 64, eps=1e-05, affine=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv4): Sequential(
          (0): Sequential(
            (0): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
            (1): GroupNorm(32, 64, eps=1e-05, affine=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv5): Sequential(
          (0): ConvTranspose3d(64, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), output_padding=(1, 1, 1), bias=False)
          (1): GroupNorm(32, 64, eps=1e-05, affine=True)
        )
        (conv6): Sequential(
          (0): ConvTranspose3d(64, 32, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1), output_padding=(1, 1, 1), bias=False)
          (1): GroupNorm(32, 32, eps=1e-05, affine=True)
        )
      )
    )
    (pred_mono): ModuleList(
      (0): Sequential(
        (0): ConvModule(
          (conv): Conv3d(32, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
        (1): Conv3d(32, 1, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
      )
    )
    (aggregate_cost): Conv2d(144, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)
  )
  (backbone_3d): BEVHourglass(
    (compress_conv): ConvModule(
      (conv): Conv2d(160, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
      (activate): ReLU(inplace=True)
    )
    (bev_hourglass): hourglass2d(
      (conv1): Sequential(
        (0): Sequential(
          (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
          (1): GroupNorm(32, 128, eps=1e-05, affine=True)
        )
        (1): ReLU(inplace=True)
      )
      (conv2): Sequential(
        (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (1): GroupNorm(32, 128, eps=1e-05, affine=True)
      )
      (conv3): Sequential(
        (0): Sequential(
          (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
          (1): GroupNorm(32, 128, eps=1e-05, affine=True)
        )
        (1): ReLU(inplace=True)
      )
      (conv4): Sequential(
        (0): Sequential(
          (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (1): GroupNorm(32, 128, eps=1e-05, affine=True)
        )
        (1): ReLU(inplace=True)
      )
      (conv5): Sequential(
        (0): ConvTranspose2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1), bias=False)
        (1): GroupNorm(32, 128, eps=1e-05, affine=True)
      )
      (conv6): Sequential(
        (0): ConvTranspose2d(128, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1), bias=False)
        (1): GroupNorm(32, 64, eps=1e-05, affine=True)
      )
    )
  )
  (neck_2d): FPN(
    (lateral_convs): ModuleList(
      (0): ConvModule(
        (conv): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (fpn_convs): ModuleList(
      (0): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      )
      (1): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      )
      (2): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      )
      (3): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      )
      (4): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      )
    )
  )
  init_cfg={'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
  (feature_transformation): FrustumToVoxel(
    (voxel_convs): Sequential(
      (0): Sequential(
        (0): ConvModule(
          (conv): Conv3d(64, 32, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), bias=False)
          (gn): GroupNorm(32, 32, eps=1e-05, affine=True)
          (activate): ReLU(inplace=True)
        )
      )
    )
    (voxel_pool): AvgPool3d(kernel_size=(4, 1, 1), stride=(4, 1, 1), padding=0)
  )
  (bbox_head_2d): LIGAATSSHead(
    (loss_cls): FocalLoss()
    (loss_bbox): GIoULoss()
    (relu): ReLU(inplace=True)
    (cls_convs): ModuleList(
      (0): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (1): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (2): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (3): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
    )
    (reg_convs): ModuleList(
      (0): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (1): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (2): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (3): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
    )
    (atss_cls): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (atss_reg): Conv2d(64, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (atss_centerness): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (scales): ModuleList(
      (0): Scale()
      (1): Scale()
      (2): Scale()
      (3): Scale()
      (4): Scale()
    )
    (loss_centerness): CrossEntropyLoss(avg_non_ignore=False)
  )
  init_cfg={'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01, 'override': {'type': 'Normal', 'name': 'atss_cls', 'std': 0.01, 'bias_prob': 0.01}}
  (depth_head): DepthHead(
    (upsample_cost): Upsample(scale_factor=4.0, mode=trilinear)
  )
  (lidar_model): VoxelNet(
    (backbone): BEVHourglass(
      (compress_conv): ConvModule(
        (conv): Conv2d(160, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (activate): ReLU(inplace=True)
      )
      (bev_hourglass): hourglass2d(
        (conv1): Sequential(
          (0): Sequential(
            (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
            (1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv2): Sequential(
          (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (conv3): Sequential(
          (0): Sequential(
            (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
            (1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv4): Sequential(
          (0): Sequential(
            (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          )
          (1): ReLU(inplace=True)
        )
        (conv5): Sequential(
          (0): ConvTranspose2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1), bias=False)
          (1): SyncBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (conv6): Sequential(
          (0): ConvTranspose2d(128, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1), bias=False)
          (1): SyncBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
    )
    (voxel_layer): Voxelization(voxel_size=[0.05, 0.05, 0.1], point_cloud_range=[2, -30.4, -3, 59.6, 30.4, 1], max_num_points=5, max_voxels=(40000, 40000), deterministic=True)
    (voxel_encoder): HardSimpleVFE()
    (middle_encoder): CustomSparseEncoder(
      (conv_input): SparseSequential(
        (0): SubMConv3d()
        (1): SyncBatchNorm(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (encoder_layers): SparseSequential(
        (encoder_layer1): SparseSequential(
          (0): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
        )
        (encoder_layer2): SparseSequential(
          (0): SparseSequential(
            (0): SparseConv3d()
            (1): SyncBatchNorm(32, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
          (1): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(32, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
          (2): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(32, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
        )
        (encoder_layer3): SparseSequential(
          (0): SparseSequential(
            (0): SparseConv3d()
            (1): SyncBatchNorm(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
          (1): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
          (2): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
        )
        (encoder_layer4): SparseSequential(
          (0): SparseSequential(
            (0): SparseConv3d()
            (1): SyncBatchNorm(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
          (1): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
          (2): SparseSequential(
            (0): SubMConv3d()
            (1): SyncBatchNorm(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
            (2): ReLU(inplace=True)
          )
        )
      )
      (conv_out): SparseSequential(
        (0): SparseConv3d()
      )
    )
  )
  init_cfg={'type': 'Pretrained', 'checkpoint': 'checkpoints/mmdet3d-second-teacher.pth'}
  (norm_imitation): ModuleDict(
    (spatial_features_2d): NormalizeLayer()
    (volume_features): NormalizeLayer()
  )
  (conv_imitation): ModuleList(
    (0): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
    (1): Conv3d(32, 32, kernel_size=(1, 1, 1), stride=(1, 1, 1))
  )
  (loss_imitation): WeightedL2WithSigmaLoss()
  (bbox_head_3d): LIGAAnchor3DHead(
    (loss_cls): FocalLoss()
    (loss_bbox): SmoothL1Loss()
    (loss_dir): CrossEntropyLoss(avg_non_ignore=False)
    (loss_iou): IOU3DLoss()
    (cls_convs): Sequential(
      (0): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (1): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
    )
    (reg_convs): Sequential(
      (0): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
      (1): ConvModule(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (gn): GroupNorm(32, 64, eps=1e-05, affine=True)
        (activate): ReLU(inplace=True)
      )
    )
    (conv_cls): Conv2d(64, 18, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (conv_reg): Conv2d(64, 42, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (conv_dir_cls): Conv2d(64, 12, kernel_size=(1, 1), stride=(1, 1))
  )
  init_cfg={'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01, 'override': {'type': 'Normal', 'name': 'conv_cls', 'std': 0.01, 'bias_prob': 0.01}}
)
2022-10-20 13:38:58,204 - mmdet - INFO - Start running, host: root@zy20221019, work_dir: /opt/data/private/Mono3D/work_dirs/dfm_r34_1x8_kitti-3d-3class
2022-10-20 13:38:58,204 - mmdet - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH   ) LIGALrUpdaterHook
(NORMAL      ) CheckpointHook
(LOW         ) DistEvalHook
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
before_train_epoch:
(VERY_HIGH   ) LIGALrUpdaterHook
(NORMAL      ) DistSamplerSeedHook
(LOW         ) IterTimerHook
(LOW         ) DistEvalHook
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
before_train_iter:
(VERY_HIGH   ) LIGALrUpdaterHook
(LOW         ) IterTimerHook
(LOW         ) DistEvalHook
 --------------------
after_train_iter:
(ABOVE_NORMAL) OptimizerHook
(NORMAL      ) CheckpointHook
(LOW         ) IterTimerHook
(LOW         ) DistEvalHook
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
after_train_epoch:
(NORMAL      ) CheckpointHook
(LOW         ) DistEvalHook
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
before_val_epoch:
(NORMAL      ) DistSamplerSeedHook
(LOW         ) IterTimerHook
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
before_val_iter:
(LOW         ) IterTimerHook
 --------------------
after_val_iter:
(LOW         ) IterTimerHook
 --------------------
after_val_epoch:
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
after_run:
(VERY_LOW    ) TextLoggerHook
(VERY_LOW    ) TensorboardLoggerHook
 --------------------
2022-10-20 13:38:58,204 - mmdet - INFO - workflow: [('train', 1)], max: 60 epochs
2022-10-20 13:38:58,204 - mmdet - INFO - Checkpoints will be saved to /opt/data/private/Mono3D/work_dirs/dfm_r34_1x8_kitti-3d-3class by HardDiskBackend.
See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode)
Traceback (most recent call last):
  File "tools/train.py", line 263, in <module>
    main()
  File "tools/train.py", line 259, in main
    meta=meta)
  File "/opt/data/private/Mono3D/mmdet3d/apis/train.py", line 351, in train_model
    meta=meta)
  File "/opt/data/private/Mono3D/mmdet3d/apis/train.py", line 319, in train_detector
    runner.run(data_loaders, cfg.workflow)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
    self.run_iter(data_batch, train_mode=True, **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
    **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 59, in train_step
    output = self.module.train_step(*inputs[0], **kwargs[0])
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 248, in train_step
    losses = self(**data)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 110, in new_func
    return old_func(*args, **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 172, in forward
    return self.forward_train(img, img_metas, **kwargs)
  File "/opt/data/private/Mono3D/mmdet3d/models/detectors/dfm.py", line 360, in forward_train
    points)
  File "/opt/data/private/Mono3D/mmdet3d/models/detectors/dfm.py", line 380, in extract_lidar_model_feat
    voxel_features, coors, batch_size)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 110, in new_func
    return old_func(*args, **kwargs)
  File "/opt/data/private/Mono3D/mmdet3d/models/middle_encoders/sparse_encoder.py", line 342, in forward
    x = self.conv_input(input_sp_tensor)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/ops/sparse_modules.py", line 133, in forward
    input = module(input)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/ops/sparse_conv.py", line 168, in forward
    grid=input.grid)
  File "/home/miniconda3/envs/dfm/lib/python3.7/site-packages/mmcv/ops/sparse_ops.py", line 102, in get_indice_pairs
    int(transpose))
RuntimeError: /tmp/mmcv/mmcv/ops/csrc/pytorch/cuda/sparse_indice.cu 123
cuda execution failed with error 2

tai-wang / depth-from-motion Goto Github PK

depth-from-motion's People

Contributors

Stargazers

Watchers

Forkers

depth-from-motion's Issues

Welcome update to OpenMMLab 2.0

Recommend Projects

Recommend Topics

Recommend Org