larskreuzberg / 4d-stop Goto Github PK

Official code for "4D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregation"

Shell 0.09% Batchfile 0.01% C++ 14.14% Python 83.70% C 0.24% Cuda 1.82%

4d-stop's People

Contributors

Stargazers

Watchers

Forkers

engomarwasfy zhangzw12319 jackzhousz

4d-stop's Issues

About experiments on the NuScenes dataset

Hi, thank you for the great work! In the code I saw options to run on nuScenes dataset. Seems like you converted the nuScenes dataset to the SemanticKITTI format. I did not find existing tools to do this on the internet. May I ask how you did this? Could you share the converted nuScenes dataset or your code to convert the dataset? Thank you again!

Release of the network weights

Hello and thanks a lot for sharing your code !

Do you plan to share the network weights ?

I tried training the network but I only have 8 GiB of GPU memory available, do you think there is a choice of parameter that would be suitable ? It tried reducing batch_num but it does not seems to be sufficient.

Question regarding supplementary material

In the supplementary material of the paper you compare the speed of 4D-PLS and 4D-StOP. You mention reporting the time in scan per second, but the legend of the graph is in seconds per scan ? Which is the correct one ?
I assume seconds per scan is the right one as 4 scans-based methods should be slower than 2 scans based methods.
Does it mean 4-PLS and 4D-StOP would run at best at 0.35FPS in an online setting ?

The size of the 'merged_coords" and 'new_coords' do not match

Hi,

While using "create_center_label.py" to build .pny files, it is different to 4D PLS below code.

StOP:
center_labels = np.zeros((new_points.shape[0], 1))

PLS:
center_labels = np.zeros((new_points.shape[0], 4))

After created .npy files, I tried to train StOP , but encountered the following error in 'SemanticKitti.py" code(merged_coords = np.vstack((merged_coords, new_coords))).

"ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 9 and the array at index 1 has size 6"

Could you help me how to fix it?
Thanks.

Problem of creating conda environment

Thanks for your excellent work!

But I encountered problems when I created the conda environment as you mentioned conda create --name 4dstopp --file requirements.txt. But it failed with following output:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - pointnet2==0.0.0=pypi_0
  - importlib-metadata==4.11.3=pypi_0
  - pandocfilters==1.4.2=pypi_0
  - rsa==4.8=pypi_0
  - vtk==8.1.2=pypi_0
  - bleach==3.1.5=pypi_0
  - fire==0.4.0=pypi_0
  - entrypoints==0.3=pypi_0
  - torchdiffeq==0.1.0=pypi_0
  - a==1.0=pypi_0
  - tensorboard-data-server==0.6.1=pypi_0
  - joblib==0.14.1=pypi_0
  - protobuf==3.20.0=pypi_0
  - nbformat==5.0.6=pypi_0
  - traitlets==4.3.3=pypi_0
  - scikit-learn==0.22.2.post1=pypi_0
  - nuscenes-devkit==1.1.9=pypi_0
  - ffmpeg==4.3=hf484d3e_0
  - pexpect==4.8.0=pypi_0
  - zipp==3.1.0=pypi_0
  - open3d==0.9.0.0=pypi_0
  - parso==0.7.0=pypi_0
  - prompt-toolkit==3.0.5=pypi_0
  - ipykernel==5.2.1=pypi_0
  - google-auth==2.6.2=pypi_0
  - glfw==1.11.2=pypi_0
  - scipy==1.4.1=pypi_0
  - terminado==0.8.3=pypi_0
  - pyzmq==19.0.1=pypi_0
  - tornado==6.0.4=pypi_0
  - jinja2==2.11.2=pypi_0
  - traits==6.0.0=pypi_0
  - traitsui==7.0.0=pypi_0
  - pyqt5==5.14.0=pypi_0
  - pyasn1-modules==0.2.8=pypi_0
  - defusedxml==0.6.0=pypi_0
  - grpcio==1.44.0=pypi_0
  - kiwisolver==1.2.0=pypi_0
  - numpy==1.18.4=pypi_0
  - ipython-genutils==0.2.0=pypi_0
  - decorator==4.4.2=pypi_0
  - mistune==0.8.4=pypi_0
  - jupyter-core==4.6.3=pypi_0
  - nbconvert==5.6.1=pypi_0
  - descartes==1.1.0=pypi_0
  - configobj==5.0.6=pypi_0
  - future==0.18.2=pypi_0
  - apptools==4.5.0=pypi_0
  - pytorch==1.10.2=py3.7_cuda11.3_cudnn8.2.0_0
  - six==1.14.0=pypi_0
  - send2trash==1.5.0=pypi_0
  - msgpack-python==0.5.6=pypi_0
  - envisage==4.9.2=pypi_0
  - jedi==0.17.0=pypi_0
  - miniball==1.1.0=pypi_0
  - pillow==7.1.2=pypi_0
  - werkzeug==2.1.1=pypi_0
  - backcall==0.1.0=pypi_0
  - vispy==0.6.4=pypi_0
  - packaging==20.3=pypi_0
  - ipython==7.14.0=pypi_0
  - pyparsing==2.4.7=pypi_0
  - attrs==19.3.0=pypi_0
  - pyface==7.0.0=pypi_0
  - testpath==0.4.4=pypi_0
  - gnureadline==8.0.0=pypi_0
  - umsgpack==0.1.0=pypi_0
  - ptyprocess==0.6.0=pypi_0
  - tensorboard-plugin-wit==1.8.1=pypi_0
  - jsonschema==3.2.0=pypi_0
  - markupsafe==1.1.1=pypi_0
  - notebook==6.0.3=pypi_0
  - threadpoolctl==3.1.0=pypi_0
  - pyqt5-sip==12.7.2=pypi_0
  - requests-oauthlib==1.3.1=pypi_0
  - pyviz3d==0.2.28=pypi_0
  - opencv-python==4.5.5.64=pypi_0
  - pytorch-mutex==1.0=cuda
  - widgetsnbextension==3.5.1=pypi_0
  - pyyaml==5.3.1=pypi_0
  - webencodings==0.5.1=pypi_0
  - torch==1.5.0=pypi_0
  - torchaudio==0.10.2=py37_cu113
  - pycocotools==2.0.4=pypi_0
  - python-dateutil==2.8.1=pypi_0
  - pyopengl==3.1.5=pypi_0
  - torchvision==0.6.0=pypi_0
  - pygments==2.6.1=pypi_0
  - ipywidgets==7.5.1=pypi_0
  - nvidia-ml-py3==7.352.0=pypi_0
  - oauthlib==3.2.0=pypi_0
  - cycler==0.10.0=pypi_0
  - markdown==3.3.6=pypi_0
  - matplotlib==3.2.1=pypi_0
  - shapely==1.8.1.post1=pypi_0
  - cachetools==5.0.0=pypi_0
  - filterpy==1.4.5=pypi_0
  - pickleshare==0.7.5=pypi_0
  - tensorboard==2.8.0=pypi_0
  - pyrsistent==0.16.0=pypi_0
  - imgui==1.1.0=pypi_0
  - freetype-py==2.1.0.post1=pypi_0
  - jupyter-client==6.1.3=pypi_0
  - wcwidth==0.1.9=pypi_0
  - absl-py==1.0.0=pypi_0
  - google-auth-oauthlib==0.4.6=pypi_0
  - prometheus-client==0.7.1=pypi_0
  - termcolor==1.1.0=pypi_0
  - tqdm==4.48.2=pypi_0
  - pyquaternion==0.9.9=pypi_0
  - pyasn1==0.4.8=pypi_0

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

I don't know how to solve this problem, I struggled by following this "solution", but it also failed with following output:

The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - defaults/linux-64::libgcc-ng==9.3.0=h5101ec6_17
  - defaults/linux-64::libwebp-base==1.2.2=h7f8727e_0
  - defaults/linux-64::libidn2==2.3.2=h7f8727e_0
  - defaults/noarch::pyopenssl==22.0.0=pyhd3eb1b0_0
  - defaults/linux-64::gmp==6.2.1=h2531618_2
  - defaults/linux-64::lcms2==2.12=h3be6417_0
  - defaults/noarch::requests==2.27.1=pyhd3eb1b0_0
  - defaults/linux-64::numpy==1.21.5=py37he7a7128_2
  - defaults/linux-64::libiconv==1.15=h63c8f33_5
  - defaults/linux-64::libffi==3.3=he6710b0_2
  - defaults/linux-64::sqlite==3.38.5=hc218d9a_0
  - defaults/linux-64::giflib==5.2.1=h7b6447c_0
  - defaults/linux-64::bzip2==1.0.8=h7b6447c_0
  - defaults/linux-64::openh264==2.1.1=h4ff587b_0
  - defaults/linux-64::mkl_fft==1.3.1=py37hd3c417c_0
  - defaults/noarch::charset-normalizer==2.0.4=pyhd3eb1b0_0
  - defaults/linux-64::xz==5.2.5=h7f8727e_1
  - defaults/linux-64::ninja==1.10.2=py37hd09550d_3
  - defaults/linux-64::zstd==1.4.9=haebb681_0
  - defaults/linux-64::cudatoolkit==11.3.1=h2bc3f7f_2
  - defaults/linux-64::libstdcxx-ng==9.3.0=hd4cf53a_17
  - defaults/linux-64::pysocks==1.7.1=py37_1
  - defaults/linux-64::mkl_random==1.2.2=py37h51133e4_0
  - defaults/linux-64::gnutls==3.6.15=he1e5248_0
  - conda-forge/linux-64::python_abi==3.7=2_cp37m
  - defaults/linux-64::certifi==2022.9.24=py37h06a4308_0
  - defaults/noarch::pycparser==2.21=pyhd3eb1b0_0
  - defaults/linux-64::libtiff==4.2.0=h85742a9_0
  - defaults/linux-64::brotlipy==0.7.0=py37h27cfd23_1003
  - defaults/noarch::typing_extensions==4.1.1=pyh06a4308_0
  - defaults/linux-64::cffi==1.15.0=py37hd667e15_1
  - defaults/linux-64::lame==3.100=h7b6447c_0
  - defaults/linux-64::pip==22.2.2=py37h06a4308_0
  - defaults/linux-64::ncurses==6.3=h7f8727e_2
  - defaults/linux-64::libunistring==0.9.10=h27cfd23_0
  - defaults/noarch::wheel==0.37.1=pyhd3eb1b0_0
  - defaults/linux-64::lz4-c==1.9.3=h295c915_1
  - defaults/linux-64::openssl==1.1.1q=h7f8727e_0
  - conda-forge/noarch::olefile==0.46=pyh9f0ad1d_1
  - defaults/linux-64::cryptography==36.0.0=py37h9ce1e76_0
  - defaults/linux-64::freetype==2.11.0=h70c0345_0
  - defaults/linux-64::libpng==1.6.37=hbc83047_0
  - defaults/linux-64::python==3.7.13=h12debd9_0
  - defaults/linux-64::tk==8.6.12=h1ccaba5_0
  - defaults/linux-64::numpy-base==1.21.5=py37hf524024_2
  - defaults/linux-64::libuv==1.40.0=h7b6447c_0
  - conda-forge/linux-64::openjpeg==2.4.0=hb52868f_1
  - defaults/linux-64::mkl-service==2.4.0=py37h7f8727e_0
  - defaults/linux-64::setuptools==65.5.0=py37h06a4308_0
  - defaults/noarch::idna==3.3=pyhd3eb1b0_0
  - defaults/linux-64::urllib3==1.26.12=py37h06a4308_0
  - defaults/linux-64::nettle==3.7.3=hbbd107a_1
  - defaults/linux-64::libtasn1==4.16.0=h27cfd23_0
  - defaults/linux-64::zlib==1.2.12=h7f8727e_2
  - defaults/linux-64::jpeg==9b=h024ee3a_2
  - defaults/linux-64::readline==8.1.2=h7f8727e_1
  - defaults/noarch::six==1.16.0=pyhd3eb1b0_1                                                                                                        done

I'm really hope that I can get your help!

Thanks again!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.