cvg / deeplsd Goto Github PK

View Code? Open in Web Editor NEW

434.0 434.0 58.0 7.49 MB

Implementation of the paper "DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients"

License: MIT License

Python 19.36% Shell 0.09% CMake 0.11% C++ 2.05% Jupyter Notebook 78.39%

deeplsd's People

Contributors

Stargazers

Watchers

deeplsd's Issues

Does DeepLSD assume the coordinates of the center of the upper left pixel to be (0, 0) or (0.5, 0.5)?

Hi, thanks for the great work!
Does DeepLSD assume the coordinates of the center of the upper left pixel to be (0, 0) or (0.5, 0.5)?

third party packages failed to build

System:

ubuntu 20.04
python 3.8 / 3.9 / 3.6

Steps Taken:

git clone including recursive (as the readme suggests)
install ceres
bash install.sh
build_failure_log.txt
- pip requirement installed sucssesfuly
- progressive-x lib failed to build due to

fatal error: Python.h: No such file or directory

Could not find a package configuration file provided by "glog" with any of
the following names:
build_failure_log.txt

glogConfig.cmake
glog-config.cmake

more errors

Additional information

tried to build with a terminal sourced to basrhc (normally) and via venv as well.
Fixed the first error by changing the path of the python libs in the cmakelist.txt but still error accured

log attached

Can this be applied in gluestick?

Hello, wondeful work !

Your research in Gluestick says learned line detectors struggle to generalize power to other contexts, tasks, or types of images, so you choose LSD. I am considering that have you ever consider using this half-learned & half-manual DeepLSD as the line detector component of Gluestick and what are the reasons for not using it?

Looking forward to your reply very much！

A question about training on custom dataset

Hi, I am trying to train this model on some data I've collected online to test its training capabilities. I have the images annotated with lines and they are saved in a txt format like what Yolov8 uses. Is there an example of how I should transform this data to input as training for this model?
Is there any way to train the model to detect only some particular lines instead of all of them?

Memory leak in pytlsd.

I am trying to train your network on some other dataset. However, when generating groundtruth with LSD, I found that there's a heavy memory leak in pytlsd. My PC has 32G ram, and my PC will OOM in several minutes when generating groundtruth. Shown in image:

I found that your pytlsd code is forked from https://github.com/iago-suarez/pytlsd, and I substitute your version with iago-suarez's version, and there is no memory leak anymore. I think it might be useful by merging iago-suarez's repository into yours.

No such file or directory

No such file or directory: '../weights/deeplsd_md.tar'

what is the way to estimate homography with Line Feature in your job

Thanks a lot for your wonderful job!
I notice that it has been mentioned in paper that the homography is estimated like the way of Super Point, while point is different from line.
Could you tell me how exactly the homography is estimated in your job?
Does the end point of matched line regard as the same point feature so that the homography estimation for point could work?

fatal: Could not read from remote repository. “git clone --recurse-submodules [email protected]:cvg/DeepLSD.git” can’t work

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Question about angle field’s loss function

Hello, thank you for your work.
I tried drawing the plot of the angle field’s loss function with a fixed groundtruth A and found the following problem. I am wondering if I have misunderstood something in the paper.

third party cant complie

docker environment:gcc 12, cuda 11, python3.10, third party/progress-x cant make pass,and afm-lib also has problem;can u provide a docker ?

train run fail

DeepLSD/deeplsd/models/deeplsd.py", line 17, in
from line_refinement import line_optim
ImportError: /lib/libgdal.so.30: undefined symbol: GEOSMakeValidWithParams_r

how to solve this problem ,thanks!

What is the training loss value of deeplsd training to the best state?

When training the wireframe dataset, the learning rate drops slowly under the default parameters.
In sharpen mode, the training loss I finally achieved is 0.59, and in non-Sharpen mode, the training loss was reduced to 0.38 .
I am curious about what value can the loss of the model converge to in the end during training?

Quickstart install failure - ERROR: Failed building wheel for pytlsd

Following your readme

git clone --recurse-submodules https://github.com/cvg/DeepLSD.git

and

bash quickstart_install.sh

getting the following logs on Mac M2, in virtual environment:

Building wheels for collected packages: pytlsd
  Building wheel for pytlsd (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for pytlsd (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [156 lines of output]
      running bdist_wheel
      running build
      running build_ext
      CMake Deprecation Warning at pybind11/CMakeLists.txt:8 (cmake_minimum_required):
        Compatibility with CMake < 3.5 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      -- pybind11 v2.6.2
      CMake Warning (dev) at pybind11/tools/FindPythonLibsNew.cmake:98 (find_package):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning.
      
      Call Stack (most recent call first):
        pybind11/tools/pybind11Tools.cmake:45 (find_package)
        pybind11/tools/pybind11Common.cmake:201 (include)
        pybind11/CMakeLists.txt:188 (include)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      --     OpenCV_DIR: /opt/homebrew/lib/cmake/opencv4
      --     OpenCV_INCLUDE_DIRS: /opt/homebrew/Cellar/opencv/4.8.1_1/include/opencv4
      --     OpenCV_LIBS: opencv_calib3dopencv_coreopencv_dnnopencv_features2dopencv_flannopencv_gapiopencv_highguiopencv_imgcodecsopencv_imgprocopencv_mlopencv_objdetectopencv_photoopencv_stitchingopencv_videoopencv_videoioopencv_alphamatopencv_arucoopencv_bgsegmopencv_bioinspiredopencv_ccalibopencv_datasetsopencv_dnn_objdetectopencv_dnn_superresopencv_dpmopencv_faceopencv_freetypeopencv_fuzzyopencv_hfsopencv_img_hashopencv_intensity_transformopencv_line_descriptoropencv_mccopencv_optflowopencv_phase_unwrappingopencv_plotopencv_qualityopencv_rapidopencv_regopencv_rgbdopencv_saliencyopencv_sfmopencv_shapeopencv_stereoopencv_structured_lightopencv_superresopencv_surface_matchingopencv_textopencv_trackingopencv_videostabopencv_vizopencv_wechat_qrcodeopencv_xfeatures2dopencv_ximgprocopencv_xobjdetectopencv_xphoto
      --     OpenCV_DIR: /opt/homebrew/lib/cmake/opencv4
      --     OpenCV_INCLUDE_DIRS: /opt/homebrew/Cellar/opencv/4.8.1_1/include/opencv4
      --     OpenCV_LIBS: opencv_calib3dopencv_coreopencv_dnnopencv_features2dopencv_flannopencv_gapiopencv_highguiopencv_imgcodecsopencv_imgprocopencv_mlopencv_objdetectopencv_photoopencv_stitchingopencv_videoopencv_videoioopencv_alphamatopencv_arucoopencv_bgsegmopencv_bioinspiredopencv_ccalibopencv_datasetsopencv_dnn_objdetectopencv_dnn_superresopencv_dpmopencv_faceopencv_freetypeopencv_fuzzyopencv_hfsopencv_img_hashopencv_intensity_transformopencv_line_descriptoropencv_mccopencv_optflowopencv_phase_unwrappingopencv_plotopencv_qualityopencv_rapidopencv_regopencv_rgbdopencv_saliencyopencv_sfmopencv_shapeopencv_stereoopencv_structured_lightopencv_superresopencv_surface_matchingopencv_textopencv_trackingopencv_videostabopencv_vizopencv_wechat_qrcodeopencv_xfeatures2dopencv_ximgprocopencv_xobjdetectopencv_xphotoopencv_calib3dopencv_coreopencv_dnnopencv_features2dopencv_flannopencv_gapiopencv_highguiopencv_imgcodecsopencv_imgprocopencv_mlopencv_objdetectopencv_photoopencv_stitchingopencv_videoopencv_videoioopencv_alphamatopencv_arucoopencv_bgsegmopencv_bioinspiredopencv_ccalibopencv_datasetsopencv_dnn_objdetectopencv_dnn_superresopencv_dpmopencv_faceopencv_freetypeopencv_fuzzyopencv_hfsopencv_img_hashopencv_intensity_transformopencv_line_descriptoropencv_mccopencv_optflowopencv_phase_unwrappingopencv_plotopencv_qualityopencv_rapidopencv_regopencv_rgbdopencv_saliencyopencv_sfmopencv_shapeopencv_stereoopencv_structured_lightopencv_superresopencv_surface_matchingopencv_textopencv_trackingopencv_videostabopencv_vizopencv_wechat_qrcodeopencv_xfeatures2dopencv_ximgprocopencv_xobjdetectopencv_xphoto
      -- Configuring done (0.1s)
      -- Generating done (0.0s)
      -- Build files have been written to: /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/build/temp.macosx-14-arm64-cpython-311
      [ 16%] Linking CXX static library libtlsd.a
      [ 33%] Built target tlsd
      [ 50%] Building CXX object CMakeFiles/pytlsd.dir/src/PYAPI.cpp.o
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:45:
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/attr.h:13:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/cast.h:446:36: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
                      "  " + handle(frame->f_code->co_filename).cast<std::string>() +
                                         ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:45:
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/attr.h:13:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/cast.h:448:29: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
                      handle(frame->f_code->co_name).cast<std::string>() + "\n";
                                  ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:45:
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/attr.h:13:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/cast.h:449:26: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
                  frame = frame->f_back;
                               ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:2291:49: error: no member named 'frame' in '_ts'
          PyFrameObject *frame = PyThreadState_Get()->frame;
                                 ~~~~~~~~~~~~~~~~~~~  ^
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:2292:41: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
          if (frame && (std::string) str(frame->f_code->co_name) == name &&
                                              ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:2293:14: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
              frame->f_code->co_argcount > 0) {
                   ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:2296:18: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
                  frame->f_locals, PyTuple_GET_ITEM(frame->f_code->co_varnames, 0));
                       ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      In file included from /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/src/PYAPI.cpp:1:
      /Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/third_party/pytlsd/pybind11/include/pybind11/pybind11.h:2296:52: error: member access into incomplete type 'PyFrameObject' (aka '_frame')
                  frame->f_locals, PyTuple_GET_ITEM(frame->f_code->co_varnames, 0));
                                                         ^
      /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of '_frame'
      typedef struct _frame PyFrameObject;
                     ^
      8 errors generated.
      make[2]: *** [CMakeFiles/pytlsd.dir/src/PYAPI.cpp.o] Error 1
      make[1]: *** [CMakeFiles/pytlsd.dir/all] Error 2
      make: *** [all] Error 2
      Traceback (most recent call last):
        File "/Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/deeplsd_venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/deeplsd_venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/ouz/Desktop/deepLSD/DeepLSD/DeepLSD/deeplsd_venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 434, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 419, in _build_with_temp_dir
          self.run_setup()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 65, in <module>
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/private/var/folders/rp/d8kwybr118xd68k4qcxw93vw0000gn/T/pip-build-env-aq86ggs1/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "<string>", line 58, in build_extension
        File "/opt/homebrew/Cellar/[email protected]/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '--build', '.']' returned non-zero exit status 2.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pytlsd
Failed to build pytlsd

cmake --version returns cmake version 3.27.4 so that is not an issue.

Any tips would be appreciated.

A question about Math

Firstly I am very interested in your work and I think it do broaden my horizon.
After running the demo, I have a question: after line refinement, is there any mathematical expressions with the detected lines.
Because I want to do Visual Line Matching baesd on your nice work.
SINCERELY!
Wish for your answer.

ImportError: cannot import name 'line_optim' from 'line_refinement' (unknown location)

Has anyone encountered a similar problem?
I tried to install on mac os and windows. Here is the traceback from jupyter:

ImportError
Traceback (most recent call last)
Input In [1], in <cell line: 11>()
8 import h5py
10 from deeplsd.utils.tensor import batch_to_device
---> 11 from deeplsd.models.deeplsd import DeepLSD
12 from deeplsd.geometry.viz_2d import plot_images, plot_lines

File ~/Documents/DeepLSD/deeplsd/models/deeplsd.py:17, in
15 from ..utils.tensor import preprocess_angle
16 from pytlsd import lsd
---> 17 from line_refinement import line_optim
20 class DeepLSD(BaseModel):
21 default_conf = {
22 'tiny': False,
23 'sharpen': True,
(...)
43 },
44 }

ImportError: cannot import name 'line_optim' from 'line_refinement' (unknown location)

There was a lot of errors in the process of execution install.sh.
If necessary, I will also leave the entire output when executing install.sh

pytlsd submodule problem

You have some problem with pytlsd, it's not working.

Inference on Point Clouds

Would there be a possibility to make this work on point cloud ? with inputs as x, y, z or even retrain it so that it works on point clouds to detect lines on it?

I tried to rasterize the point cloud data and try to detect lines, but that doesn't work that well

Training process: Questions about file paths and reading

Sorry to bother you, stuck with the following question, please help, thank you very much.
The Wireframe dataset is used.
deeplsd/scripts/train.py About line 116：
train_loader = dataset.get_data_loader('train')
val_loader = dataset.get_data_loader('val')
Is 'train' here the path to the training images of the dataset? So 'val' is the generated truth path?

error：

Traceback (most recent call last):
  File "/home/lxq/miniconda3/envs/dl-line/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/lxq/miniconda3/envs/dl-line/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dataset/catkin_multi/V_or_L_or_I/line、point/DL-lines/DeepLSD/deeplsd/scripts/train.py", line 252, in <module>
    training(conf, output_dir, args)
  File "/home/dataset/catkin_multi/V_or_L_or_I/line、point/DL-lines/DeepLSD/deeplsd/scripts/train.py", line 116, in training
    train_loader = dataset.get_data_loader('train')
  File "/home/dataset/catkin_multi/V_or_L_or_I/line、point/DL-lines/DeepLSD/deeplsd/datasets/wireframe_ha.py", line 109, in get_data_loader
    return DataLoader(self.get_dataset(split), batch_size=batch_size,
  File "/home/lxq/miniconda3/envs/dl-line/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 344, in __init__
    sampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]
  File "/home/lxq/miniconda3/envs/dl-line/lib/python3.8/site-packages/torch/utils/data/sampler.py", line 107, in __init__
    raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0

[Question] Inferences that ignore the short line

I want to detect only long lines in the image.
Is it possible to adjust the results by changing the value of "conf" in demo_line_detection.ipynb?

The pre-trained models cannot be downloaded！

Angle Field Visualization Code

Hi,

How can I get the angle field visualization to look similar to the bottom left image in your README?

I have tried using get_flow_vis() by passing out['df'] and out['line_level'] as inputs, but I get the following image.

I understand that the lines and points require additional code, but the flow part also doesn't look similar to the README, should I use a different code? Or, is line_level, not the correct variable?

Thanks!

How to optimise DeepLSD with ortho-image ?

Hi guys!
Thank you for your work !
I work with image from scan 3D like cut or front of a building

Is it possible to work and optimize this type of image with no perspective or angle ?
Do I have to change some parameters ?
Thank you !

ImportError: cannot import name 'line_optim' from 'line_refinement' (unknown location)

there is no such a file named 'line_optim' in the file'line_refinement',
how to import line_optim from line_refinement in the file deeplsd/models/deeplsd.py?

TypeError: lsd(): incompatible function arguments. The following argument types are supported:

Has anyone encountered a similar problem?
I tried to install on ubuntu. Here is the traceback from jupyter:
TypeError: lsd(): incompatible function arguments. The following argument types are supported:

(img: numpy.ndarray[numpy.float64], scale: float = 0.8, sigma_scale: float = 0.6, density_th: float = 0.0, gradnorm: numpy.ndarray[numpy.float64] = array([], dtype=float64), gradangle: numpy.ndarray[numpy.float64] = array([], dtype=float64)) -> numpy.ndarray[numpy.float32]

Invoked with: array([[100., 102., 103., ..., 111., 111., 113.],
[101., 102., 103., ..., 111., 112., 113.],
[104., 104., 104., ..., 112., 112., 113.],
...,
[ 93., 92., 93., ..., 243., 241., 240.],
[103., 103., 101., ..., 244., 244., 242.],
[ 78., 76., 76., ..., 241., 241., 244.]]); kwargs: scale=1.0, gradnorm=array([[2.57419753, 2.52489138, 2.18968916, ..., 2.68456316, 2.77042079,
2.80449152],
[2.61466122, 2.35479498, 1.70597696, ..., 2.60563469, 2.84062576,
2.75185728],
[2.58136368, 2.45527935, 2.25546384, ..., 2.19458222, 2.5106287 ,
2.68859076],
...,
[3.71072507, 3.71005678, 3.70416141, ..., 2.97271562, 2.40432501,
1.80609179],
[3.29464531, 3.19876194, 3.17765403, ..., 2.25287461, 1.85981727,
1.52646542],
[2.83806252, 2.7701757 , 2.65922499, ..., 1.6292944 , 1.41910815,
1.35227203]]), gradangle=array([[-1024. , -1024. , -1024. , ...,
-1024. , -1024. , -1024. ],
[-1024. , -1024. , -1024. , ...,
-1024. , -1024. , -1024. ],
[-1024. , -1024. , -1024. , ...,
-1024. , -1024. , -1024. ],
...,
[-1024. , 5.99458066, 6.0488495 , ...,
-1024. , -1024. , -1024. ],
[-1024. , 6.04460517, 6.05717483, ...,
-1024. , -1024. , -1024. ],
[-1024. , -1024. , -1024. , ...,
-1024. , -1024. , -1024. ]]), grad_nfa=True

How to create my own dataloader

Hello Guys !
Thank you for your work,

I have a question, how to create my own DataLoader based only with lines in a JSON (x1,x2,y1,y2) for example. What DeepLSD need to be trainable ?

Thank you in advance

Can use sAP to evaluate this method, i know that output does not contain confident socre of predicted line.

executing in windows

Does your code also run in Windows?

cannot import name 'line_optim' from 'line_refinement' (unknown location)

homography_adaptation_df.py crashes on num_jobs >1

Hi,
I am running this script with joblib==1.2.0, and the script crashes on start with the following output. Changing backend from loky to e.g. multiprocessing lets the script run seemingly normal. (Edited for brevity)

Not sure if this is an issue with joblib or an unfortunate combination of library-versions? Can you possibly share a pip freeze or similar of known good combinations?

joblib.externals.loky.process_executor._RemoteTraceback: 
[...]
TypeError: cannot pickle 'PyCapsule' object
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...]
  File "/home/asbe/projects/SWIFTER/DeepLSD/deeplsd/scripts/homography_adaptation_df.py", line 178, in <module>
    export_ha(args.images_list, args.output_folder, args.num_H,
  File "/home/asbe/projects/SWIFTER/DeepLSD/deeplsd/scripts/homography_adaptation_df.py", line 159, in export_ha
    Parallel(n_jobs=n_jobs, backend='loky')(delayed(process_image)(
  [...]
  File "/home/asbe/anaconda3/envs/deeplsd/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
_pickle.PicklingError: Could not pickle the task to send it to the workers.

A small bug when running evaluate_line_detection.py

Hi,
When running evaluate_line_detection.py, an IndexError occurs due to match_lines().
The my_matches array of type float, because the pytlbd.lbd_matching_multiscale() also returns descriptor distances.
So would raise IndexError: arrays used as indices must be of integer (or boolean) type

To fix this issue, replace line 66 with:

pred_matches[my_matches[:, 0].astype(np.int32)] = my_matches[:, 1].astype(np.int32)

Finally, Your work has been incredibly helpful to me.

PINTO model zoo ONNX models for DeepLSD work. How to get from heatmaps to line segments?

PINTO_model_zoo converted your models to onnx for inference-time optimization and they indeed work out of the box.

Several models are provided:

deeplsd_md_180x320.onnx		deeplsd_wireframe_225x400.onnx
deeplsd_md_225x400.onnx		deeplsd_wireframe_240x320.onnx
deeplsd_md_240x320.onnx		deeplsd_wireframe_270x480.onnx
deeplsd_md_270x480.onnx		deeplsd_wireframe_360x640.onnx
deeplsd_md_360x640.onnx		deeplsd_wireframe_450x800.onnx
deeplsd_md_450x800.onnx		deeplsd_wireframe_480x640.onnx
deeplsd_md_480x640.onnx		deeplsd_wireframe_540x960.onnx
deeplsd_md_540x960.onnx		deeplsd_wireframe_720x1280.onnx
deeplsd_md_720x1280.onnx	        deeplsd_wireframe_HxW.onnx
deeplsd_md_HxW.onnx                  deeplsd_wireframe_180x320.onnx

and inference can be made with simple:

lsd_model = InferenceSession(
    "deeplsd_md_HxW.onnx",
    providers=["CPUExecutionProvider"],
)
input_name = lsd_model.get_inputs()[0].name
output = lsd_model.run(None, {input_name: image})

output consists of 3 numpy arrays that are the same shape as input image. I believe these are the probability heatmaps/fields.

Is there a straightforward way to infer the line segments using these 3 numpy arrays? If so, which function(s) I should be looking at in the codebase?

omegaconf.errors.ConfigAttributeError: Missing key optimize

Hello, thank you for your work.
When exporting line detections on York Urban, I met the error of missing key optimize. I didn't modify anything of the configs and everything went well with other datasets.
Here is the output in terminal.

cvg / deeplsd Goto Github PK

deeplsd's People

Contributors

Stargazers

Watchers

Forkers

deeplsd's Issues

System:

Steps Taken:

Additional information

Recommend Projects

Recommend Topics

Recommend Org