yaochih / gcvd-release Goto Github PK

Hi, thank you for the great work!
I've tried to reproduce the results you've published for TUM and 7 scenes datasets. However, the depth estimation metric (abs rel) values I obtained are significantly worse then the ones reported in the paper.
For 7_scenes/chess/seq-01 sequence I got abs rel = 0.83 with the command you provided in Readme:
python3 main.py test-dataset/chess/seq-01.mkv --name test --pose_graph. I tried using --post_filter as well, but it did not change the result significantly. You reported abs rel = 0.124 on 7 scenes, so I suppose you had << 83% relative error on chess/seq-01.

For TUM/freiburg1_desk I got 0.2718 relative error, while you reported 0.0940 for this sequence.

I tried to find the reason of the high depth estimation errors. It seems like GCVD depth estimates often have wrong scale. If I adjust scales to the estimated depth maps to match ground truth depth (individual constant scale for each frame), I can get 0.0897 relative error for TUM/freiburg1_desk sequence, which seems to be a reasonable result, therfore correcting the wrong scale could potentially lead to the results reported in the paper.
Here is an example of a frame from TUM/freiburg1_desk where GCVD output has high relative error (0.507). In the original TUM dataset its name is rgbd_dataset_freiburg1_desk/rgb/1305031464.759740.png

As you can see, GCVD output has a reasonable structure, but the scale is wrong, which affects abs rel metric.

I used the files that main.py script generated in outputs/test/depths/final directory (or outputs/test/depths/filtered for runs with --post_filter option) as the final output of GCVD algorithm. Is it right, or should I additionally scale these depth maps somehow?
Should the command python3 main.py video.mkv --name test --pose_graph --post_filter reproduce the result reported in the paper or should I use some additional options? Are there any other tips for reproducing these results?

I got a problem when I run testing code.
The platform is ubuntu22.04 cuda11.1 pytorch1.8.0 python3.8
I guess this is due to numpy version?

=>Reconstruct keyframes
  100%|████████████████████| 1200/1200 [02:50<00:00Traceback (most recent call last):
  File "main.py", line 246, in <module>
    vd.process_keyframes()
  File "main.py", line 101, in process_keyframes
    np.save(self.seq_io.root/'keyframe_groups.npy', keyframe_groups)
  File "<__array_function__ internals>", line 200, in save
  File "/home/lwy/.conda/envs/gcvd/lib/python3.8/site-packages/numpy/lib/npyio.py", line 521, in save
    arr = np.asanyarray(arr)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.
  100%|████████████████████| 1200/1200 [02:52<00:00

Hi, thanks for the excellent work. However, during test as proposed "python3 main.py test-dataset/chess/seq-01.mkv --name test --pose_graph", I encountered a runtime error. It seems untraceable even when I set detect_anomally to "True". Has anyone encountered the same problem? Here are the outputs in the shell.

=>Load models
  Loading weights: weights/models/midas_v21-f6b98070.pt
=>Reconstruct keyframes
    0%|                    | 0/1200 [00:00<?Traceback (most recent call last):
  File "main.py", line 249, in <module>
    vd.process_keyframes()
  File "main.py", line 82, in process_keyframes
    keyframes = self.sfm.reconstruct_images(
  File "/home/pc/GCVD-release/sfm.py", line 113, in reconstruct_images
    loss['full'].backward()
  File "/home/pc/.conda/envs/GCVD/lib/python3.8/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/pc/.conda/envs/GCVD/lib/python3.8/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [41, 4, 4]], which is output 0 of AsStridedBackward0, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
    0%|                    | 0/1200 [00:01<?

Following the install instructions, when I'm installing g2opy I get the following error
python setup.py install

error: Multiple top-level packages discovered in a flat-layout: ['g2o', 'lib', 'script', 'contrib', 'EXTERNAL', 'cmake_modules'].

To avoid accidental inclusion of unwanted files or directories,
setuptools will not proceed with this build.

If you are trying to create a single distribution with multiple packages
on purpose, you should not rely on automatic discovery.
Instead, consider the following options:

1. set up custom discovery (`find` directive with `include` or `exclude`)
2. use a `src-layout`
3. explicitly set `py_modules` or `packages` with a list of names

To find more information, look for "package discovery" on setuptools docs.