zhenpeiyang / mvs2d Goto Github PK

View Code? Open in Web Editor NEW

123.0 123.0 13.0 44.32 MB

Python 98.63% Shell 1.37%

mvs2d's People

Contributors

Stargazers

Watchers

Forkers

peterzhousz hiyyg leejaeyong7 huangqx minlattnwe space25 yannnnnnnnnnnn daydreamer2023 ys-forks rteklewold nburgdorfer cindycxy xbillowy

mvs2d's Issues

Question about Bts*

Additionally, we use an asterisk sign ‘∗’ to denote an oracle version Bts∗, where we use the ground truth depth map to
factor out the global scale.

Is it means that the mono depth network's prediction may not have the same scale as the depth map and Bts* is to estimate some parameter (e.g. mean and std in (depth-mean)/std) to adjust its scale according to the gt depth map?

使用自己的数据进行重建

您好，想问下如何使用自己的数据进行重建，有提供相关的脚本吗

What is the frame selection(sampling) criterion of ScanNet dataset?

As you mentioned at the MVS2D paper,

ScanNet dataset sampled from original set by 86324 triple images for training and 666 triple images for testing.

I wanna know the criterion of frame sampling.

'tuple' object is not callable

Hi, there is a bug in your code that cause: 'tuple' object is not callable
training process is ok to run but always showing 'tuple' object is not callable

the log is:

Training
'tuple' object is not callable
'tuple' object is not callable
'tuple' object is not callable
/mypath/anaconda3/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:129: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call of lr_scheduler.step() before optimizer.step(). "
/mypath/anaconda3/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:154: UserWarning: The epoch parameter in scheduler.step() was not necessary and is being deprecated where possible. Please use scheduler.step() to step the scheduler. During the deprecation, if epoch is different from None, the closed form is used instead of the new chainable form, where available. Please open an issue if you are unable to replicate your use case: https://github.com/pytorch/pytorch/issues/new/choose.
warnings.warn(EPOCH_DEPRECATION_WARNING, UserWarning)
'tuple' object is not callable
/mypath/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
'tuple' object is not callable
'tuple' object is not callable
'tuple' object is not callable

RuntimeError: CUDA error: an illegal memory access was encountered

Hi guys, Thanks for your great work
Would you please let me know what is the following error about ?
I did some search but I couldnt find the solution
I would appresiate it if you could help
thanks

**Traceback (most recent call last):
File "demo.py", line 72, in
outputs = model(imgs[0], imgs[1:], proj_mats[0], proj_mats[1:], inv_K_pool)
File "/home/akarami/anaconda3/envs/mvs2d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, kwargs)
File "/home/akarami/mvs2d/networks/mvs2d.py", line 311, in forward
src_imgs,
File "/home/akarami/mvs2d/networks/mvs2d.py", line 230, in epipolar_fusion
k, proj_mask, grid = homo_warping(k, src_proj, ref_proj,depth_values)
File "/home/akarami/mvs2d/networks/module.py", line 72, in homo_warping
proj = torch.matmul(src_proj, torch.inverse(ref_proj))
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

waiting for the code!

In testing，I downloaded the eval data，but can't find the test dataset I need about DTU.

when is the code open to the public?

Thank you for the wonderful work.
I wonder when is the code implementation open to the public.

Saving color images

In the 'train.py' code, the input color images are resized and transposed by (1,2,0).

Why did you transposed the original input images when you save the "data" including original images if opt.mode=='test' and opt.save_prediction==True. ?

requirement.txt has two version of pyhocon and training issue

I deleted the 0.3.59 version

I changed the dtu.conf data_path to absolute path

then I ran sh ***/train.sh

I got the following traceback

Traceback (most recent call last):
  File "train.py", line 6, in <module>
    from tensorboardX import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/__init__.py", line 5, in <module>
    from .torchvis import TorchVis
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/torchvis.py", line 11, in <module>
    from .writer import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/writer.py", line 17, in <module>
    from .comet_utils import CometLogger
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/comet_utils.py", line 6, in <module>
    from .summary import _clean_tag
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/summary.py", line 13, in <module>
    from .proto.summary_pb2 import Summary
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/summary_pb2.py", line 16, in <module>
    from tensorboardX.proto import tensor_pb2 as tensorboardX_dot_proto_dot_tensor__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/tensor_pb2.py", line 16, in <module>
    from tensorboardX.proto import resource_handle_pb2 as tensorboardX_dot_proto_dot_resource__handle__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 42, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 561, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
  File "train.py", line 6, in <module>
    from tensorboardX import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/__init__.py", line 5, in <module>
    from .torchvis import TorchVis
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/torchvis.py", line 11, in <module>
    from .writer import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/writer.py", line 17, in <module>
    from .comet_utils import CometLogger
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/comet_utils.py", line 6, in <module>
    from .summary import _clean_tag
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/summary.py", line 13, in <module>
    from .proto.summary_pb2 import Summary
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/summary_pb2.py", line 16, in <module>
    from tensorboardX.proto import tensor_pb2 as tensorboardX_dot_proto_dot_tensor__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/tensor_pb2.py", line 16, in <module>
    from tensorboardX.proto import resource_handle_pb2 as tensorboardX_dot_proto_dot_resource__handle__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 42, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 561, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
  File "train.py", line 6, in <module>
    from tensorboardX import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/__init__.py", line 5, in <module>
    from .torchvis import TorchVis
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/torchvis.py", line 11, in <module>
    from .writer import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/writer.py", line 17, in <module>
    from .comet_utils import CometLogger
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/comet_utils.py", line 6, in <module>
    from .summary import _clean_tag
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/summary.py", line 13, in <module>
    from .proto.summary_pb2 import Summary
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/summary_pb2.py", line 16, in <module>
    from tensorboardX.proto import tensor_pb2 as tensorboardX_dot_proto_dot_tensor__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/tensor_pb2.py", line 16, in <module>
    from tensorboardX.proto import resource_handle_pb2 as tensorboardX_dot_proto_dot_resource__handle__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 42, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 561, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
  File "train.py", line 6, in <module>
    from tensorboardX import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/__init__.py", line 5, in <module>
    from .torchvis import TorchVis
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/torchvis.py", line 11, in <module>
    from .writer import SummaryWriter
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/writer.py", line 17, in <module>
    from .comet_utils import CometLogger
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/comet_utils.py", line 6, in <module>
    from .summary import _clean_tag
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/summary.py", line 13, in <module>
    from .proto.summary_pb2 import Summary
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/summary_pb2.py", line 16, in <module>
    from tensorboardX.proto import tensor_pb2 as tensorboardX_dot_proto_dot_tensor__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/tensor_pb2.py", line 16, in <module>
    from tensorboardX.proto import resource_handle_pb2 as tensorboardX_dot_proto_dot_resource__handle__pb2
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 42, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 561, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in <module>
    main()
  File "/home/kong/anaconda3/envs/mvs2d/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/home/kong/anaconda3/envs/mvs2d/bin/python', '-u', 'train.py', '--local_rank=3', '--model_name=config0', '--num_epochs=80', '--DECAY_STEP_LIST', '40', '70', '--cfg', './configs/dtu/release.conf']' returned non-zero exit status 1.

When will you release the code?

Convergence speed

hi I have some questions on convergence speed of MVS2D

the cost-volume based method (MVSNet, etc.) can see the outline of objects after 1 epoch or even several iters

however, MVS2D seems hard to convergence and the depthmap are very vague at the beginning.

does this due to the network design?
Could you explain about this phenomenon?

Question about generalization

Thanks for sharing of code.

But I wonder why you did not test your method on 7Scenes or Tanks&Temple.

Besides, I tried to test your method on 7Scenes, but the results are pretty poor.
Can you give me some suggestions?

My code is here: https://github.com/Yannnnnnnnnnnn/MVS2D.git at test_7scenes_long, as I use the split from Long.

My results is here:

7scenes:{'a1': 0.39494346839485406,
 'a2': 0.6459305167578225,
 'a3': 0.8022140821127047,
 'abs_diff': 0.5733749950019752,
 'abs_diff_median': 0.5094702287105953,
 'abs_rel': 0.32380217413691914,
 'log10': 0.17217029032900052,
 'rmse': 0.6894540686467114,
 'rmse_log': 0.47906266186605484,
 'sq_rel': 0.27828435772756005,
 'thre1': 0.23810647116173855,
 'thre3': 0.5398887457282459,
 'thre5': 0.8326691103244529}

Code Release

Hi,
Thanks for your great work! Now it is November~. Do you plan to open source code in the near future? Hoping it will come soon~

val set in DTU.py not consistent with those in patchmatch_fusion.py

There is a point in your repo that is confusing for newbies(for me), that the val set in DTU.py is not consistent with those in patchmatch_fusion.py

in DTU.py val it is

        data_set = [

            3, 5, 17, 21, 28, 35, 37, 38, 40, 43, 56, 59, 66, 67, 82, 86,
            106, 117
        ]

however in patchmatch_fusion.py / dtu_pyeval.py it is

scans = [

    1, 4, 9, 10, 11, 12, 13, 15, 23, 24, 29, 32, 33, 34, 48, 49, 62, 75,
    77, 110, 114, 118
]

hope you could fix it,
or at least this post may be helpful for users who want to reproduce your great work！

Waiting for releasing the official code

Hi,
Thanks for your great work! I'm wondering when will you release the code? Hoping it will come soon~

请问可以再发布一下预训练的权重文件吗

底下的文件链接已经不能下载了

The code is inconsistent with the paper

Thank you for your excellent work. However, I have a doubt. According to the statement of your paper, the depth assumption of SRC image is used to obtain the similarity score by projection transformation to ref image. However, in Code, the depth assumption is set in the Ref Image coordinate system to obtain 3D points, which are transformed to the SRC image coordinate system and sampled from the SRC image. Finally, the similarity score is obtained by dot product with the REF image. This is actually helpful to get the depth of the ref image, instead of SRC image. I don't understand why you make the depth assumption in the Ref image coordinate system and then project it.

Question about the GPUs used for evaluation FPS

Hi, thanks for your excellent jobs and readily code!

I'm confused about the GPUs used for evaluation FPS. In the original paper, it says in table 1 note : "We use a single Nvidia V100 GPU for measuring FPS", but in the supp section1.3: "We benchmark the FPS of each methods on the same machine with a Intel(R) Xeon(R) E5-2637 v4 @ 3.50GHz CPU and a GeForce GTX 1080 Ti GPU." But there is no FPS results in the supp.

So what kind of GPU you used for evaluation? Hope to hear from you. Thanks!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.