lzhnb / gs-ir Goto Github PK

Hi!

There is no backward process for depthToNormal, is this part of the gradient calculated correctly?

Hello, I tried reproducing the bicycle scene, I ran the training, evaluation and relighting scripts but the final result doesn't look right. Any ideas what the issue could be ?

Envmap used (but in HDR format not PNG) :

Thanks :)

Hi,
thanks for your wonderful work. I tried the garden scene and it seems that it generally appears to bright. Any idea why this is the case?

Sorry to bother you again. I encountered some confusion in the process of replication. I would like to ask about the generation strategy of the occlusion volumes mentioned in the paper. It seems that the paper does not detail how the occlusion volumes are generated, such as how many there are, whether they are uniformly distributed or generated close to the surface of the object...

Shiny Blender Dataset contains some reflective objects and is frequently used for evaluating inverse rendering. If you have conducted relative tests, could you kindly provide the corresponding configurations or some results? Your assistance would be greatly appreciated.

I am trying to replicate the normal recovery step in GS-IR. When I pass the gradient of the normal to alpha in backward.cu, it produces somewhat strange results. It seems that not considering the gradient of the normal for alpha yields better outcomes. It leads to more severe floaters.
By the way, when will the GS-IR code be released? Thanks.

See figures below:

不知道为啥，可以检查一下代码，我直接安装官方的submodule是可以行的通的

Thank you for your work first!

When the work progressed to the second stage, the buiding of the cpp_extension module failed.

# Stage2 (Decomposition Stage)
python train.py \
-m outputs/lego/ \
-s datasets/TensoIR/lego/ \
--start_checkpoint outputs/lego/chkpnt30000.pth \
--iterations 35000 \
--eval \
--gamma \
--indirect

Can I get some advice or further information about the experimental environment of GS-IR?

Errors as follows,

Traceback (most recent call last):
  File "train.py", line 789, in <module>
    indirect=args.indirect,
  File "train.py", line 309, in training
    cubemap.build_mips() # build mip for environment light
  File "./GS-IR/pbr/light.py", line 101, in build_mips
    self.diffuse = diffuse_cubemap(self.specular[-1])
  File "./GS-IR/pbr/renderutils/ops.py", line 408, in diffuse_cubemap
    out = _diffuse_cubemap_func.apply(cubemap)
  File "./GS-IR/pbr/renderutils/ops.py", line 394, in forward
    out = _get_plugin().diffuse_cubemap_fwd(cubemap)
  File "./GS-IR/pbr/renderutils/ops.py", line 79, in _get_plugin
    extra_cuda_cflags=opts, extra_ldflags=ldflags, with_cuda=True, verbose=True)
  File "./anaconda3/envs/gsir/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1214, in load
    keep_intermediates=keep_intermediates)
  File "./anaconda3/envs/gsir/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1435, in _jit_compile
    is_standalone=is_standalone)
  File "./anaconda3/envs/gsir/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1540, in _write_ninja_file_and_build_library
    error_prefix=f"Error building extension '{name}'")
  File "./anaconda3/envs/gsir/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'renderutils_plugin'

Thank you for your great work!
I just curious about how to precompute brdf_256_256.bin? Is there any public code available? Thanks in advance.

Some details in the article cannot be reproduced. Will the code be made available?

Hi, Good job! and thanks for your release your code.

But when i reproduce these scene step by step, the decompose result is not enough good like paper. For example,

lego:

armadillo:

And I also want to ask how to set the occlusion threshold on different scenes.

I'm very apperiate it If you can reply!

best,
regards

Thanks for such a great work.

Can you please provide an estimated time for releasing the code?

Hi,
Thanks for your nice work! I noticed some inconsistencies between the main paper's figures and the supplemental figures. For example, The Albedo of the Lego truck in Figure 5 and Figure 10 are different (Highlighted by a red frame). Could you explain why they have a different reflectance?

Really looking forward to it.

Hello

Thank you for your great work.

When relighting, is it possible to cut out and render only a part of the scene?
For example, in the garden scene, I'd like to cut out and render only around the central table.

Thank you.

亲爱的作者，原谅我的菜，我想问一下为什么论文里面没有提到表面重建也就是几何重建的方法呢？

Hi!

Thanks for releasing the code for your interesting paper. I'm trying to get the code working on Ubuntu 22.04 LTS; however, I faced some issues that I solved and there are some that still exist.

First, by simply installing from the conda env file, when we want to run the command cd gs-ir && python setup.py develop && cd .. we get to this issue:

The detected CUDA version (12.3) mismatches the version that was used to compile
PyTorch (11.6). Please make sure to use the same CUDA versions.

I simply fixed it with the following command:
conda install -c "nvidia/label/cuda-11.6.0" cuda-toolkit

After that in the baking step, I have the following issue:

File "baking.py", line 348, in <module>                                                                                                              
    boundary_mode="cube",                                                                                                                              
  File "/home/samp8/.conda/envs/gsir/lib/python3.7/site-packages/nvdiffrast/torch/ops.py", line 615, in texture                                        
    return _texture_func.apply(filter_mode, tex, uv, filter_mode_enum, boundary_mode_enum)                                                             
  File "/home/samp8/.conda/envs/gsir/lib/python3.7/site-packages/nvdiffrast/torch/ops.py", line 504, in forward                                        
    out = _get_plugin().texture_fwd(tex, uv, filter_mode_enum, boundary_mode_enum)                                                                     
  File "/home/samp8/.conda/envs/gsir/lib/python3.7/site-packages/nvdiffrast/torch/ops.py", line 118, in _get_plugin                                    
    torch.utils.cpp_extension.load(name=plugin_name, sources=source_paths, extra_cflags=opts, extra_cuda_cflags=opts+['-lineinfo'], extra_ldflags=ldfla
gs, with_cuda=True, verbose=False)       


fatal error: cusparse.h: No such file or directory
    6 | #include <cusparse.h>

Basically, the issue is about the differences in the environments. Could you please keep the repo updated reflecting how exactly we should prepare the environment for the code?

Thanks!

(gsir) @:~/data/python/GS-IR$ python train.py -m /home/data/datainuse/models/bicycle/ -s /home/data/datainuse/360_v2/bicycle/ --iterations 30000 -i images_4 -r 1 --eval
Traceback (most recent call last):
File "train.py", line 453, in
def prepare_output_and_logger(args: GroupParams) -> Optional[SummaryWriter]:
NameError: name 'SummaryWriter' is not defined

Thanks for your work. Can you share some implementation details about the point lighting results at the end of your project page? Is the shadows computed for each Gaussian or for each pixel fragment?

Hi,

Excellent work! It seems that the submodule folder is ignored in the last commit, especially the modified version of diff-gaussian-rasterization.