ethanweber / nerfbusters Goto Github PK

A few dependencies seem to have moved out from under your project. I'm struggling to make it run, even with an editable install of your nerfstudio branch.

Any chance you could put together a Dockerfile that layers your stuff on top of a specific version of dromni/nerfstudio?

Hello, I really love your application, and I have seen your outputs, and they do a pretty good job. I also understand that you use the Shapenet dataset to train and remove the noise from the point cloud. My project involves a plain surface like a wall, floor, ceiling, etc. Let's say there is a pillory inside the hall. Will the model help me to remove the noise around the pillor or remove the pillor since the pillor is not trained on the shapenet. If I'm right ? Also, what will happen to the wall, ceiling, and floor?

Hi authors,

Great thanks for this interesting work!

I wonder if you could release the final nerfbuster's checkpoints for the nerfbuster dataset? I would like to check the per-scene results.
It seems that my re-training cannot reproduce the reported scores well.

Thanks!

how calculate camera parameter ?

$ pip install --no-deps git+https://github.com/ethanweber/nerfbusters
...does not provide config/shapenet.yaml in the installed package.

".../lib/python3.11/site-packages/nerfbusters/nerf/nerfbusters_pipeline.py", line 355, in load_diffusion_model
config = yaml.load(open(diffusion_config_path, "r"), Loader=yaml.Loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'config/shapenet.yaml'

.../lib/python3.11/site-packages/nerfbusters
$ ls -lgG
total 36
drwxr-xr-x 2 4096 Nov 27 13:57 __pycache__
drwxr-xr-x 3 4096 Nov 27 13:57 cubes
drwxr-xr-x 3 4096 Nov 27 13:57 data_modules
-rw-r--r-- 1 3273 Nov 27 13:57 download_nerfbusters_dataset.py
drwxr-xr-x 3 4096 Nov 27 13:57 lightning
drwxr-xr-x 3 4096 Nov 27 13:57 models
drwxr-xr-x 4 4096 Nov 28 00:52 nerf
-rw-r--r-- 1 3367 Nov 27 13:57 run.py
drwxr-xr-x 3 4096 Nov 27 13:57 utils

checkpoint in the first page cant download.

When running nerfbusters (githash 1f42403) with nerfstudio (4d80f0ceeefa13e91e14e9cdebec39232b5cb72e) I get ModuleNotFoundError: No module named 'nerfstudio.fields.visibility_field' error. Is there a particular version/githash of nerfstudio that I need?

hi, when i train this code, i got this error
"ValueError : string is not a file: /path/ShapeNetCore.v2/04379243/models/model_normalized.obj"
at "cubes3d.py : 131" trimesh.load function

should i change some format of shapenet?

and Do I need to create a train_labels.txt to train the shapenetcore.v2 dataset?
there's no train_labels.txt in shapenet dataset

thank you

Hi
I installed nerfstudio as a pip package in a conda environment (cannot tell exact version)
Then I installed nerfbusters from source (git clone, pip install -e .)
Nerfbusters is trying to import from visibility_field.py, which doesn't seem to exist anywhere under /lib/python3.8/site-packages/nerfstudio

from nerfstudio.fields.visibility_field import VisibilityField
line 42 in nerfbusters/nerf/nerfbusters_pipeline.py

Hi,

Thanks for the amazing work. I ran into a problem trying to replicate baseline results. The commands provided for training the nerfacto models for baseline, has --eval-mode argument that doesn't seem to be supported in nerfstudio. When I run the command I get ns-train: error: unrecognized arguments: --eval-mode.
Is there a workaround for this without needing to restructure the whole dataset files? And what nerfstudio version was used that worked with this command?

Here is the error. I can run the exporter of nerfstudio successfully. But after I installed nerfbusters, from nerfstudio.utils.eval_utils import eval_setup this command reports me an error.

Previously I met with almost all the errors reported in previous issues, such as visibility_fields, depth_method, etc.

>>> from nerfstudio.utils.eval_utils import eval_setup
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mypath/projects/nerfstudio/nerfstudio/utils/eval_utils.py", line 28, in <module>
    from nerfstudio.configs.method_configs import all_methods
  File "/home/mypath/projects/nerfstudio/nerfstudio/configs/method_configs.py", line 669, in <module>
    all_methods, all_descriptions = merge_methods(all_methods, all_descriptions, *discover_methods())
  File "/home/mypath/projects/nerfstudio/nerfstudio/plugins/registry.py", line 43, in discover_methods
    spec = discovered_entry_points[name].load()
  File "/home/mypath/.conda/envs/nerfstudio/lib/python3.8/site-packages/importlib_metadata/__init__.py", line 184, in load
    module = import_module(match.group('module'))
  File "/home/mypath/.conda/envs/nerfstudio/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/mypath/projects/nerfbusters/nerfbusters/nerf/nerfbusters_config.py", line 16, in <module>
    from nerfbusters.nerf.nerfbusters_pipeline import NerfbustersPipelineConfig
  File "/home/mypath/projects/nerfbusters/nerfbusters/nerf/nerfbusters_pipeline.py", line 29, in <module>
    from nerfbusters.cubes.visualize3D import get_image_grid
  File "/home/mypath/projects/nerfbusters/nerfbusters/cubes/visualize3D.py", line 13, in <module>
    from nerfbusters.cubes.datasets3D import Crop
  File "/home/mypath/projects/nerfbusters/nerfbusters/cubes/datasets3D.py", line 29, in <module>
    from nerfstudio.utils.eval_utils import eval_setup
ImportError: cannot import name 'eval_setup' from partially initialized module 'nerfstudio.utils.eval_utils' (most likely due to a circular import) (/home/mypath/projects/nerfstudio/nerfstudio/utils/eval_utils.py)

Hey Ethan! :)
Where's the preferable location for the data/ dir?
At present it seems relative to the local run of nerfbusters-setup
(I happen to be in my ~/bin/ right now):

42     def download(self, save_dir: Path):
43         # Download the files
44         url = "https://drive.google.com/uc?id=1iioFQrH8cCmYxjLSoBb6DHHP7pP4QnIp"
45         download_path = str(save_dir / "nerfbusters-diffusion-cube-weights.ckpt")
46         import ipdb; ipdb.set_trace(context=16); pass
---> 47         gdown.download(url, output=download_path)
48
49
50 @dataclass
51 class NerfbustersCapturesDownload(DatasetDownload):
52     """Download the Nerfbusters captures."""
53
54     def download(self, save_dir: Path):

ipdb> download_path
'data/nerfbusters-diffusion-cube-weights.ckpt'

Hi

I want to compute visibility median count map for my own scene and mipnerf360 dataset.

I understand how the process works, but I couldn't find the code for processing this map. I would appreciate it if you could let me know where the code is.

Thanks in advance.

Hi, i've been trying to use nerfstudio but the provided link to download the trained diffusion model doesn't work.

Could you reupload it somewhere?

$ ns-train nerfbusters --help
Traceback (most recent call last):
...
"...python3.11/site-packages/nerfbusters/nerf/nerfbusters_pipeline.py", line 42, in <module>
from nerfstudio.fields.visibility_field import VisibilityField
ModuleNotFoundError: No module named 'nerfstudio.fields.visibility_field'

I don't see that in their source or my web searches.

I also meet this problem. I find another person post this issue but is not replied. I'm looking forward to a solution. Thank you!

Traceback (most recent call last):
  File "/home/mypath/.conda/envs/nerfstudio/bin/ns-train", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/mypath/projects/nerfstudio/nerfstudio/scripts/train.py", line 275, in entrypoint
    main(
  File "/home/mypath/projects/nerfstudio/nerfstudio/scripts/train.py", line 258, in main
    launch(
  File "/home/mypath/projects/nerfstudio/nerfstudio/scripts/train.py", line 193, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "/home/mypath/projects/nerfstudio/nerfstudio/scripts/train.py", line 102, in train_loop
    trainer.setup()
  File "/home/mypath/projects/nerfstudio/nerfstudio/engine/trainer.py", line 149, in setup
    self.pipeline = self.config.pipeline.setup(
  File "/home/mypath/projects/nerfstudio/nerfstudio/configs/base_config.py", line 59, in setup
    return self._target(self, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'grad_scaler'

Thanks for your great work, but I found there is something wrong with the link to the pretrained 3D diffusion checkpoint.

Error code explanation: HTTPStatus.NOT_FOUND - Nothing matches the given URI.

Okay not sure if this is from nerfbusters or ns (but I feel I'm getting closer to this working! (famous last words?))

Viewer at: https://viewer.nerf.studio/versions/23-03-9-0/?websocket_url=ws://localhost:7007
Printing profiling stats, from longest to shortest duration in seconds
Trainer.train_iteration: 0.4158              
NerfbustersPipeline.get_train_loss_dict: 0.3650              
Traceback (most recent call last):
  File "/home/j/venvs/ns/bin/ns-train", line 8, in <module>
    sys.exit(entrypoint())
             ^^^^^^^^^^^^
  File "/home/j/venvs/ns/lib/python3.11/site-packages/scripts/train.py", line 247, in entrypoint
    main(
  File "/home/j/venvs/ns/lib/python3.11/site-packages/scripts/train.py", line 233, in main
    launch(
  File "/home/j/venvs/ns/lib/python3.11/site-packages/scripts/train.py", line 172, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "/home/j/venvs/ns/lib/python3.11/site-packages/scripts/train.py", line 87, in train_loop
    trainer.train()
  File "/home/j/src/nerf/nerfstudio/nerfstudio/engine/trainer.py", line 225, in train
    loss, loss_dict, metrics_dict = self.train_iteration(step)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/j/src/nerf/nerfstudio/nerfstudio/utils/profiler.py", line 43, in wrapper
    ret = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/j/src/nerf/nerfstudio/nerfstudio/engine/trainer.py", line 421, in train_iteration
    self.optimizers.optimizer_scaler_step_all(self.grad_scaler)
  File "/home/j/src/nerf/nerfstudio/nerfstudio/engine/optimizers.py", line 130, in optimizer_scaler_step_all
    grad_scaler.step(optimizer)
  File "/home/j/venvs/ns/lib/python3.11/site-packages/torch/cuda/amp/grad_scaler.py", line 413, in step
    len(optimizer_state["found_inf_per_device"]) > 0
AssertionError: No inf checks were recorded for this optimizer.

Hi, and thanks for the great work!

I have reproduced the experiments in Table 1, and for the most part the metrics are very similar to reported in the paper. The only inconsistency I found was the Depth metric.

Paper:

Reproduced:

I was wondering if this type of behavior is expected between different runs?

Hi Ethan,

Thanks for your great work on post-cleaning trained nerf results. I have problems when launching the standard ns-train routine after installing the nerfbuster repo. Your file in /nerfbusters/nerfbusters/nerf/nerfbusters_pipeline.py currently imports the visibility field class by:
from nerfstudio.fields.visibility_field import VisibilityField

The original nerfstudio package in /nerfstudio/nerfstudio/fields contains several field calculations for different types of nerf. None of those however contains the class VisibilityField (in fact I can't find one that calculates visibility as a field). Could you please supplement the required files/codes that allow the class to be imported?

Regards,
Pomoron

Hey! Is there an image available to reproduce the environment for this project? I'm having some trouble getting this code working, specifically with functorch (says that vmap does not exist) and tcnn(tcnn not found error -tinycudann does exist though). Still, I think there's a CUDA version issue too w tinycudann. I imagine an image would make this a lot easier.

On a related note, would love to see nerfbusters use the latest release of nerfstudio if possible - as it's quite tough to integrate this into existing pipelines which use the latest release.

It's me again. Here is another problem.

Traceback (most recent call last):
  File "nerfstudio/scripts/train.py", line 286, in <module>
    entrypoint()
  File "nerfstudio/scripts/train.py", line 275, in entrypoint
    main(
  File "nerfstudio/scripts/train.py", line 258, in main
    launch(
  File "nerfstudio/scripts/train.py", line 193, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "nerfstudio/scripts/train.py", line 102, in train_loop
    trainer.setup()
  File "/home/mypath/projects/nerfstudio/nerfstudio/engine/trainer.py", line 178, in setup
    self.viewer_state = ViewerState(
  File "/home/mypath/projects/nerfstudio/nerfstudio/viewer/viewer.py", line 240, in __init__
    if len(parse_object(pipeline, LegacyViewerElement, "Custom Elements")) > 0:
  File "/home/mypath/projects/nerfstudio/nerfstudio/viewer/utils.py", line 151, in parse_object
    lower_rets = parse_object(v, type_check, new_tree_stub)
  File "/home/mypath/projects/nerfstudio/nerfstudio/viewer/utils.py", line 139, in parse_object
    obj_props = [(k, getattr(obj, k)) for k in dir(obj)]
  File "/home/mypath/projects/nerfstudio/nerfstudio/viewer/utils.py", line 139, in <listcomp>
    obj_props = [(k, getattr(obj, k)) for k in dir(obj)]
  File "/home/mypath/.conda/envs/nerfstudio/lib/python3.8/site-packages/pytorch_lightning/core/module.py", line 179, in trainer
    raise RuntimeError(f"{self.__class__.__qualname__} is not attached to a `Trainer`.")
RuntimeError: NerfbustersTrainer is not attached to a `Trainer`.

May I ask where the file named diffusers is? thanks

I've nerfstudio already installed. So I followed from step 3 to install the nerfbusters but after executing all the commands successfully I am getting the below error when I try to use nerfbusters

$ ns-train nerfbusters --help

ns-train: error: argument {nerfacto,depth-nerfacto,instant-ngp,instant-ngp-bounded,mipnerf,semantic-nerfw,vanilla-nerf,tensorf,dnerf,phototourism}: invalid choice: 'nerfbusters' (choose from 'nerfacto', 'depth-nerfacto', 'instant-ngp', 'instant-ngp-bounded', 'mipnerf', 'semantic-nerfw', 'vanilla-nerf', 'tensorf', 'dnerf', 'phototourism')

Am I missing something?