leaplabthu / activenerf Goto Github PK

Thank you for your impressive work! I have a couple of questions regarding your Equation 11 and its corresponding code snippet:
Equation 11 corresponds to the following code
img2mse_uncert_alpha = lambda x, y, uncert, alpha, w : torch.mean((1 / (2*(uncert+1e-9).unsqueeze(-1))) *((x - y) ** 2)) + 0.5*torch.mean(torch.log(uncert+1e-9)) + w * alpha.mean() + 4.0
My main query is regarding the difference between the regularization term in the formula. The regularization term in formula (11) uses the volume density , while the code segment uses alpha.mean(). Could you please clarify why the volume density isn't utilized in the code? Additionally, I'm curious about the significance of adding 4.0 at the end of the loss function.
Why not use the volume density and why plus 4 at the end of the loss function?

Hi, can I visualize the scene while training? Thanks!

Hi,

Thanks for the great work! I notice that in Table 2 there are two settings NeRF+Random and NeRF†, and I'm a little confused about the difference between these two settings.

Is it that the NeRF† uses all training images from the beginning of training and NeRF+Random gradually adds images during training? If so, do they use the same randomly selected images?

And when you mentioned "difficulties introduced by continuous learning", did that refer to the difficulties caused by fewer training iterations for later added images, or did the NeRF† setting select some optimal images?

Thanks in advance!

Hello,

When I ran the code, I was having this issue:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB (GPU 0; 5.80 GiB total capacity; 4.91 GiB already allocated; 51.38 MiB free; 4.93 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

From nvidia-smi I saw there was still memory to be used ( 867MiB/6138MiB, and my GPU is RTX A2000) and I've also tried checking float size (I believe in default you were using float32), following the error instruction to use setting max_split_size_mb or adding torch.cuda.empty_cache() in run_nerf.py

None of them worked.

Could you please help me identify what problems I have with implementing your code? Or probably I just need a better GPU?

I'd appreciate your help!
Thanks so much! I look forward to your reply.

Hello, I'm trying to reproduce the synthetic scene results in setting 1, but I can't.
I believe the released code is ActiveNeRF-CL, right?
Could you check the config settings I used?
The average PSNR I got is 13...

N_importance = 128
N_rand = 512
N_samples = 64
active_iter = [40000, 80000, 120000, 160000]
basedir = ./logs/materials_active
beta_min = 0.01
choose_k = 4
chunk = 8192
config = configs/blender_active_20k.txt
datadir = ./data/nerf_synthetic/materials
dataset_type = blender
ds_rate = 2
case_name = materials
expname = active_materials
factor = 8
ft_path = None
half_res = False
i_all = 200000
i_embed = 0
i_img = 500
i_print = 500
i_testset = 10000
i_video = 50000
i_weights = 20000
init_image = 4
lindisp = False
llffhold = 8
lrate = 0.0005
lrate_decay = 500
multires = 10
multires_views = 4
netchunk = 16384
netdepth = 8
netdepth_fine = 8
netwidth = 256
netwidth_fine = 256
no_ndc = False
no_reload = False
perturb = 1.0
precrop_frac = 0.5
precrop_iters = 500
raw_noise_std = 0.0
render_factor = 4
render_only = True
render_test = True
spherify = False
testskip = 1
use_viewdirs = True
w = 0.01
white_bkgd = True

Hi, I'm trying to make run_nerf.py use a specific gpu on my machine by setting os.environ['CUDA_VISIBLE_DEVICES'] = 'gpu id'

before importing torch, but it always tries uses gpu 0. How can I solve this problem? Thanks.

hi , when will the code be released?

@Panxuran
Hi Xuran, thanks for your awesome work!

In Sec 4, you mentioned 'Following previous researches in Bayesian neural networks, we take the model output as the mean'. However, the model outputs the uncertainty/variance β^2(r) in a shallower layer, which is even not related to the input d. It seems more intuitive to have the mean c(r, d) and variance β^2(r) both depend on the same input r(t) and d (just like the following figure, ref to NeurAR). Have you conducted any related experiments or had any insight?

Look forward to your reply!

Hi, what I understand is that ActiveNeRF selects the best subset of samples for the model from a large number of samples, but in this way, it still need a larger number of samples first. In practical applications, it may only have less than ten images for a scene, and in this case, ActiveNeRF cannot be used. I wonder if I misunderstand the model. Secondly, compared to NeRF, ActiveNeRF only uses a small number of samples to train and ultimately achieves similar results. Can it be considered that its feature extraction ability and model fitting ability are stronger, similar to those models with feew-shot input. Looking forward to your reply!

Hi, thank you for the amazing work. After reading the paper, I have a few questions about the information gain function. In your work, you define it as the subtraction of the variance from prior to posterior (e.g., Equation[20][21]).

1. Have you tried to only use the variance of prior or the variance of posterior as the information gain function?

2. Is there any insight that you choose Equation[20][21] instead of the definition mentioned in 1?