codestella / putting-nerf-on-a-diet Goto Github PK

It seems the script is only prepared to run on blender synthetic dataset. Could we enable it to run on our own data using COLMAP generated poses_bounds ?
Thanks!

hey! is it possible to share the diet loss with pixel-nerf? we try to add diet loss on pixel-nerf but it doesn't work at all. May be we did something wrong?

Could I get the few-shot data list of each object?
(= data index for 4-shot, 8-shot, 14-shot(onside))

Thank you

aswin@t1v-n-093bc110-w-0:~/hf-flax-project/putting-nerf-on-a-diet$ /usr/bin/python3 /home/aswin/hf-flax-project/putting-nerf-on-a-diet/main.py
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
1179 images
Pose data loaded - dict_keys(['c2w_mats', 'kinv_mats', 'bds', 'res_mats'])
Test PSNR
sacre_ius_64_ilr_1_olr_0.0005_bs_64_test (min: 16.222, max: 16.222, cur: 16.222)
Train PSNR
sacre_ius_64_ilr_1_olr_0.0005_bs_64_train (min: 15.479, max: 18.230, cur: 18.230)
0%|▎ | 500/150000 [21:30<107:11:08, 2.58s/it]
Traceback (most recent call last):
File "/home/aswin/hf-flax-project/putting-nerf-on-a-diet/main.py", line 84, in
my_trainer.train()
File "/home/aswin/hf-flax-project/putting-nerf-on-a-diet/src/trainer.py", line 183, in train
plt.savefig(os.path.join(temp_eval_result_dir, "{:06d}.png".format(step)))
File "/home/aswin/.local/lib/python3.8/site-packages/matplotlib/pyplot.py", line 966, in savefig
res = fig.savefig(*args, **kwargs)
File "/home/aswin/.local/lib/python3.8/site-packages/matplotlib/figure.py", line 3005, in savefig
self.canvas.print_figure(fname, **kwargs)
File "/home/aswin/.local/lib/python3.8/site-packages/matplotlib/backend_bases.py", line 2255, in print_figure
result = print_method(
File "/home/aswin/.local/lib/python3.8/site-packages/matplotlib/backend_bases.py", line 1669, in wrapper
return func(*args, **kwargs)
File "/home/aswin/.local/lib/python3.8/site-packages/matplotlib/backends/backend_agg.py", line 509, in print_png
mpl.image.imsave(
File "/home/aswin/.local/lib/python3.8/site-packages/matplotlib/image.py", line 1616, in imsave
image.save(fname, **pil_kwargs)
File "/usr/local/lib/python3.8/dist-packages/PIL/Image.py", line 2169, in save
fp = builtins.open(filename, "w+b")
FileNotFoundError: [Errno 2] No such file or directory: 'temp/temp_eval_result_dir/sacre_ius_64_ilr_1_olr_0.0005_bs_64/000500.png'

Hi, is there some resource for implementing semantic consistency loss in pytorch
@ksasi @howtowhy @robot0321 @masapasa

Hi, I see your code includes the dataset loader for LLFF dataset. Could you share any tips on how to run your code on LLFF dataset?

Hi @codestella and @riven314 ,

Thanks for the work on this repo! I just have a question about the function pose_spherical here. Specifically, it is not clear to me how you derive the C2W transformation from phi and theta. I don't understand what is the spherical coordinates system that you put in the comment of the function. Also what is the coordinate system of your NeRF? Is the OpenGL system?
Finally, what is the reasoning behind the matrix transformation: translation, followed by rot_phi, followed by rot_theta?

Thank you!