Comments (3)
Hi, thanks for your attention to our work!
Unfortunately, I cannot give you a direct answer for lack of information. I would say using some tools to visualize the cameras will help a lot when you debug.
from dsnerf.
Hello @dunbar12138 ,
Thank you for your help.
I tried to visualize the rays with the package you suggested.
import pytransform3d.visualizer as p
import pytransform3d.transformations as pt
index_pose = i_train[0]
rays_o, rays_d = get_rays_by_coord_np(H, W, focal, poses[index_pose,:3,:4], depth_gts[index_pose]['coord'])
rays_o, rays_d = torch.Tensor(rays_o).to(device), torch.Tensor(rays_d).to(device)
pose = poses[index_pose, :, :]
pose_hom = np.eye(4)
pose_hom[:3, :4] = pose
pose = pose_hom
fig = pv.figure()
cam2world = pose
virtual_image_distance = 1
sensor_size = np.array([W, H])
intrinsic = [
[focal, 0, sensor_size[0] / 2], [0, focal, sensor_size[1] / 2], [0, 0, 1]
]
intrinsic_matrix = np.array(intrinsic)
fig.plot_transform(A2B=cam2world, s=0.2)
fig.plot_camera(
cam2world=cam2world, M=intrinsic_matrix, sensor_size=sensor_size,
virtual_image_distance=virtual_image_distance)
for i in range(rays_d.shape[0]):
ray_o = rays_o[i].cpu()
ray_d = rays_d[i].cpu()
fig.plot_vector(start=ray_o,
direction=ray_d,
c=(1.0, 0.5, 0.0))
P = np.zeros((2, 3))
colors = np.empty((2, 3))
P[0] = ray_o
# ray_o + (depth_gts[index_pose]['depth'][0] * rays_d)
P[1] = ray_o + (depth_gts[index_pose]['depth'][i] * -ray_d)
colors[:, 0] = np.linspace(1, 0, len(colors))
colors[:, 1] = np.linspace(0, 1, len(colors))
fig.plot(P, colors)
pcd = np.load('colmap_pointcloud.npy')
pcd_colmap = open3d.geometry.PointCloud()
pcd_colmap.points = open3d.utility.Vector3dVector(pcd)
pcd_colmap.paint_uniform_color([1, 0, 0])
pcd_colmap.transform(pose)
fig.add_geometry(pcd_colmap)
fig.view_init()
if "__file__" in globals():
fig.show()
else:
fig.save_image("__open3d_rendered_image.jpg")
But the resulting visualization is like this:
Shouldn't the rays intersect with the points as we go draw a line from camera in the ray direction with the depth value?
Also, the rays are pointing in the negative direction of the camera.
Is this correct?
When I try the above implementation on my own dataset with known poses and point cloud I get a similar output.
which I think is wrong.
Can you please tell me what am I doing wrong :)
from dsnerf.
I'm also interested in this problem, and I think it may have something to do with the t vector in the transform matrix. I would appreciate it if you could provide your own discovery or solution.
from dsnerf.
Related Issues (20)
- Do we need to run COLMAP with the exact poses given in the datasets like DTU and Blender HOT 2
- depth = (poses[id_im-1,:3,2].T @ (point3D - poses[id_im-1,:3,3])) * sc ,WHY? HOT 3
- Generate colmap data HOT 5
- Colmap with load_llff function
- May I ask if you can provide the DTU self-made dataset required for the source code, including the relevant scenarios of scan2 \ 14 \ 21 \ 64
- problem between pose form colmap sparse point cloud and pose in the dataset HOT 1
- Surface Normals from downsampled image rays HOT 1
- OSError: cannot write mode F as PNG HOT 2
- How to define training set and test set? Is it randomly generated?
- chapter 3.2 line 9. taking the re-projected z value? HOT 1
- OOM error HOT 1
- How to partition the DTU dataset? HOT 1
- sigma_ Loss and depth_ Differences in loss HOT 1
- RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! HOT 3
- Hi,I can't run your code because of line 1278, in _save rawmode, mode = _OUTMODES[mode] KeyError: 'F' HOT 2
- configs issue HOT 9
- new experiment by kangle HOT 1
- Is there sometime that sigmaloss may be inf? HOT 1
- 1
- difference of Poses created by colmap and poses created for each dataset
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dsnerf.