nileshkulkarni / csm Goto Github PK
View Code? Open in Web Editor NEWCode release for "Canonical Surface Mapping via Geometric Cycle Consistency"
Home Page: https://nileshkulkarni.github.io/csm/
Code release for "Canonical Surface Mapping via Geometric Cycle Consistency"
Home Page: https://nileshkulkarni.github.io/csm/
Hello,
Great work on CSM.
So I had two questions:
Hi, I get this error for running the "Eval keypoint transfer" commands.
File "csm/utils/render_utils.py", line 1, in
import pymesh
ImportError: No module named pymesh
I tried to install Pymesh but it only supports Python 3.7 - in your requirements it says Python 2.7 though. What do you suggest?
Hi!
Can you please provide pre-processed mesh templates used?
best,
Anand
Thank you for your wonderful work!
I have two questions about the code.
Q1. In mean shape, Giving a point on the UV map, how to find its corresponding face(triangle)?
Q2. Asking for more references about the calculation of φ.
I have got that the calculation of φ(the process from UV map to 3d coordinate) was referred to this paper.
Could you please show me more references, such as links and formulations, about the calculation of φ.
For example, I know these lines are calculating the barycentric coordinates, but I want to know the formulations behind these codes.
Thanks a lot!!
Hi, it seems the training instructions doesn't provide information about how to prepare the Pascal 3D and ImageNet class dataset, could you please add some information? Thanks.
An extra null pointer check is not needed in functions like the following.
Thank you for your excellent work!
We get (Batchsize, 256,256,2) as prediction now.
Could you tell me how can we convert this to RGB (to get the colorful image just like your results)?
Thanks!
Eval keypoint transfer on birds throws error:
IOError: [Errno 2] No such file or directory: '/nfs.yoda/nileshk/CorrespNet/datasets/cubs/images/083.White_breasted_Kingfisher/White_Breasted_Kingfisher_0115_73252.jpg'
Eval keypoint transfer on cars throws error:
IOError: [Errno 2] No such file or directory: '/nfs.yoda/nileshk/CorrespNet/datasets/PASCAL3D+_release1.1/Images/car_imagenet/n04037443_13265.JPEG'
Hi, thanks for open-sourcing this wonderful work. Could you also provide weights of pretrained CSM model? Thanks!
Hey, first off thanks for the great work; it's great seeing progress towards less supervision even in 3D computer vision.
I was setting up the repo in order to run some comparisons and faced some issues while creating the environment from csm.yml
. I run into the following issue due to missing packages
> conda env create -f csm.yml
ResolvePackageNotFound:
- bleach=1.5.0=py27_0
- html5lib=0.9999999=py27_0
- cudatoolkit=8.0=3
Upon further inspection, it seemed the packages were outdated. I changed the package versions to something still available in conda forge but even that didn't help. After a few more changes i.e. shifting a couple libraries under pip in csm.yml, I was blocked by multiple conflicts.
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Package wheel conflicts for:
pip==18.1=py27_0 -> wheel
Package sqlite conflicts for:
qt==5.6.3=h8bf5577_3 -> sqlite[version='>=3.25.3,<4.0a0']
python==2.7.15=h9bab390_4 -> sqlite[version='>=3.25.2,<4.0a0']
Package six conflicts for:
tensorflow-gpu-base==1.4.1=py27h01caf0a_0 -> six[version='>=1.10.0']
mock==2.0.0=py27_0 -> six
protobuf==3.6.1=py27he6710b0_0 -> six
Would you have any suggestions for the above? Should I create my own environment and install the necessary packages by myself? Would really appreciate your help with this, cheers!
Hi,
As part of a college project, we implemented CSM using the latest packages and renderers.
While we were able to generate visually good looking UV mappings the Percentage Correct Key Points metric values for the categories bird and car are only at 10% for the alpha 0.1.
We were wondering if there were any training tricks used by the author to get the number up to the 50s.
Also is it possible to provide the trained model dictionary at least for the bird or car category?. So that we can test if its the model that's causing the low PCK or something else.
Thanks
Hey, installation.md refers to csm_root/csm/external/ which appears to be missing
Hello,
I would like to know if I could get access to the images from ImageNet used in your project or at least the mapping of image URLs and names used in the annotation files.
interesting
Hello Nilesh,
In the quaternion azimuth-elevation predictor, linear layer output is never used. Is that intended? On this line.
Thanks.
I want to implement GCC loss in my project and I'm trying to figure out the details of your implementation.
Question-0: the branch predicting uv for each foreground pixel (used for GCC loss) is only used during training. So it is only used to indirectly help train the 'camera pose' branch, and 'deformation' branch and won't be used during inference. Is my understanding correct?
Here are your relevant code and my understanding/questions of the implementation of GCC loss.
def get_sample_grid(img_size):
| x = torch.linspace(-1, 1, img_size[1]).view(1, -1).repeat(img_size[0],1)
| y = torch.linspace(-1, 1, img_size[0]).view(-1, 1).repeat(1, img_size[1])
| grid = torch.cat((x.unsqueeze(2), y.unsqueeze(2)), 2)
| grid.unsqueeze(0)
| return grid
self.grid = cub_parse.get_sample_grid(self.upsample_img_size).repeat( 1, 1, 1, 1).to(self.device)
grid = self.grid.repeat(b_size,1,1,1)
self.codes_gt['xy_map'] = grid
def project_uv_to_3d(uv2points, uv_map):
| B = uv_map.size(0)
| H = uv_map.size(1)
| W = uv_map.size(2)
| uv_map_flatten = uv_map.view(-1, 2)
| points3d = uv2points.forward(uv_map_flatten)
| points3d = points3d.view(B, H*W, 3)
| return points3d
self.uv2points = cub_parse.UVTo3D(self.mean_shape)
points3d = geom_utils.project_uv_to_3d(self.uv2points, codes_pred['uv_map'])
Question-1: Why do you map UV to 3D first using 'UVTo3D', then use 'project_uv_to_3d'? What's the graphics formula/theory behind these two functions?
def project_3d_to_image(points3d, cam, offset_z):
| projected_points = orthographic_proj_withz(points3d, cam, offset_z)
| return projected_points
def orthographic_proj_withz(X, cam, offset_z=0.):
| """
| X: B x N x 3
| cam: B x 7: [sc, tx, ty, quaternions]
| Orth preserving the z.
| """
| quat = cam[:, -4:]
| X_rot = quat_rotate(X, quat)
| scale = cam[:, 0].contiguous().view(-1, 1, 1)
| trans = cam[:, 1:3].contiguous().view(cam.size(0), 1, -1)
|
| proj = scale * X_rot
|
| proj_xy = proj[:, :, :2] + trans
| proj_z = proj[:, :, 2, None] + offset_z
|
| return torch.cat((proj_xy, proj_z), 2)
codes_pred['project_points_cam_pred'] = geom_utils.project_3d_to_image(points3d, codes_pred['cam'], self.offset_z)
codes_pred['project_points_cam_pred'] = codes_pred['project_points_cam_pred'][..., 0:2].view(self.codes_gt['xy_map'].size())
codes_pred['project_points'] = codes_pred['project_points_cam_pred']
Reprojection Loss
project_points = codes_pred['project_points']
| if opts.ignore_mask_gcc:
| reproject_loss = reproject_loss_l2(project_points, codes_gt['xy_map'], seg_mask*0+1)
| else:
| reproject_loss = reproject_loss_l2(project_points, codes_gt['xy_map'], seg_mask)
Question-2: why does the 'codes_pred['project_points']' obtained by 'orthographic_proj_withz' range from 0 to 255 (e.g. when input image size is 256x256) ? I think 'points3d' is already in this range, but how does step-2 make this happen?
Hi,I was trying to run model training command python -m csm.experiments.csm.csp --name=csm_bird_net --n_data_workers=4 --dataset=cub --display_port=8094 --scale_bias=0.75 --warmup_pose_iter=2000
, it report:
from ._conv import register_converters as _register_converters
loading /media/lab601/000555B8000E726C/wuy/csm/csm/data/../cachedir/cub/data/train_cub_cleaned.mat
5964 images
Loading Mean shape from /media/lab601/000555B8000E726C/wuy/csm/csm/data/../cachedir/cub/../shapenet/bird/shape.mat
Visdom Env Name csm_bird_net_wpose
create web directory /media/lab601/000555B8000E726C/wuy/csm/csm/experiments/csm/../../cachedir/web...
Traceback (most recent call last):
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/media/lab601/000555B8000E726C/wuy/csm/csm/experiments/csm/csp.py", line 439, in <module>
app.run(main)
File "/home/lab601/.local/lib/python2.7/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/lab601/.local/lib/python2.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/media/lab601/000555B8000E726C/wuy/csm/csm/experiments/csm/csp.py", line 434, in main
trainer.train()
File "csm/nnutils/train_utils.py", line 181, in train
self.forward()
File "/media/lab601/000555B8000E726C/wuy/csm/csm/experiments/csm/csp.py", line 219, in forward
codes_pred = self.model.forward(feed_dict)
File "csm/nnutils/icn_net.py", line 330, in forward
unet_output = self.unet_gen.forward(img)
File "csm/nnutils/unet.py", line 61, in forward
return self.model(input)
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "csm/nnutils/unet.py", line 111, in forward
self.x_enc = self.down(x_inp)
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/lab601/anaconda2/envs/csm/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
RuntimeError: CuDNN error: CUDNN_STATUS_MAPPING_ERROR
What's the solution to it?It's the cuda version problem?
Thanks for your excellent work!
It seems that links to download annotations doesn't work?
https://people.eecs.berkeley.edu/~shubhtuls/cachedir/vpsKps/segkps.zip
https://cmu.box.com/s/goter3meyi7rssbh2rujg36dfwygnc3f
Could you please update them?
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.