ai-med / vox2cortex Goto Github PK

What is this file? _white_reduced_0.3.thickness could not be found, inserting dummy
is this a concern?

Hello, I noticed that there are three (because of deep supervision) final output maps for segmentation? Could you please post the weights you used for the three segmentations for the loss? And why the N_V_CLASSES (the number of vertex classes to distinguish) is 2? Why should it including background? Thanks a lot!

Hi all,

Thanks for this wonderful repo. I have a doubt since you seem to be using pytorch3d.

When training your model, do you normalise your meshes to be in the space [-1, 1] and then transform to original space using scaling and translation after you get model's output mesh or do you directly train your model so that the coordinates of your output Mesh from model are in their original space?

Thanks!

Hello, how long did it takes to converge when you were training the final model?

I ran preprocessing on this dataset, but still get this error.
[INFO] Created training loader of length 5
Traceback (most recent call last):
File "main.py", line 260, in
main(hyper_ps)
File "main.py", line 256, in main
loglevel=hps['LOGLEVEL'], resume=args.resume)
File "/v2c/vox2cortex/utils/train.py", line 556, in training_routine
start_epoch=start_epoch)
File "/v2c/vox2cortex/utils/train.py", line 322, in train
for iter_in_epoch, data in enumerate(training_loader):
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in next
data = self._next_data()
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
return self.collate_fn(data)
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/utils/
data/_utils/collate.py", line 83, in default_collate
return [default_collate(samples) for samples in transposed]
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 83, in
return [default_collate(samples) for samples in transposed]
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate
return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [4, 40962, 3] at entry 0 and [4, 36685, 3] at entry 1

Hi, could you please describe the steps/method you used to perform inference of high resolution (about 160000 vertices for each template) with the model trained on the lower resolution (about 40000 vertices for each template)?

Thanks a lot!!!

Thank you for publishing a great repo.
I am building the environment for the repo and got this error after running git checkout tags/vox2cortex_cvpr2022 -b vox2cortex_pytorch3d pip install -e . Im running on CUDA 11.7 on NVIDIA A6000; Python 3.8
Could you address it?

`Running setup.py develop for pytorch3d
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [53 lines of output]
    running develop
    running egg_info
    creating pytorch3d.egg-info
    writing pytorch3d.egg-info/PKG-INFO
    writing dependency_links to pytorch3d.egg-info/dependency_links.txt
    writing requirements to pytorch3d.egg-info/requires.txt
    writing top-level names to pytorch3d.egg-info/top_level.txt
    writing manifest file 'pytorch3d.egg-info/SOURCES.txt'
    reading manifest file 'pytorch3d.egg-info/SOURCES.txt'
    adding license file 'LICENSE'
    writing manifest file 'pytorch3d.egg-info/SOURCES.txt'
    running build_ext
    building 'pytorch3d._C' extension
    creating build
    creating build/temp.linux-x86_64-cpython-38
    creating build/temp.linux-x86_64-cpython-38/home
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/blending
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/compositing
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/face_areas_normals
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/gather_scatter
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/interp_face_attrs
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/knn
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/mesh_normal_consistency
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/packed_to_padded_tensor
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/point_mesh
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/pulsar
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/pulsar/cuda
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/pulsar/host
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/pulsar/pytorch
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/rasterize_meshes
    creating build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/rasterize_points
    /usr/bin/nvcc -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/thanhtul/code/pytorch3d/pytorch3d/csrc -I/home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/torch/include -I/home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/torch/include/TH -I/home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/torch/include/THC -I/home/thanhtul/miniconda3/envs/py38/include/python3.8 -c /home/thanhtul/code/pytorch3d/pytorch3d/csrc/blending/sigmoid_alpha_blend.cu -o build/temp.linux-x86_64-cpython-38/home/thanhtul/code/pytorch3d/pytorch3d/csrc/blending/sigmoid_alpha_blend.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -std=c++14 -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=sm_86
    nvcc fatal   : Unsupported gpu architecture 'compute_86'
    /home/thanhtul/code/pytorch3d/setup.py:72: UserWarning: The environment variable `CUB_HOME` was not found. NVIDIA CUB is required for compilation and can be downloaded from `https://github.com/NVIDIA/cub/releases`. You can unpack it to a location of your choice and set the environment variable `CUB_HOME` to the folder containing the `CMakeListst.txt` file.
      warnings.warn(
    /home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/cpp_extension.py:352: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    /home/thanhtul/miniconda3/envs/py38/lib/python3.8/site-packages/torch/cuda/__init__.py:104: UserWarning:
    NVIDIA RTX A6000 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
    The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70 sm_75.
    If you want to use the NVIDIA RTX A6000 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

      warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
    error: command '/usr/bin/nvcc' failed with exit code 1
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.`

Hello, I can't found the code of "ChamferAndNormalsAndCurvatureLoss", could you upload it? Thanks a lot!

Hi, I saw the size of the image the paper used is (128, 144, 128). While in my own dataset, the size of the image is always extremely anisotropic(such as (192, 192, 32)). Is it possible to use Vox2Cortex to train on extremely anisotropic data without any modification directly?

It seems that the training
loss = self.training_step(model, data, iteration)
is intractable for meshes in our dataset with current settings.

RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 31.75 GiB total capacity; 30.37 GiB already allocated; 13.94 MiB free; 30.52 GiB reserved in total by PyTorch)

I exited before and after this line of code and it is the line which fails. I assume it must be the loss function calculation since I don't appear to have that much data loaded.

Hi,
Thanks for your work first.
I wanna know what is the meaning of "NNs, ID" in the figure.
In line 342 of graph_net.py, "new_meshes.move_verts(deltaV_padded)", it seems after updating the location of meshes, the next block will use the new location to calculate. So in this way I'm confused about the meaning of "NNs, ID", and there is no explanation in the paper as well(I apologize if I missed anything).

Looking forward to your reply.

Dear author, thank you for sharing such excellent work, I have learned a lot from it.

I would like to ask how to generate the template of other organs by myself. I looked it up online and read that FreeSurfer is a brain processing tool.

Looking forward to your reply.

Hi, I wanted to use the chamfer curvature loss in some experiments. I was wondering if you could upload the version of pytorch3d.loss.chamfer_distance you were using. I may be missing something, but I am not sure how to interpret the points_weights parameter. I tried modifying the source code of the function, but I wanted to be sure on the implementation. Thanks in advance!

the code references some templates that haven't been uploaded. could you check that all supplementary materials are uploaded.
Great paper btw.
Thanks.
'../supplementary_material/white_pial/cortex_4_1000_3_smoothed_42016_sps[192, 208, 192]_ps[128, 144, 128].obj'
not in github

do you have any suggestions on how to compile with gpu support? I ended up installing facebook research's pytorch3d only to figure out you modified their loss functions because I can't compile with gpu support for some reason.
Traceback (most recent call last):
File "main.py", line 260, in
main(hyper_ps)
File "main.py", line 256, in main
loglevel=hps['LOGLEVEL'], resume=args.resume)
File "/v2c/vox2cortex/utils/train.py", line 553, in training_routine
start_epoch=start_epoch)
File "/v2c/vox2cortex/utils/train.py", line 328, in train
loss = self.training_step(model, data, iteration)
File "/v2c/vox2cortex/utils/logging.py", line 252, in time_wrapper
return_value = func(*args, **kwargs)
File "/v2c/vox2cortex/utils/train.py", line 157, in training_step
loss_total = self.compute_loss(model, data, iteration)
File "/v2c/vox2cortex/utils/logging.py", line 252, in time_wrapper
return_value = func(*args, **kwargs)
File "/v2c/vox2cortex/utils/train.py", line 205, in compute_loss
pred = model(x.cuda())
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/v2c/vox2cortex/utils/logging.py", line 252, in time_wrapper
return_value = func(*args, **kwargs)
File "/v2c/vox2cortex/models/vox2cortex.py", line 121, in forward
pred_meshes, pred_deltaV = self.graph_net(encoder_skips + decoder_skips)
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/v2c/vox2cortex/utils/logging.py", line 252, in time_wrapper
return_value = func(*args, **kwargs)
File "/v2c/vox2cortex/models/graph_net.py", line 238, in forward
latent_features = self.graph_conv_first(verts_packed, edges_packed)
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/v2c/vox2cortex/utils/utils_vox2cortex/graph_conv.py", line 80, in forward
features = F.relu(self.norm_first(self.gconv_first(features, edges)))
File "/opt/miniconda3/envs/vox2cortex/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/v2c/vox2cortex/utils/utils_vox2cortex/graph_conv.py", line 35, in forward
return D_inv * super().forward(verts, edges)
File "/app/pytorch3d/pytorch3d/ops/graph_conv.py", line 76, in forward
neighbor_sums = gather_scatter(verts_w1, edges, self.directed)
File "/app/pytorch3d/pytorch3d/ops/graph_conv.py", line 160, in forward
output = _C.gather_scatter(input, edges, directed, backward)
RuntimeError: Not compiled with GPU support.