vcl3d / structurenet Goto Github PK

Hello,

I have created an anaconda environment called structurenet for this project. After installing pytorch, cuda, tinyobjloader and the remaining requirements, I moved on to installing the patch for disabling multisampling in pyrender.
However, after downloading UnixUtils and adding the executable to path, executing the "patch.exe -u <path_to renderer.py> pyrender_patch/renderer.diff" and remaining patches, the following error is raised:

Assertion failed: hunk, file patch.c, line 321

I have tried installing other patch.exe versions such as this one, but the same error occurs (with a different line number)...

Do you recommend any approach to solve this?
Thank you for your time.

P.S.
I am using:

• Windows 10
• conda 4.10.1
• pytorch 1.2.0
• torchvision 0.4.0
• cudatoolkit 10.0

I ran inference.py with your pretrained model, but output files are black images. I tested with depthmap images from azure kinect as input file for an inference step. And I also tested with imported depthmap images from an asymmetric_box.obj using blender.
I wonder if there is any specific format or condition for input depthmap files.

• specific pixel size ( 8bit, 16bit, 24bit, ... )
• specific resolution ( 320x160, ... )
• image captured from specific angle or distance
  If there is specific condition for testing, could you please explain it?
  or If it is possible, could you provides the sample depthmap images for testing?

Thank you for your exciting work!

I would like to know when I have run calibration.py with my own .pgm and .png files of multiple cameras and got the calibration.json, how I select the main camera, and how to get extrinsic of other cameras relative to the main camera

such as:

Thanks!!!

Hello, can you give me more pieces of input data?

Hello,

I tried executing inference.py with the pre-trained model you provide:

python inference.py --input_path <path_to_repo>\example --output_path <path_to_output_dir>\output --saved_params_path <path_to_model>\model_e6.pth.tar --scale 0.001

Scale was set to 0.001 as mentioned by @vladsterz here.
But an error is raised:

<path_to_repo>\src\dataset\rendering\transformations.py:1926: UserWarning: No module named 'src.dataset.rendering._transformations'
warnings.warn(str(err))
Loading previously saved model from <path_to_model>\model_e6.pth.tar
Loaded model name: full_mask_max > Traceback (most recent call last): > File "inference.py", line 175, in
inference(args, device)
File "inference.py", line 112, in inference > dsiterator = depthmap_dataset.DepthmapDataset(datasetparams)
File "<path_to_repo>\src\dataset\depthmap_dataset.py", line 22, in init
self._get_filelist()
File "<path_to_repo>\src\dataset\depthmap_dataset.py", line 30, in _get_filelist
self.filepaths.sort(key = lambda x: int(x.split("\")[-1].split(".")[0].split("")[2]))
File "<path_to_repo>\src\dataset\depthmap_dataset.py", line 30, in
self.filepaths.sort(key = lambda x: int(x.split("\")[-1].split(".")[0].split("")[2]))
IndexError: list index out of range

Could you shed some light into what I am doing wrong?

What do you mean by backgrounds for training data. Can I get more information about training the model? I want to train it on smaller boxes so that the cameras can be closer to the subject when using Volumetric Capture

I run inference.py with example data, and I got some label_pred.png image.
How to get calibration result (intrinsics and extrinsics matrix)?
Thanks.

Hello,

I am trying to use StructureNet to align perspectives of multiple Azure Kinects.
Is it possible to produce transformation matrices based on the output of inference.py or calibration.py ? If so, how?

I understand that your VolumetricCapture system provides a GUI to calibrate multiple sensors, but it requires the installation of the entire infrastructure. I do not currently own the entire hardware required for that, so I was hoping I could achieve the alignment by providing only the input depth images from my Azure Kinects.

Thank you for your time.

Hi and thank you for sharing your code! I just followed the installation guide, but run into an issue trying to load the pretrained model:

(structurenet367) D:\Projects\StructureNet>python inference.py --input_path input --output_path output --saved_params_path model\model_e8.pth
D:\Projects\StructureNet\src\dataset\rendering\transformations.py:1926: UserWarning: No module named 'src.dataset.rendering._transformations'
  warnings.warn(str(err))
Loading previously saved model from model\model_e8.pth
Traceback (most recent call last):
  File "inference.py", line 175, in <module>
    inference(args, device)
  File "inference.py", line 104, in inference
    model = models.get_UNet_model(model_name, model_params)
  File "D:\Projects\StructureNet\src\models\__init__.py", line 15, in get_UNet_model
    return UNet_max(params['width'], params['height'], params['ndf'], params['upsample_type'], params['nclasses'])
NameError: name 'UNet_max' is not defined

Upon closer inspection, the model name stored in model_e8.pth is 'full_max', which corresponds to the UNet_max function inside models/init.py However, only UNet_mask_max.py is provided. I can force model name to 'full_mask_max' and the code will execute, but I suspect this is not the right way (and I do not have any data yet to test it any further). Could you confirm this is the right combination of pretrained model and code? Thanks!

Could you provide an example script for the calibration of mulitple Kinect Azure DK with your approach?

Could you please explain the format of the end result calibration from this? I see that there are several different files (extrinsics.json in the main Calibration folder, calibration.json in the 'initial_calibration' folder, and calibration.json in the 'crf_refined_calibration' folder) that are output after running the calibration. I'd be interested to try this method's results within something like TouchDesigner - I've tried using the matrices under 'extrinsics' in the the calibration.json files as transforms, but it doesn't seem to be the correct use. If you could go over how to use the calibration output from this method that would be much appreciated. Thank you!