zju3dv / pvnet-rendering Goto Github PK

你生成的RT时，为什么好像没有与物体在图像中位置有关系，如果没有关系，你是如何让3D点与图像中2D点形成映射关系

Hello! We want to train PVNet on a custom dataset containing renderings of the objects of interest. We have .ply files for these objects and wonder if this repo can be used to create pose-annotated training data for PVNet.

If this is the case, what files need to be modified?

how to solve this problem?

(pytorch041) cc@cc:~/pvnet-rendering$ python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

from transforms3d.euler import euler2mat

[1]+ Stopped python
(pytorch041) cc@cc:~/pvnet-rendering$ python run.py --type rendering
found bundled python: /home/cc/Downloads/blender-2.79b-linux-glibc219-x86_64/2.79/python
Read blend: /home/cc/pvnet-rendering/data/blank.blend
Traceback (most recent call last):
File "/home/cc/pvnet-rendering/blender/render_backend.py", line 7, in
from transforms3d.euler import euler2mat
ImportError: No module named 'transforms3d'

Blender quit

Traceback (most recent call last):
File "run.py", line 24, in
globals()'run_' + args.type
File "run.py", line 20, in run_fuse
run()
File "/pvnet-rendering/fuse/fuse.py", line 336, in run
prepare_dataset_parallel(output_dir, linemod_dir, linemod_orig_dir, fuse_num, background_dir, cache_dir, worker_num)
File "/pvnet-rendering/fuse/fuse.py", line 217, in prepare_dataset_parallel
f.result()
File "/packages/miniconda2/envs/blender/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/packages/miniconda2/envs/blender/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
ValueError: operands could not be broadcast together with shapes (96,44) (96,44,1) (96,44)

I read all closed issues, and I read the source code. Although I have run the sample successfully, but there are so many params in the codes, I have no idea about customing my data. Because I found the pvnet test results aren't good, I want to synthesize more data for training. Could you help me if it's not very complicated?

Hi, thanks to share your code.
I am not clear about the following parts:
azimuths, elevations = self.sample_sphere(num_samples)
euler_sampler = stats.gaussian_kde(eulers.T)
eulers = euler_sampler.resample(num_samples).T
eulers[:, 0] = azimuths
eulers[:, 1] = elevations
So what the meaning of the last two steps? I know these eulers are 'szxz'.
I am also not clear about the meaning of eulers[:, 2]. Why it requires no changes?

Thanks!

Hi, something went wrong with CUDA version when rendering.
I am using Ubuntu18.04, GTX1660Ti, nvidia driver430, CUDA9, and blender2.79a.

The error message is as follow:
CUDA version 9.10 detected, build may succeed but only CUDA 8.0 is officially supported.
Compiling CUDA kernel ...
"/usr/local/cuda/bin/nvcc" -arch=sm_75 --cubin "/home/username/lib/blender2.79a/2.79/scripts/addons/cycles/source/kernel/kernels/cuda/kernel.cu" -o "/home/username/.cache/cycles/kernels/cycles_kernel_sm75_9738CB424246C25C69BDF166A1B9BDD7.cubin" -m64 --ptxas-options="-v" --use_fast_math -DNVCC -D__KERNEL_CUDA_VERSION__=90 -I"/home/username/lib/blender2.79a/2.79/scripts/addons/cycles/source"
nvcc fatal : Value 'sm_75' is not defined for option 'gpu-architecture'
CUDA kernel compilation failed, see console for details.

Could you help me solve this problem? Thanks very much.

Hi, thanks for sharing your code. I want to make sure of the way that you used to sample the pose when rendering.
I saw the code and it seems that the rotations are uniformly sampled around a sphere and the translations are generated from the Linemod dataset(add with Gaussian noise). Am I right?

Hello everyone,

I try to use this repo to synthesize a dataset for keypoint extraction like pvnet. I've got .ply-models from my objects and wanted to cumstomize the code to use my own models instead of LINEMOD.
So I changed the follwing:
linemod_cls_names in fuse.py to my own obejct classes
cfg.BLENDER_PATH in config.py to my blender path
cfg.linemod_cls_names in config.py to my own object classes

Furthermore I downloaded the recommended SUN-Dataset with background images and placed it in /pvnet-rendering/data/, I also placed my .ply-model in an order-structure similar to the recommended LINEMOD-Dataset-Structure in /pvnet-rendering/data/LINEMOD.

If I try to run pvnet-rendering with the command python run.py --type fuse I get the following errors:
Traceback (most recent call last):
File "run.py", line 24, in
globals()'run_' + args.type
File "run.py", line 20, in run_fuse
run()
File "D:\Masterarbeit\NN\pvnet-rendering\fuse\fuse.py", line 337, in run
prepare_dataset_parallel(output_dir, linemod_dir, linemod_orig_dir, fuse_num, background_dir, cache_dir, worker_num)
File "D:\Masterarbeit\NN\pvnet-rendering\fuse\fuse.py", line 210, in prepare_dataset_parallel
randomly_read_background(background_dir,cache_dir)
File "D:\Masterarbeit\NN\pvnet-rendering\fuse\fuse.py", line 201, in randomly_read_background
return imread(fns[np.random.randint(0,len(fns))])
File "mtrand.pyx", line 741, in numpy.random.mtrand.RandomState.randint
File "_bounded_integers.pyx", line 1353, in numpy.random._bounded_integers._rand_int32
ValueError: low >= high

Can you tell me, what I am missing in my customization? Or why this error occurs?
Thanks in advance :)

您好，我看了#3
然后按照您给的链接https://github.com/pengsida/configuration/blob/master/blender/install_pip.py去安装pip for blender，可是还是没有安装成功transforms3d，您能帮我看看哪里错误吗，感谢！

root@terminal35f8111b:~# cd /root/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin

root@terminal35f8111b:~/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin# ./python3.5m /root/get-pip.py
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting pip
Using cached https://mirrors.aliyun.com/pypi/packages/54/2e/df11ea7e23e7e761d484ed3740285a34e38548cf2bad2bed3dd5768ec8b9/pip-20.1-py2.py3-none-any.whl (1.5 MB)
Collecting setuptools
Using cached https://mirrors.aliyun.com/pypi/packages/73/22/156aac338c4ac8ffb1c3c6c3d4841611fdd4e40a9be7518bf5be432fd38c/setuptools-46.2.0-py3-none-any.whl (582 kB)
Collecting wheel
Using cached https://mirrors.aliyun.com/pypi/packages/8c/23/848298cccf8e40f5bbb59009b32848a4c38f4e7f3364297ab3c3e2e2cd14/wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-20.1 setuptools-46.2.0 wheel-0.34.2

root@terminal35f8111b:~/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin# pip install transforms3d
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Requirement already satisfied: transforms3d in /root/anaconda3/lib/python3.8/site-packages (0.3.1)

root@terminal35f8111b:~/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin# cd /opt/data/private/zhuyazhi/pvnet-rendering-master
root@terminal35f8111b:/opt/data/private/zhuyazhi/pvnet-rendering-master# python run.py --type rendering
AL lib: (EE) ALCplaybackOSS_open: Could not open /dev/dsp: No such file or directory
found bundled python: /root/blender-2.79a-linux-glibc219-x86_64/2.79/python
Read blend: /opt/data/private/zhuyazhi/pvnet-rendering-master/data/blank.blend
Traceback (most recent call last):
File "/opt/data/private/zhuyazhi/pvnet-rendering-master/blender/render_backend.py", line 7, in
from transforms3d.euler import euler2mat
ImportError: No module named 'transforms3d'

Blender quit

十分感谢作者的分享！
我想在渲染时将目标物体固定在渲染图片的中心位置，或自由调整目标物体出现在图片中的位置。
我尝试修改了render_backend.py render类中的camera.location 但没有成功。
我认为obj模型是固定在世界坐标系原点的，我该如何调整相机，使其对准模型的中心？

Hi.

the pvnet renders seem to belong with the blender intrinsics, what are the right camera intrinsics for the fuse images?

Thanks.

Adam

I want to render the pascal3d like dataset, and find that your code may include pascal 3d, how can I use it. And I am a newcomer in this fields, it's difficult for me to understand the whole render process and your code, du u have any suggestion? thank u very much!

你好，我想问一下，在生成训练图像的过程中，为了得到关键点在图像上的ground truth。通过标记在物体的三维点，如何计算得到其在对应图像上的坐标。我尝试调用opencv中的cv2.projectPoints，但是得到的结果并不正确。望指教！

I wonder why the euler order was set to "szxz" here? I learned that the default euler order is "sxyz" in blender. Am I missing something? @pengsida

Dear author,
Can you help me solve this problem? I have installed transforms3d and I can import it from other python files. I do not know why I can't import it in render_backend.py.
The error information is as follows:
found bundled python: /home/xxx/Documents/config/blender-2.79a-linux-glibc219-x86_64/2.79/python
File "/home/xxx/rendering/blender/render_backend.py", line 7, in
Read blend: /home/xxx/rendering/data/blank.blend
from transforms3d.euler import euler2mat
ImportError: No module named 'transforms3d'

Hi.
I'm trying to interpret the pose formats in the {n}_info.pkl files generated by fuse. When I test the blender renderings by projecting the 3d bounding box it works, that doesn't work for the fuse output. The rotation looks roughly right though.

Any assistance very welcome, I hope you can see the attached images.

Thanks.

Adam

How long does it take to render 10000 images and fuse 10000 images with single GTX1080Ti GPU, please?

Hi,
The link to download LINEMOD seems to be dead, and the LINEMOD dataset downloaded from the clean-pvnet repository lacks a file "rot4.rot" in $class_type/data.
Could you update the link or precise what this file is ?

Hello, thanks for the sharing great tools!

Currently I am trying to make this works on blender 2.82.
I have a few question on structuring the dataset.
Based on https://github.com/zju3dv/clean-pvnet#training-on-the-custom-object, I would need some information like camera position, diameter, poses, etc.

./data/blender_poses would be pose data. This data should be split by array and saved as .npy files

camera.txt would be linemod from

intrinsic_matrix = {
        'linemod': np.array([[572.4114, 0., 325.2611],
                              [0., 573.57043, 242.04899],
                              [0., 0., 1.]]),
        'blender': np.array([[700.,    0.,  320.],
                             [0.,  700.,  240.],
                             [0.,    0.,    1.]])
    }

For the mask images, how can I generate them?
should I read them from exr files, then make my own mask?

您好，请问以下的错误是什么原因呀
root@terminal35f8111b:/opt/data/private/zhuyazhi/pvnet-rendering-master# python run.py --type rendering
AL lib: (EE) ALCplaybackOSS_open: Could not open /dev/dsp: No such file or directory
found bundled python: /root/blender-2.79a-linux-glibc219-x86_64/2.79/python
Read blend: /opt/data/private/zhuyazhi/pvnet-rendering-master/data/blank.blend

Successfully imported '/opt/data/private/zhuyazhi/pvnet-rendering-master/data/LINEMOD/ape/ape.ply' in 1.891 sec
Traceback (most recent call last):
File "/opt/data/private/zhuyazhi/pvnet-rendering-master/blender/render_backend.py", line 456, in
batch_render_with_linemod(args, camera)
File "/opt/data/private/zhuyazhi/pvnet-rendering-master/blender/render_backend.py", line 366, in batch_render_with_linemod
bg_imgs = np.random.choice(bg_imgs, size=cfg.NUM_SYN)
File "mtrand.pyx", line 1083, in mtrand.RandomState.choice (numpy/random/mtrand/mtrand.c:10406)
ValueError: a must be non-empty

Blender quit

您好，我看了#3
然后按照您给的链接https://github.com/pengsida/configuration/blob/master/blender/install_pip.py去安装pip for blender，可是还是没有安装成功transforms3d，您能帮我看看哪里错误吗，感谢！

root@terminal35f8111b:~# cd ~ /root/Downloads
root@terminal35f8111b:~# wget https://bootstrap.pypa.io/get-pip.py
--2020-05-12 06:47:07-- https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 151.101.108.175, 2a04:4e42:36::175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|151.101.108.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1868059 (1.8M) [text/x-python]
Saving to: 'get-pip.py'

get-pip.py 100%[===================================>] 1.78M 34.5KB/s in 53s

2020-05-12 06:48:01 (34.4 KB/s) - 'get-pip.py' saved [1868059/1868059]

root@terminal35f8111b:~# cd /root/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin

root@terminal35f8111b:~/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin# python3.5m /get-pip.py
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting pip
Using cached https://mirrors.aliyun.com/pypi/packages/54/2e/df11ea7e23e7e761d484ed3740285a34e38548cf2bad2bed3dd5768ec8b9/pip-20.1-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.1
Uninstalling pip-20.1:
Successfully uninstalled pip-20.1
Successfully installed pip-20.1
root@terminal35f8111b:/blender-2.79a-linux-glibc219-x86_64/2.79/python/bin# pip install transforms3d
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Requirement already satisfied: transforms3d in /root/anaconda3/lib/python3.8/site-packages (0.3.1)

In render_utils.py, Line 511 from lib.utils.opengl_render_backend import render

I can't find the file in the folder, please check the code, thank you!

1. 有关渲染的代码中“num_samples = cfg.NUM_SYN=10，”ratio = (begin_elevation + 90) / 180
   num_points = int(num_samples // (1 - ratio))“，num_samples和num_points有啥关系？
   2.采样完pose后用blender去渲染，对应的程序段在哪里？
   多谢，希望的到您的指点！

thans for sharing. when I was running run.py this error occurred:

found bundled python: /home/xxx/blender-2.79a-linux-glibc219-x86_64/2.79/python
Read blend: /home/xxx/pvnet-rendering/data/blank.blend

Successfully imported '/home/xxx/pvnet-rendering/data/LINEMOD/cat/cat.ply' in 1.154 sec
Traceback (most recent call last):
File "/home/xxx/pvnet-rendering/blender/render_backend.py", line 456, in
batch_render_with_linemod(args, camera)
File "/home/xxx/pvnet-rendering/blender/render_backend.py", line 366, in batch_render_with_linemod
bg_imgs = np.random.choice(bg_imgs, size=cfg.NUM_SYN)
File "mtrand.pyx", line 1083, in mtrand.RandomState.choice (numpy/random/mtrand/mtrand.c:10406)
ValueError: a must be non-empty

Blender quit

would you help me with this?

Hi!

thank you for the pvnet rendering tool that integrates very well into the LINEMOD data set!

I would like to use pvnet-rendering to generate my own data set for pvnet and have some questions regarding the camera pose:

In render_utils.py:

`

def get_quat_translation(self, object_to_camera_pose):
    object_to_camera_pose = np.append(object_to_camera_pose, [[0, 0, 0, 1]], axis=0)
    world_to_camera_pose = np.append(self.world_to_camera_pose, [[0, 0, 0, 1]], axis=0)
    object_to_world_pose = np.dot(np.linalg.inv(world_to_camera_pose), object_to_camera_pose)
    quat = mat2quat(object_to_world_pose[:3, :3])
    translation = object_to_world_pose[:3, 3]
    return quat, translation

`

Why is the object pose multiplied with world_to_camera_pose and how did you determine the hard coded parameters in world_to_camera_pose?

Is this the pose that is computed with obj_centened_camera_pos
in render_backend.py?

`

def render(camera, outfile, pose):
    bpy.context.scene.render.filepath = outfile
    depth_file_output.file_slots[0].path = bpy.context.scene.render.filepath + '_depth.png'

    azimuth, elevation, theta = pose[:3]
    cx, cy, cz = obj_centened_camera_pos(cfg.cam_dist, azimuth, elevation)
    q1 = camPosToQuaternion(cx, cy, cz)
    q2 = camRotQuaternion(cx, cy, cz, theta)
    q = quaternionProduct(q2, q1)
    camera.location[0] = cx  # + np.random.uniform(-cfg.pose_noise,g_camPos_noise)
    camera.location[1] = cy  # + np.random.uniform(-g_camPos_noise,g_camPos_noise)
    camera.location[2] = cz  # + np.random.uniform(-g_camPos_noise,g_camPos_noise)
    camera.rotation_mode = 'QUATERNION'

    camera.rotation_quaternion[0] = q[0]
    camera.rotation_quaternion[1] = q[1]
    camera.rotation_quaternion[2] = q[2]
    camera.rotation_quaternion[3] = q[3]
    # camera.location = [0, 1, 0]
    # camera.rotation_euler = [np.pi / 2, 0, np.pi]

    setup_light(bpy.context.scene)
    rotation_matrix = get_K_P_from_blender(camera)['RT'][:, :3]
    camera.location = -np.dot(rotation_matrix.T, pose[3:])
    bpy.ops.render.render(write_still=True)

`

cx, cy, cz seem to be used for the camera rotation only (camera.location is overwritten later with -np.dot(rotation_matrix.T, pose[3:])).

Is there a reason for the particular choice of the parameter cfg.cam_dist = 0.5 instead of 1.0?

Thank you for your advice!

i have the error "ImportError: libIex-2_5.so.25: cannot open shared object file: No such file or directory" when import openexr. how to install openexr?

Hi, pengsida
thank you for sharing the code！
I followed the guide of pvnet was completed demo and test, the step of train have some errors: no '.pkl' file of render and fuse, so i downloaded the datasets of LINEMOD,LINEMOD_ORIG and SUN，when runing the command "python run.py --type rendering", error: ‘ImportError: No module named 'transforms3d'’. but i installed transforms3d in requirements.txt （transforms3d version is 0.3.1），then i uninstall and reinstall in python environment，no well.

I will be looking forward to your reply.

Do you have a license for this code?

Hi, I am not familiar with blender.

When I used the code for rendering , I noticed that you convert the rendered depth map to binary masks (see https://github.com/zju3dv/pvnet-rendering/blob/master/blender/render_utils.py#L218).

In addition, I found that the values in depth map are not equal to the depth map rendered by an OpenGL renderer.

Could you give me some hints on this? How can I get the same depth map results to that of an OpenGL renderer?

Hi. In the original ply I had a texture and some texture coordinates. It seems that information about texture and textcoords is completely ignored during import. How can I import all data from ply file including texture info? Is there a custom script for this task or I have to write it myself? I cannot believe that nobody faced the same problem.

Hi @pengsida , could you please share the procedure of generating the blender models used for LINEMOD? I found that the points are far more than the original models.

For some reason, the modules from system python are not being imported in 'render_backend.py'.
Seems like blender is using it's own build of python. I have already tried exporting the PYTHONPATH.
I though it could be a problem with conda env so i tried setting up everything in my usr/local/python.
Still the same problem. All modules are there and are imported outside the 'render_backend.py' script. But this script has problem importing them

~/pvnet-rendering$ python run.py --type rendering
found bundled python: /home/ahmad3/blender-2.79a-linux-glibc219-x86_64/2.79/python
Read blend: /home/ahmad3/pvnet-rendering/data/blank.blend
/home/ahmad3/blender-2.79a-linux-glibc219-x86_64
Traceback (most recent call last):
File "/home/ahmad3/pvnet-rendering/blender/render_backend.py", line 9, in
from transforms3d.euler import euler2mat
ImportError: No module named 'transforms3d'

I had rendered some cat images. So in renders/cat folder there are 0.jpg 0_depth.png 0_RT.pkl ...
And jpg images are normal, but the values of png images are all 0. And the values of png images in LINEMOD/cat/mask is 0 or 1. So I want to ask if the png of rendered images are normal?

is there away to change how the background textures from SUN2012 are cropped in the renders, because they are zoomed on and the resolution is not good.

您好，我想请问您遇到过这种情况吗？是因为blender下载错误了吗
root@terminal35f8111b:/opt/data/private/zhuyazhi/pvnet-rendering-master# python run.py --type rendering
sh: 1: /opt/data/private/zhuyazhi/blender-2.79a-linux-glibc219-x86_64/blender: Permission denied

found bundled python: /home/lxj/blender-2.79b-linux-glibc219-x86_64/2.79/python
Read blend: /home/Storage/lxj/PoseEst/pvnet/pvnet-rendering/data/blank.blend
Traceback (most recent call last):
File "/home/Storage/lxj/PoseEst/pvnet/pvnet-rendering/blender/render_backend.py", line 7, in
from transforms3d.euler import euler2mat
ImportError: No module named 'transforms3d'

Blender quit

I install blender-2.79 and configured the path in .config. the code was run in virtual environment and i've installed transforms3d library. Could you help me with this problem?Tks.

i download blender-2.79a-linux-glibc219-x86_64.tar.bz2 and try to use its budule python to run pvnet-rendering. step by step i have these python library installed:
Package Version
------------- --------
easydict 1.9
numpy 1.17.3
opencv-python 4.1.1.26
pip 19.3.1
setuptools 41.6.0
torch 1.1.0
wheel 0.33.6
then i run the command bellow under the pvnet-rendering directory:
../../3_soft/blender-2.79a/2.79/python/bin/python3.5m run.py --type rendering
got error: ImportError: No module named 'lmdb'
so i try to install lmdb with blender bundle python but meet these problem:
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-
prototypes -fPIC -Ilib/py-lmdb -Ilib -I/home/rlvc-one/workspace/3_soft/blender- 2.79a/2.79/python/include/python3.5m -c lmdb/cpython.c -o build/temp.linux-x86_64-3.5/lmdb/cpython.o -UNDEBUG -w
lmdb/cpython.c:42:20: fatal error: Python.h: no such file or directory
i tried in pycharm to install lmdb and told me to "Make sure that you have installed Python development packages for your operating system.", but i already install python-dev

i found some solutions on google, like download correspoding python source release and copy the include directory to blender bundle python include, but met these problem:

gcc -pthread -shared -L/opt/lib/bz2/lib -L/opt/lib/lzma/lib -L/opt/lib/ssl/lib -L/opt/lib/zlib/lib -L/opt/lib/libc6/lib -L/opt/lib/ncurses/lib -L/opt/lib/gpm/lib -L/opt/lib/sqlite/lib -L/opt/lib/readline/lib -Wl,--version-script=python.map build/temp.linux-x86_64-3.5/lmdb/cpython.o build/temp.linux-x86_64-3.5/lib/mdb.o build/temp.linux-x86_64-3.5/lib/midl.o -o build/lib.linux-x86_64-3.5/lmdb/cpython.cpython-35m-x86_64-linux-gnu.so
/usr/bin/ld：无法打开链接脚本文件 python.map：no such file or directory

sorry, i am new to python..

您好,想请问几个问题,麻烦了,我想用clean-pvnet跑自己的数据集(不是custom那个). 但是发现需要合成的render和fuse合成虚拟数据集,于是我尝试了,把自己的一个物体换到cat中,现在可以合成render数据集,但是在fuse时,出现了点问题
1:
运行` python run.py --type fuse

begin generate database cat
Traceback (most recent call last):
File "run.py", line 24, in
globals()'run_' + args.type
File "run.py", line 20, in run_fuse
run()
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 336, in run
prepare_dataset_parallel(output_dir, linemod_dir, linemod_orig_dir, fuse_num, background_dir, cache_dir, worker_num)
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 208, in prepare_dataset_parallel
collect_linemod_set_info(linemod_dir,cls_name,linemod_orig_dir,cache_dir)
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 188, in collect_linemod_set_info
data['RT'] = pose_transformer.orig_pose_to_blender_pose(pose).astype(np.float32)
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 131, in orig_pose_to_blender_pose
tra = tra + np.dot(rot, self.model_aligner.get_translation_transform())
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 99, in get_translation_transform
orig_model = self.load_orig_model()
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 89, in load_orig_model
transform = self.read_transform_dat()
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 81, in read_transform_dat
transform_dat = np.loadtxt(self.transform_dat_path, skiprows=1)[:, 1]
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/site-packages/numpy/lib/npyio.py", line 961, in loadtxt
fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/site-packages/numpy/lib/_datasource.py", line 195, in open
return ds.open(path, mode, encoding=encoding, newline=newline)
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/site-packages/numpy/lib/_datasource.py", line 535, in open
raise IOError("%s not found." % path)
OSError: /home/lcz/PycharmProjects/xin/pvnet-rendering/data/LINEMOD_ORIG/cat/transform.dat not found.
发现没有transform.dat not found,请问transform.dat是什么,怎么获得?以及在LINEMOD_ORIG文件夹中OLDmesh和cat.ply有啥区别 我用了官方cat的transform.dat文件和自己的ply换成OLDmesh文件 .但是有新的错误 2: python run.py --type fuse
begin generate database cat
success generate database cat len 226
begin generate database duck
success generate database duck len 189
begin generate database glue
success generate database glue len 184
begin generate database iron
success generate database iron len 173
begin generate database phone
success generate database phone len 184
begin generate database benchvise
success generate database benchvise len 183
begin generate database can
success generate database can len 180
begin generate database driller
success generate database driller len 179
begin generate database eggbox
success generate database eggbox len 188
begin generate database holepuncher
success generate database holepuncher len 186
begin generate database lamp
success generate database lamp len 185
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/concurrent/futures/process.py", line 175, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 229, in prepare_dataset_single
rgb, mask, begin, pose=randomly_sample_foreground(image_dbs[cls_id], linemod_dir)
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 273, in randomly_sample_foreground
mask=np.sum(mask,2)>0
File "<array_function internals>", line 6, in sum
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 2242, in sum
initial=initial, where=where)
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 87, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
numpy.AxisError: axis 2 is out of bounds for array of dimension 2
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "run.py", line 24, in
globals()'run_' + args.type
File "run.py", line 20, in run_fuse
run()
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 336, in run
prepare_dataset_parallel(output_dir, linemod_dir, linemod_orig_dir, fuse_num, background_dir, cache_dir, worker_num)
File "/home/lcz/PycharmProjects/xin/pvnet-rendering/fuse/fuse.py", line 217, in prepare_dataset_parallel
f.result()
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/home/lcz/anaconda3/envs/ren/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
numpy.AxisError: axis 2 is out of bounds for array of dimension 2
请问这个是什么情况
3:如果我只想用render这个数据集,不使用fuse,该怎么改
`
期待您的回复,万分感谢

I am using GTX1080Ti, only one piece is used for three graphics cards, but why is there insufficient memory? Thank you for your reply

我用rendering进行渲染的时候，希望根据我指定的位姿将三维模型渲染到背景图片上。但背景图片总会有缩放，无法像fuse中将整张背景图片载入。想请问有什么解决方法呢？

/home/lxj/blender-2.79-linux-glibc219-x86_64/2.79/python/bin/python3.5m /home/Storage/lxj/PoseEst/pvnet/pvnet-rendering/run.py --type rendering

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

When i installed all necessary library for python3.5m in Blender, the code just exited and i cannot find any reason. In the lib-installing stage, I directly copied corresponding lib directory(lmdb, OpenEXR of python3.5) to /path/to/blender/2.70/python/lib/python3.5/site-packages directory for i couldn't install them in pycharm(it reported some errors), am not sure if the problem comes from here?Could you help me with this?

Could you provide some rendered images sample in README?

The README mentions both SUN2012 and SUN397:

Download the SUN397

wget http://groups.csail.mit.edu/vision/SUN/releases/SUN2012.tar.gz

I downloaded SUN2012, but it does not appear to have the correct format. From render_utils.py:
img_paths = glob.glob(os.path.join(cfg.SUN, 'JPEGImages/*'))

I don't see 'JPEGImages' in SUN2012.

Traceback (most recent call last):
File "run.py", line 24, in
globals()'run_' + args.type
File "run.py", line 11, in run_rendering
from blender.render_utils import Renderer, YCBRenderer
File "/home/net663/pvnet-rendering-master/blender/render_utils.py", line 9, in
import OpenEXR
ModuleNotFoundError: No module named 'OpenEXR'

Hi, I wonder what is the effect of b_empty = parent_obj_to_camera(camera) here https://github.com/zju3dv/pvnet-rendering/blob/master/blender/render_backend.py#L100.

It is created but not used, does it have any special effect?