mmatl / pyrender Goto Github PK
View Code? Open in Web Editor NEWEasy-to-use glTF 2.0-compliant OpenGL renderer for visualization of 3D scenes.
Home Page: http://pyrender.readthedocs.io/
License: MIT License
Easy-to-use glTF 2.0-compliant OpenGL renderer for visualization of 3D scenes.
Home Page: http://pyrender.readthedocs.io/
License: MIT License
I have a mesh stored in .obj format, the relative positions of different objects in the mesh are shown big difference, who can give me some advice? Thanks in advance~
Hey,
I am quiet new to pyrender but I get an error that I do not know what is causing it when I run the tutorial code. The error I get is for the command pyrender.Viewer(scene), and it says
"Require WGL_ARB_create_context extension to create OpenGL 3 contexts". I am not sure how to proceed with this as I do not know what to do to fix this.
Hi,
First of all thanks for this tool.
I tried to follow offscreen renderering guide on a Google Colab notebook, but it failed, you can try it yourself live here (google account needed):
---------------------------------------------------------------------------
NoSuchDisplayException Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/pyrender/platforms.py in init_context(self)
209 resizable=False,
--> 210 width=1, height=1)
211 except Exception as e:
/usr/local/lib/python3.6/dist-packages/pyglet/window/xlib/__init__.py in __init__(self, *args, **kwargs)
169
--> 170 super(XlibWindow, self).__init__(*args, **kwargs)
171
/usr/local/lib/python3.6/dist-packages/pyglet/window/__init__.py in __init__(self, width, height, caption, resizable, style, fullscreen, visible, vsync, display, screen, config, context, mode)
572 if not display:
--> 573 display = pyglet.canvas.get_display()
574
/usr/local/lib/python3.6/dist-packages/pyglet/canvas/__init__.py in get_display()
94 # Otherwise, create a new display and return it.
---> 95 return Display()
96
/usr/local/lib/python3.6/dist-packages/pyglet/canvas/xlib.py in __init__(self, name, x_screen)
118 if not self._display:
--> 119 raise NoSuchDisplayException('Cannot connect to "%s"' % name)
120
NoSuchDisplayException: Cannot connect to "None"
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
<ipython-input-11-9ac112cbad03> in <module>()
20 innerConeAngle=np.pi/16.0)
21 scene.add(light, pose=camera_pose)
---> 22 r = pyrender.OffscreenRenderer(400, 400)
23 color, depth = r.render(scene)
24 (PIL.Image.fromarray(color)).save('color.png')
/usr/local/lib/python3.6/dist-packages/pyrender/offscreen.py in __init__(self, viewport_width, viewport_height, point_size)
30 self._platform = None
31 self._renderer = None
---> 32 self._create()
33
34 @property
/usr/local/lib/python3.6/dist-packages/pyrender/offscreen.py in _create(self)
127 os.environ['PYOPENGL_PLATFORM']
128 ))
--> 129 self._platform.init_context()
130 self._platform.make_current()
131 self._renderer = Renderer(self.viewport_width, self.viewport_height)
/usr/local/lib/python3.6/dist-packages/pyrender/platforms.py in init_context(self)
214 'context. If you\'re logged in via SSH, ensure that you\'re '
215 'running your script with vglrun (i.e. VirtualGL). The '
--> 216 'internal error message was "{}"'.format(e.message)
217 )
218
AttributeError: 'NoSuchDisplayException' object has no attribute 'message'
What can I do to overpass this limitation? Thanks.
In the get_projection_matrix method of IntrinsicsCamera there are the lines:
P[0][2] = 1.0 - 2.0 * cx / (width - 1.0)
P[1][2] = 2.0 * cy / (height - 1.0) - 1.0
I am unsure of why cx and cy are divided by width - 1.0/height - 1.0, and not just width/height. Specifying cx = 0.5 * width and cy = 0.5 * height should reproduce the behavior of PerspectiveCamera for those matrix entries, where those entries are 0.0 (which would occur if dividing by width/height, but not width - 1.0/height - 1.0. Is this perhaps a bug? Or maybe this is meant to address some pixel centering issue? When I modify to remove the -1s, I get the behavior I expect, but leaving them in produces renderings that are shifted with respect to what I expect.
Thanks @mmatl for a great and lightweight rendering library. I just wanted to double check that our smoothing option is done correctly. Here's a minimal example, running this:
import trimesh
import pyrender
import numpy as np
import pickle
cloth_file = 'cloth_file.pkl'
with open(cloth_file, 'rb') as fh:
cloth_data = pickle.load(fh)
cloth_pts = cloth_data['pts']
cloth = [[p.x, p.y, p.z] for p in cloth_pts]
cloth = np.array(cloth) # shape (625,3)
# wh = width/height, both 25 in our case as the cloth is a 25x25 grid.
wh = 25
faces = []
for r in range(wh-1):
for c in range(wh-1):
pp = r*wh + c
faces.append( [pp, pp+wh, pp+1] )
faces.append( [pp+1, pp+wh, pp+wh+1] )
faces = np.array(faces)
#print(faces.shape) (1152,3)
# Form the scene, with smoothing turned on.
tm = trimesh.Trimesh(vertices=cloth, faces=faces)
mesh = pyrender.Mesh.from_trimesh(tm, smooth=True)
scene = pyrender.Scene()
scene.add(mesh)
camera = pyrender.PerspectiveCamera(yfov=np.pi / 3.0, aspectRatio=1.0)
s = np.sqrt(2)/2
camera_pose = np.array([
[0.0, -s, s, 0.3],
[1.0, 0.0, 0.0, 0.0],
[0.0, s, s, 0.35],
[0.0, 0.0, 0.0, 1.0],
])
scene.add(camera, pose=camera_pose)
light = pyrender.SpotLight(color=np.ones(3), intensity=3.0,
innerConeAngle=np.pi/16.0,
outerConeAngle=np.pi/6.0)
scene.add(light, pose=camera_pose)
pyrender.Viewer(scene, use_raymond_lighting=True)
I get this after rotating the viewer appropriately
Main question: is the smoothing set correctly? Or in other words, is this as smooth as we can realistically get with our 25x25 cloth representation?
I'm using Ubuntu 18.04, in a clean virtualenv with pip install pyrender
.
It requires a pickle file at 'cloth_file.pkl'. Unfortunately I can't upload these on the issues tracker so I emailed it to you.
Hi
I am using this awesome repo alongside trimesh, but the run_in_thread does not seem to work on macos.
It seems to not work on the sample provided in the docs, and I get something like below.
My environment is
10.14.2 Mojave
xcode 10.2
just in case.
2019-05-16 17:18:54.912 python[10096:8443809] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff2abf0ded __exceptionPreprocess + 256
1 libobjc.A.dylib 0x00007fff56cbc720 objc_exception_throw + 48
2 CoreFoundation 0x00007fff2ac0a87d -[NSException raise] + 9
3 AppKit 0x00007fff2810bc8e -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 324
4 AppKit 0x00007fff2810907c -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1488
5 AppKit 0x00007fff28108aa6 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 45
6 _ctypes.cpython-37m-darwin.so 0x000000010ebb8377 ffi_call_unix64 + 79
7 ??? 0x000070000c8c8460 0x0 + 123145512846432
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Is there a way to get a point cloud of a scene?
If not, I think this would be a useful feature.
For some reason, the shadow framebuffer gets corrupted randomly on MacOS in the current version. Unsure if this is a problem with Pyglet, PyOpenGL, or something to do with retina displays. Will investigate soon.
('Shader compile failure (0): 0(337) : error C1503: undefined variable "uv_0"\n', ['#version 330 core\n///////////////////////////////////////////////////////////////////////////////\n// Structs\n///////////////////////////////////////////////////////////////////////////////\n\nstruct SpotLight {\n vec3 color;\n float intensity;\n float range;\n vec3 position;\n vec3 direction;\n float light_angle_scale;\n float light_angle_offset;\n\n #if 0\n sampler2D shadow_map;\n mat4 light_matrix;\n #endif\n};\n\nstruct DirectionalLight {\n vec3 color;\n float intensity;\n vec3 direction;\n\n #if 0\n sampler2D shadow_map;\n mat4 light_matrix;\n #endif\n};\n\nstruct PointLight {\n vec3 color;\n float intensity;\n float range;\n vec3 position;\n\n #if 0\n samplerCube shadow_map;\n #endif\n};\n\nstruct Material {\n vec3 emissive_factor;\n\n#if 1\n vec4 base_color_factor;\n float metallic_factor;\n float roughness_factor;\n#endif\n\n#if 0\n vec4 diffuse_factor;\n vec3 specular_factor;\n float glossiness_factor;\n#endif\n\n#if 0\n sampler2D normal_texture;\n#endif\n#if 0\n sampler2D occlusion_texture;\n#endif\n#if 0\n sampler2D emissive_texture;\n#endif\n#if 1\n sampler2D base_color_texture;\n#endif\n#if 0\n sampler2D metallic_roughness_texture;\n#endif\n#if 0\n sampler2D diffuse_texture;\n#endif\n#if 0\n sampler2D specular_glossiness;\n#endif\n};\n...
I tried to run it with the view'er and it looks correct, but running it offscreen just gives the rgb values [38,38,38] where the object should be. The background has the correct color, and the depth image also looks correct.
I tried to run pyrender on the Leonard cluster at ETH Zurich.
https://scicomp.ethz.ch/wiki/Leonhard_applications_and_libraries
I get the following error:
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 351, in getattr
return getattr(self._module, name)
AttributeError: 'NoneType' object has no attribute 'Window'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 351, in getattr
return getattr(self._module, name)
AttributeError: 'NoneType' object has no attribute '_create_shadow_window'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "full_pipeline.py", line 354, in
import pyrender
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyrender/init.py", line 12, in
from .viewer import Viewer
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyrender/viewer.py", line 36, in
class Viewer(pyglet.window.Window):
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 357, in getattr
import(import_name)
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/window/init.py", line 1872, in
gl._create_shadow_window()
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 357, in getattr
import(import_name)
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/gl/init.py", line 100, in
from pyglet.gl.lib import GLException
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/gl/lib.py", line 143, in
from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/gl/lib_glx.py", line 51, in
glu_lib = pyglet.lib.load_library('GLU')
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/lib.py", line 168, in load_library
raise ImportError('Library "%s" not found.' % names[0])
ImportError: Library "GLU" not found.
Hi, I'm trying to run the following code:
import trimesh
import pyrender
tm = trimesh.load('/home/henry/Downloads/fuze.obj')
pymesh = pyrender.Mesh.from_trimesh(tm)
scene = pyrender.Scene()
scene.add(pymesh)
pyrender.Viewer(scene, use_raymond_lighting=True)
And I get the following error:
File "generate_pose_dataset.py", line 172, in mesh_render
pyrender.Viewer(scene, use_raymond_lighting=True)
File "/usr/local/lib/python2.7/dist-packages/pyrender/viewer.py", line 347, in __init__
self._init_and_start_app()
File "/usr/local/lib/python2.7/dist-packages/pyrender/viewer.py", line 995, in _init_and_start_app
height=self._viewport_size[1])
File "/usr/local/lib/python2.7/dist-packages/pyglet/window/xlib/__init__.py", line 170, in __init__
super(XlibWindow, self).__init__(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyglet/window/__init__.py", line 595, in __init__
context = config.create_context(gl.current_context)
File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/xlib.py", line 216, in create_context
return XlibContextARB(self, share)
File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/xlib.py", line 322, in __init__
super(XlibContext13, self).__init__(config, share)
File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/xlib.py", line 230, in __init__
raise gl.ContextException('Could not create GL context')
pyglet.gl.ContextException: Could not create GL context
My computer shows that PyOpenGL 3.1.0, PyOpenGL_accelerate 3.1.0, and pyglet appear to be correctly installed.
To test pyglet externally from pyrender, I ran the following code:
import pyglet
pyglet.window.Window()
and a window opens as expected.
Any ideas why this error is occurring?
I'm running 16.04 with 64 bit.
Thanks!
Henry C.
Are there certain parameters to specify when instantiating a pyrender.Mesh
to cull back faces when rendering wireframes? Currently, it looks like the entire wireframe is rendered including faces in the back which should be culled if the SKIP_CULL_FACES
flag is omitted.
I tried to render a high resolution depth map but get really coarse result (among 100000 pixels there are only 9 different values).
I noticed that pyrender gets depth from the z-buffer, and performs multiple divisions, that will lead to a lot of numerical loss.
A better solution is to record exact depth using another texture in shaders.
The documentation of IntrinsicsCamera states that the fx and fy input parameters are in units of meters. However, it is difficult for me to understand what that means, since to understand focal length in meter units means that you also know the size of the sensor (or image plane) in meters as well... instead, it seems from my tests that fx and fy are actually meant to be specified in pixel units. Is that correct?
Hi,
Given a mesh and camera in a given pose and location; is there a function that I can use to compute which faces/vertices of the mesh are visible from the specified camera view?
I understand that this should anyway be computed under the hood during the rendering process.
Thanks in advance.
Hi, I'm getting a TypeError: Expected a Trimesh or a list, got a <class 'trimesh.scene.scene.Scene'>
when importing a .glb file. Is there any way to go around this?
import trimesh
import pyrender
fuz_trimesh = trimesh.load("donut_calibrate2_forgltf.glb")
mesh = pyrender.Mesh.from_trimesh(fuz_trimesh)
Hi @mmatl , I am using the pyrender to render some point clouds in off-screen mode.
I am wondering if it is possible to render images with alpha channel?
Thanks!
I am running the following basic example:
import trimesh
import pyrender
fuze_trimesh = trimesh.load(OBJ FILE)
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)
and I encounter this error on a Mac:
No handlers could be found for logger "trimesh"
Traceback (most recent call last):
File "scripts/pyrender-trail.py", line 41, in <module>
pyrender.Viewer(scene, use_raymond_lighting=True)
File "/anaconda2/lib/python2.7/site-packages/pyrender/viewer.py", line 347, in __init__
self._init_and_start_app()
File "/anaconda2/lib/python2.7/site-packages/pyrender/viewer.py", line 997, in _init_and_start_app
raise ValueError('Unable to initialize an OpenGL 3+ context')
ValueError: Unable to initialize an OpenGL 3+ context
Hello,
I tried your example code to render the object in color and depth but can't get the correct result (see screenshot). In the depth image, the background is white (Value 0) and the Object has values between 1.13 and 1.2 - although it should be a float32 image. Do I have to do a normalization or something to get the correct result?
Thank you in advance!
seems like it has been moved to material.py
https://github.com/mikedh/trimesh/blob/3.0.9/trimesh/visual/material.py
Hi, I follow the step to install.But when I run the example , I got the bug "pyglet.window.NoSuchConfigException".
If you know why, please tell me how to solve it .
Thx your contribute.
Hey, I have the following setup..
..and im trying to convert image coordinates back to world coordinates the following way:
T_ext = camera_pose
T_int = camera.get_projection_matrix(w,h)
X_world = T_ext⁻¹ * T_int⁻¹ * X_img
Since it doesn't give any reasonable results there has to be some error in my approach. Would be very thankful if someone could help me out!
Hi @mmatl , after I finished the steps needed for offscreen render, I ran the example script, and I got this, do you have any clue for the reason? Thanks!
FYI, I am using docker, ubuntu 16, a headless server connected over SSH.
root@dffaaaba9cf1:/workspace/mesh_render# PYOPENGL_PLATFORM=osmesa python meshrender_example.py
No handlers could be found for logger "OpenGL.acceleratesupport"
Traceback (most recent call last):
File "meshrender_example.py", line 136, in
r = OffscreenRenderer(viewport_width=6402, viewport_height=4802)
File "/usr/local/lib/python2.7/dist-packages/pyrender/offscreen.py", line 28, in init
self._create()
File "/usr/local/lib/python2.7/dist-packages/pyrender/offscreen.py", line 102, in _create
self._platform.make_current()
File "/usr/local/lib/python2.7/dist-packages/pyrender/platforms.py", line 216, in make_current
self.viewport_width, self.viewport_height))
AssertionError
Hi,
Thanks for contributing this library.
I tried to render with objects in ply file format with following codes:
obj1_trimesh = trimesh.load('./models/obj_01.ply')
obj1_mesh = Mesh.from_trimesh(obj1_trimesh)
scene.add(obj1_mesh)
However, nothing appeared in the rendered images.
The zip format of ply file is attached.
obj_01.ply.zip
Could you please give some advice?
Thanks in advance!
Best,
Stacey
I am getting this error in the created thread when setting run_in_thread=True in pyrender.Viewer
Exception in thread Thread-6:
Traceback (most recent call last):
File "\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "\lib\site-packages\pyrender\viewer.py", line 1003, in init_and_start_app
pyglet.app.run()
File "\lib\site-packages\pyglet\app_init.py", line 142, in run
event_loop.run()
File "\lib\site-packages\pyglet\app\base.py", line 167, in run
platform_event_loop.start()
File "\lib\site-packages\pyglet\app\win32.py", line 96, in start
'thread that imports pyglet.app')
RuntimeError: EventLoop.run() must be called from the same thread that imports pyglet.app
This might have something to do with it https://stackoverflow.com/questions/27071801/can-pyglet-run-in-new-thread
I have not seen any other problems except not being able to set os.environ['PYOPENGL_PLATFORM'] = 'egl' for offline rendering.
My environment:
Windows 10 v1809
Python 3.6.8 :: Anaconda, Inc.
imageio 2.5.0
networkx 2.2
numpy 1.16.2
Pillow 5.4.1
pyglet 1.4.0b1
PyOpenGL 3.1.0
PyOpenGL-accelerate 3.1.0
pyrender 0.1.16
trimesh 2.37.5
This is an example from cuda to opengl:
https://gist.github.com/victor-shepardson/5b3d3087dc2b4817b9bffdb8e87a57c4
Could be interesting to have the inverse in the example dir.
Many want to use the render for datasets generation so it could be useful to have a small example to have the render available in cuda directly with interop and without device to host and host to device double transfer.
Hello,
I'm trying to render the depth image from my point cloud. Below is the outline of code snippet i have tried.
fuze_trimesh = trimesh.load('.../teddyUpright2.ply')
#camera pose
transform_mat = np.linalg.inv((np.loadtxt(".../frame-000000.pose.txt", delimiter=' ')))
cam_intrinsics = np.loadtxt(.../camera-intrinsics.txt", delimiter=' ')
fuze_trimesh.apply_transform(transform_mat)
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh, pose=np.eye(4))
camera = pyrender.IntrinsicsCamera(cam_intrinsics[0, 0], cam_intrinsics[1, 1], cam_intrinsics[0, 2], cam_intrinsics[1, 2], 0.0001, 10000)
camera_pose = np.eye(4)
camera_pose[2, 3] = 0.6
scene.add(camera, pose=camera_pose)
pyrender.Viewer(scene)
r = pyrender.OffscreenRenderer(640, 480)
color, depth = r.render(scene)
Now the actual issue:
The size of the object in the rendered depth image is very small as compared to the size of the object in image and even in imported mesh.
When i check the scale of the fuze_trimesh it is around 0.3045. i'm not able to figure out why the rendered depth image has small size?
any suggestions on why this is happening and how to solve this?
Hi,
in Scene
, bg_color
has the following docstring:
bg_color ((4,) float, optional) – Background color of scene.
However later in the documentation is estipulated that it should be:
Type: | (3,) float
Should it be 3 or 4?
P.S.
Great project :)!
I'm trying to render the Khronos GLTF example https://github.com/KhronosGroup/glTF-Sample-Models/blob/master/2.0/Duck/glTF-Binary/Duck.glb with pyrender:
trimesh.load
on the glb filemesh.geometry.values()
and add them to the sceneResulting image:
Note: This is the color image, not the depth image.
I added several lights to the scene and all they do is to make the black a little lighter. I think some material or texture information not loaded.
Is there anything I missed?
A tutorial/example for loading Duck.glb would be helpful for beginners.
Hi,
I've found that the command Mesh.from_trimesh()
seems to use all of my CPU cores, i.e. it uses some sort of multithreading trick to run faster. Problem is, I have a giant computer with 32 cores. So when I run this command repeatedly it uses all the cores to near 100% but doesn't run much faster, presumably because all the processing is being used to transfer information between cores rather than do something useful. I'd like to reduce this to 4 or 8 cores. Is there an easy way to do that?
Thanks!
Henry C.
Thanks!
Henry C.
Hi, thanks for this very useful library.
I was wondering if it is possible to use SpecularGlossinessMaterial
?
I have tried editing example.py
with:
bottle_material = pyrender.material.SpecularGlossinessMaterial()
bottle_mesh = Mesh.from_trimesh(bottle_trimesh, material=bottle_material)
but I receive an error:
Traceback (most recent call last):
File "example.py", line 131, in <module>
v = Viewer(scene, shadows=True)
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 347, in __init__
self._init_and_start_app()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 1003, in _init_and_start_app
pyglet.app.run()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/app/__init__.py", line 142, in run
event_loop.run()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/app/base.py", line 164, in run
self._legacy_setup()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/app/base.py", line 253, in _legacy_setup
window.dispatch_pending_events()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/window/xlib/__init__.py", line 877, in dispatch_pending_events
EventDispatcher.dispatch_event(self, *self._event_queue.pop(0))
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/event.py", line 370, in dispatch_event
if handler(*args):
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 606, in on_resize
self.on_draw()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 566, in on_draw
self._render()
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 985, in _render
self._renderer.render(self.scene, flags)
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/renderer.py", line 140, in render
retval = self._forward_pass(scene, flags)
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/renderer.py", line 345, in _forward_pass
primitive, flags, ProgramFlags.USE_MATERIAL
File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/renderer.py", line 952, in _get_primitive_program
elif isinstance(material, SpecularGlossinessMaterial):
NameError: name 'material' is not defined
Hi, @mmatl , I am not sure if I am using the orthographic camera in the right way. But I found that the value range of the depth map from orth. camera looks weird, it is not of physical unit.
When I set a perspective camera to scan a chair at distance around 1 meter, I can get a depth map which has value range between 0 and ~0.75. This is good, in meter.
However, when I set an orthographic camera to scan, like the code below:
camera = pyrender.OrthographicCamera(xmag=.5, ymag=.5, znear=0.05, zfar=100)
then, the value range of the depth map is like 0 - 0.05138555, which is obviously not physically meaningful, I have been looking into this for a while but still have no idea.
Can you please help me out. :)
Thanks!
Hi,
Suppose I have a mesh (e.g. a sphere), which I want to render while performing a series of (non-rigid) deformations (as an animation). I would like to do this as a series of snapshots of an OffscreenRenderer.
However, as soon as I render the first image, I cannot modify the mesh's positions without destroying the renderer and creating a new one. Is there a way to do this with the same renderer?
Here's what I have been doing:
# Create my scene, Create my mesh, and add to scene
renderer = OffscreenRenderer(viewport_width=640*2, viewport_height=480*2)
image_1, _ = renderer.render(self.scene)
# deform in some way the positions to animate
node.mesh.primitives[-1].positions = np.sin(node.primitives[-1].positions)
#node.primitives[-1].normals = # re-calculate normals
del renderer;
renderer = OffscreenRenderer(viewport_width=640*2, viewport_height=480*2)
image_2, _ = self.renderer.render(self.scene)
Without re-instancing the renderer, image 2 is no different from image 1. What should I do to properly update the positions?
Hi there, I'm running headless rendering on Ubuntu 14.04 and was wondering if it would be possible to host a .deb somewhere for OSMesa that works for that OS version.
I'm running in Docker and got everything to compile and install from source through my Dockerfile (the tricky part was finding the right apt
repositories to add to get llvm-6.0
and its deps to install), but the process causes my images to take a while to build. I've never created a .deb before and was wondering if anyone could post a recipe for how to do it, or better yet, has a script that's ready to go (hoping for @mmatl since you've already provided the deb for 16.04+).
Thanks, and sorry this is a bit off topic from the main business of the repo.
Relevant lines from my Dockerfile are:
# Copy and rename an apt lib file so that apt-add-repository works (cleaner way
# would be to symlink it but Dockerfiles don't seem to like symlinks). This is
# probably only necessary because there's something screwy going on with the
# different versions of Python installed on my image.
RUN cp /usr/lib/python3/dist-packages/apt_pkg.cpython-34m-x86_64-linux-gnu.so /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so
# Add new apt repositories and then apt-add some OSMesa deps
RUN apt-add-repository "deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-6.0 main"
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
RUN apt-get update && apt-get install --yes llvm-6.0 freeglut3 freeglut3-dev pkg-config
# Download OSMesa, then build and install it
RUN curl -o mesa-18.3.3.tar.gz ftp://ftp.freedesktop.org/pub/mesa/mesa-18.3.3.tar.gz
RUN tar xfv mesa-18.3.3.tar.gz
WORKDIR ./mesa-18.3.3
RUN ./configure --prefix=/usr/local \
--enable-opengl --disable-gles1 --disable-gles2 \
--disable-va --disable-xvmc --disable-vdpau \
--enable-shared-glapi \
--disable-texture-float \
--enable-gallium-llvm --enable-llvm-shared-libs \
--with-gallium-drivers=swrast,swr \
--disable-dri --with-dri-drivers= \
--disable-egl --with-egl-platforms= --disable-gbm \
--disable-glx \
--disable-osmesa --enable-gallium-osmesa \
ac_cv_path_LLVM_CONFIG=llvm-config-6.0
RUN make -j8
RUN make install
# Add some new environment variables so the OSMesa libs can be found
ENV MESA_HOME /usr/local
ENV LIBRARY_PATH $LIBRARY_PATH:$MESA_HOME/lib
ENV LD_LIBRARY_PATH $LD_LIBRARY_PATH:$MESA_HOME/lib
ENV C_INCLUDE_PATH $C_INCLUDE_PATH:$MESA_HOME/include/
ENV CPLUS_INCLUDE_PATH $CPLUS_INCLUDE_PATH:$MESA_HOME/include/
I just follow the document and install mesa deb in my headless ubuntu. After that, i install the fock PyOpengl and got the ERROR:
ERROR: pyrender 0.1.25 has requirement PyOpenGL==3.1.0, but you'll have pyopengl 3.1.3b2 which is incompatible.
(Maybe it is OK because your fock version is newer than 3.1)
But when i try using osmesa for offscreen render. I got the Error like this:
Traceback (most recent call last):
File "test.py", line 5, in
import pyrender
File "/usr/local/lib/python3.5/dist-packages/pyrender/init.py", line 3, in
from .light import Light, PointLight, DirectionalLight, SpotLight
File "/usr/local/lib/python3.5/dist-packages/pyrender/light.py", line 11, in
from .texture import Texture
File "/usr/local/lib/python3.5/dist-packages/pyrender/texture.py", line 8, in
from OpenGL.GL import *
File "/usr/local/lib/python3.5/dist-packages/OpenGL/GL/init.py", line 4, in
from OpenGL.GL.VERSION.GL_1_1 import *
File "/usr/local/lib/python3.5/dist-packages/OpenGL/GL/VERSION/GL_1_1.py", line 14, in
from OpenGL.raw.GL.VERSION.GL_1_1 import *
File "/usr/local/lib/python3.5/dist-packages/OpenGL/raw/GL/VERSION/GL_1_1.py", line 7, in
from OpenGL.raw.GL import _errors
File "/usr/local/lib/python3.5/dist-packages/OpenGL/raw/GL/_errors.py", line 4, in
_error_checker = _ErrorChecker( _p, _p.GL.glGetError )
AttributeError: 'NoneType' object has no attribute 'glGetError'
I have no idea how to solve this.
I load an .obj mesh without texture and try to add vertex colors, but the object is still rendered black. Why could this be?
Colab notebook:
https://github.com/JonathanLehner/Colab-collection/blob/master/python_render_obj.ipynb
Making this issue to track potential improvements to pyrender:
Hi @mmatl , I am trying to render 2d image with a given perspective using a given camera pose. The camera pose that I am using is :
camera pose : [[ 0.69210445 0.03735558 0.72083007 -0.48671459]
[ 0.009216 0.99812114 -0.06057437 -0.02976532]
[-0.72173852 0.04856696 0.69045982 0.42914029]
[ 0. 0. 0. 1. ]]
I have also attached the .ply file for your reference.
surface-L2-clean.ply.zip
Thanks.
Hi! I have multiple meshes forming a single scene. From these meshes, I want to render an instance mask containing the index of the mesh visible.
Here's an example of what I mean:
The scene is composed of different meshes (one for each object). The image on the left is an RGB image rendered from the scene, and the one on the right is an instance mask I
where I(x,y)
tells which object/mesh is visible at pixel (x,y)
. Here, the bed and guitar have different instance values.
I feel this should be possible by creating a custom texture for each object/mesh, but I'm not sure how exactly to do this. I'm currently rendering each object/mesh individually and comparing depth to create the instance mask but that's suboptimal.
Questions:
pyrender
let me create such instance masks directly?Such masks are fairly common for many computer vision tasks and I'm sure there are others wanting to do the same. I'd appreciate any help you can offer!
Thanks,
Shubham
when I try to render 3d file using the same code as in the example:
import trimesh
import pyrender
fuze_trimesh = trimesh.load('examples/models/fuze.obj')
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)
The 3d object is shown without the texture. But when I try it on the model 'fuze.obj' it works just fine.
What could be the problem?
Thanks for the great tool.
I am looking to capture a screenshot of my loaded model, ideally headless.
The goal is to use this to generate model thumbnails.
This is my simple code for loading:
import trimesh
import pyrender
fuze_trimesh = trimesh.load('cup.stl')
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)
I am on Windows 10.
Many thanks.
pip list:
PyOpenGL (3.1.0)
pyrender (0.1.24)
scipy (1.3.0)
Shapely (1.6.4.post2)
six (1.12.0)
trimesh (2.38.24)
I have a obj file from http://cvl-demos.cs.nott.ac.uk/vrn/
You can download and try this tool by yourself, after that you can download the .obj file
Source: https://specials-images.forbesimg.com/imageserve/5c76b4b84bbe6f24ad99c370/416x416.jpg?background=000000&cropX1=0&cropX2=4000&cropY1=0&cropY2=4000
bill-gates.obj (with VERTEX COLORS) contains something like that:
v 61.00 117.00 22.00 0.49 0.31 0.16
v 60.00 118.00 22.00 0.44 0.28 0.13
v 62.00 118.00 22.00 0.49 0.31 0.18
v 61.00 119.00 22.00 0.45 0.28 0.15
v 61.00 117.00 22.00 0.49 0.31 0.16
v 61.00 116.00 22.50 0.52 0.33 0.17
v 60.00 117.00 22.50 0.46 0.29 0.14
.... ....
.... ....
f 3630 2580 3628
f 2582 2580 3630
f 3632 2582 3630
f 3632 3635 2582
f 3635 2584 2582
f 3636 2585 2584
f 3636 2584 3637
f 3637 2584 3635
f 3637 3639 3638
f 3636 3637 3638
f 3639 3641 3640
f 3638 3639 3640
f 3641 3643 3642
f 3640 3641 3642
I have tried this code:
import trimesh
import pyrender
bill_trimesh = trimesh.load('bill-gates.obj')
mesh = pyrender.Mesh.from_trimesh(bill_trimesh, smooth=False, wireframe=True)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)
The face color is always gray.
It's not clear how to register and trigger events in the Viewer class
e.g. on_mouse_release, on_mouse_press, etc
Hi, thanks for setting up this amazing library!! I was wondering if its possible to render a scene without the need for adding lights. For example, since a mesh has an associated set of face colors and vertex colors, we should be able to directly render those RGB colors.
I noticed that OpenGL supported disabling lighting glDisable(GL_LIGHTING)
; however, I couldn't figure out how to add this functionality to pyrender -- in the even that you don't already support this :)
Hi! @mmatl , I am trying to load my own mesh, but I don't know how to choose a proper camera for pyrender. I use the camera_pose in example.py and find nothing in the rendered depth image.
What's more, I find the pyrender.Viewer shows with a good viewpoint, can I get its corresponding camera pose?
Thanks a lot!
thanks for the nice tool, who know render a scence with a transparent bgcolor
If you are interested in EGL headless check https://bitbucket.org/pyglet/pyglet/issues/219/egl-support-headless-rendering
Hi, thanks for the great project.
Is there currently an easy way to select which device to render on?
I tried changing the value passwd to EGL.eglGetDisplay()
from EGL_DEFAULT DEVICE to display_id = cast(1, EGL.NativeDisplayType)
but I get a segmentation fault instead.
I've also been trying to get device information via eglQueryDevicesEXT
but haven't had much luck.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.