In the get_projection_matrix method of IntrinsicsCamera there are the lines:

    P[0][2] = 1.0 - 2.0 * cx / (width - 1.0)
    P[1][2] = 2.0 * cy / (height - 1.0) - 1.0

I am unsure of why cx and cy are divided by width - 1.0/height - 1.0, and not just width/height. Specifying cx = 0.5 * width and cy = 0.5 * height should reproduce the behavior of PerspectiveCamera for those matrix entries, where those entries are 0.0 (which would occur if dividing by width/height, but not width - 1.0/height - 1.0. Is this perhaps a bug? Or maybe this is meant to address some pixel centering issue? When I modify to remove the -1s, I get the behavior I expect, but leaving them in produces renderings that are shifted with respect to what I expect.

Hey, I have the following setup..

• Object Pose is fixed (identitiy matrix)
• Camera is at camera_pose (identity matrix + translated on z axis)

..and im trying to convert image coordinates back to world coordinates the following way:

T_ext = camera_pose
T_int = camera.get_projection_matrix(w,h)
X_world = T_ext⁻¹ * T_int⁻¹ * X_img

Since it doesn't give any reasonable results there has to be some error in my approach. Would be very thankful if someone could help me out!

Are there certain parameters to specify when instantiating a pyrender.Mesh to cull back faces when rendering wireframes? Currently, it looks like the entire wireframe is rendered including faces in the back which should be culled if the SKIP_CULL_FACES flag is omitted.

I tried to run it with the view'er and it looks correct, but running it offscreen just gives the rgb values [38,38,38] where the object should be. The background has the correct color, and the depth image also looks correct.

For some reason, the shadow framebuffer gets corrupted randomly on MacOS in the current version. Unsure if this is a problem with Pyglet, PyOpenGL, or something to do with retina displays. Will investigate soon.

This is an example from cuda to opengl:
https://gist.github.com/victor-shepardson/5b3d3087dc2b4817b9bffdb8e87a57c4

Could be interesting to have the inverse in the example dir.

Many want to use the render for datasets generation so it could be useful to have a small example to have the render available in cuda directly with interop and without device to host and host to device double transfer.

I load an .obj mesh without texture and try to add vertex colors, but the object is still rendered black. Why could this be?

Colab notebook:
https://github.com/JonathanLehner/Colab-collection/blob/master/python_render_obj.ipynb

('Shader compile failure (0): 0(337) : error C1503: undefined variable "uv_0"\n', ['#version 330 core\n///////////////////////////////////////////////////////////////////////////////\n// Structs\n///////////////////////////////////////////////////////////////////////////////\n\nstruct SpotLight {\n vec3 color;\n float intensity;\n float range;\n vec3 position;\n vec3 direction;\n float light_angle_scale;\n float light_angle_offset;\n\n #if 0\n sampler2D shadow_map;\n mat4 light_matrix;\n #endif\n};\n\nstruct DirectionalLight {\n vec3 color;\n float intensity;\n vec3 direction;\n\n #if 0\n sampler2D shadow_map;\n mat4 light_matrix;\n #endif\n};\n\nstruct PointLight {\n vec3 color;\n float intensity;\n float range;\n vec3 position;\n\n #if 0\n samplerCube shadow_map;\n #endif\n};\n\nstruct Material {\n vec3 emissive_factor;\n\n#if 1\n vec4 base_color_factor;\n float metallic_factor;\n float roughness_factor;\n#endif\n\n#if 0\n vec4 diffuse_factor;\n vec3 specular_factor;\n float glossiness_factor;\n#endif\n\n#if 0\n sampler2D normal_texture;\n#endif\n#if 0\n sampler2D occlusion_texture;\n#endif\n#if 0\n sampler2D emissive_texture;\n#endif\n#if 1\n sampler2D base_color_texture;\n#endif\n#if 0\n sampler2D metallic_roughness_texture;\n#endif\n#if 0\n sampler2D diffuse_texture;\n#endif\n#if 0\n sampler2D specular_glossiness;\n#endif\n};\n...

Is there a way to get a point cloud of a scene?

If not, I think this would be a useful feature.

Thanks @mmatl for a great and lightweight rendering library. I just wanted to double check that our smoothing option is done correctly. Here's a minimal example, running this:

import trimesh
import pyrender
import numpy as np
import pickle

cloth_file = 'cloth_file.pkl'
with open(cloth_file, 'rb') as fh:
    cloth_data = pickle.load(fh)
cloth_pts = cloth_data['pts']
cloth = [[p.x, p.y, p.z] for p in cloth_pts]
cloth = np.array(cloth)  # shape (625,3)

# wh = width/height, both 25 in our case as the cloth is a 25x25 grid.
wh = 25
faces = []
for r in range(wh-1):
    for c in range(wh-1):
        pp = r*wh + c
        faces.append( [pp,   pp+wh, pp+1] )
        faces.append( [pp+1, pp+wh, pp+wh+1] )
faces = np.array(faces)
#print(faces.shape) (1152,3)

# Form the scene, with smoothing turned on.
tm = trimesh.Trimesh(vertices=cloth, faces=faces)
mesh = pyrender.Mesh.from_trimesh(tm, smooth=True)
scene = pyrender.Scene()
scene.add(mesh)
camera = pyrender.PerspectiveCamera(yfov=np.pi / 3.0, aspectRatio=1.0)
s = np.sqrt(2)/2
camera_pose = np.array([
   [0.0, -s,   s,   0.3],
   [1.0,  0.0, 0.0, 0.0],
   [0.0,  s,   s,   0.35],
   [0.0,  0.0, 0.0, 1.0],
])
scene.add(camera, pose=camera_pose)
light = pyrender.SpotLight(color=np.ones(3), intensity=3.0,
                           innerConeAngle=np.pi/16.0,
                           outerConeAngle=np.pi/6.0)
scene.add(light, pose=camera_pose)
pyrender.Viewer(scene, use_raymond_lighting=True)

I get this after rotating the viewer appropriately

Main question: is the smoothing set correctly? Or in other words, is this as smooth as we can realistically get with our 25x25 cloth representation?

I'm using Ubuntu 18.04, in a clean virtualenv with pip install pyrender.

It requires a pickle file at 'cloth_file.pkl'. Unfortunately I can't upload these on the issues tracker so I emailed it to you.

Making this issue to track potential improvements to pyrender:

• A deferred-rendering pipeline for improved performance with lots of meshes.
• An implementation of specular-glossiness materials in the shader.
• Cube maps for scene illumination.
• Shadow maps for point lighting.
• General performance improvements.

Hi, @mmatl , I am not sure if I am using the orthographic camera in the right way. But I found that the value range of the depth map from orth. camera looks weird, it is not of physical unit.

When I set a perspective camera to scan a chair at distance around 1 meter, I can get a depth map which has value range between 0 and ~0.75. This is good, in meter.

However, when I set an orthographic camera to scan, like the code below:
camera = pyrender.OrthographicCamera(xmag=.5, ymag=.5, znear=0.05, zfar=100)
then, the value range of the depth map is like 0 - 0.05138555, which is obviously not physically meaningful, I have been looking into this for a while but still have no idea.

Can you please help me out. :)
Thanks!

seems like it has been moved to material.py

https://github.com/mikedh/trimesh/blob/3.0.9/trimesh/visual/material.py

Hi there, I'm running headless rendering on Ubuntu 14.04 and was wondering if it would be possible to host a .deb somewhere for OSMesa that works for that OS version.

I'm running in Docker and got everything to compile and install from source through my Dockerfile (the tricky part was finding the right apt repositories to add to get llvm-6.0 and its deps to install), but the process causes my images to take a while to build. I've never created a .deb before and was wondering if anyone could post a recipe for how to do it, or better yet, has a script that's ready to go (hoping for @mmatl since you've already provided the deb for 16.04+).

Thanks, and sorry this is a bit off topic from the main business of the repo.

Relevant lines from my Dockerfile are:

# Copy and rename an apt lib file so that apt-add-repository works (cleaner way 
# would be to symlink it but Dockerfiles don't seem to like symlinks). This is 
# probably only necessary because there's something screwy going on with the 
# different versions of Python installed on my image.
RUN cp /usr/lib/python3/dist-packages/apt_pkg.cpython-34m-x86_64-linux-gnu.so /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so

# Add new apt repositories and then apt-add some OSMesa deps
RUN apt-add-repository "deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-6.0 main"
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
RUN apt-get update && apt-get install --yes llvm-6.0 freeglut3 freeglut3-dev pkg-config

# Download OSMesa, then build and install it
RUN curl -o mesa-18.3.3.tar.gz ftp://ftp.freedesktop.org/pub/mesa/mesa-18.3.3.tar.gz
RUN tar xfv mesa-18.3.3.tar.gz
WORKDIR ./mesa-18.3.3
RUN ./configure --prefix=/usr/local                           \
            --enable-opengl --disable-gles1 --disable-gles2   \
            --disable-va --disable-xvmc --disable-vdpau       \
            --enable-shared-glapi                             \
            --disable-texture-float                           \
            --enable-gallium-llvm --enable-llvm-shared-libs   \
            --with-gallium-drivers=swrast,swr                 \
            --disable-dri --with-dri-drivers=                 \
            --disable-egl --with-egl-platforms= --disable-gbm \
            --disable-glx                                     \
            --disable-osmesa --enable-gallium-osmesa          \
            ac_cv_path_LLVM_CONFIG=llvm-config-6.0
RUN make -j8
RUN make install

# Add some new environment variables so the OSMesa libs can be found
ENV MESA_HOME /usr/local
ENV LIBRARY_PATH $LIBRARY_PATH:$MESA_HOME/lib
ENV LD_LIBRARY_PATH $LD_LIBRARY_PATH:$MESA_HOME/lib
ENV C_INCLUDE_PATH $C_INCLUDE_PATH:$MESA_HOME/include/
ENV CPLUS_INCLUDE_PATH $CPLUS_INCLUDE_PATH:$MESA_HOME/include/

I just follow the document and install mesa deb in my headless ubuntu. After that, i install the fock PyOpengl and got the ERROR:
ERROR: pyrender 0.1.25 has requirement PyOpenGL==3.1.0, but you'll have pyopengl 3.1.3b2 which is incompatible.
(Maybe it is OK because your fock version is newer than 3.1)

But when i try using osmesa for offscreen render. I got the Error like this:
Traceback (most recent call last):
File "test.py", line 5, in
import pyrender
File "/usr/local/lib/python3.5/dist-packages/pyrender/init.py", line 3, in
from .light import Light, PointLight, DirectionalLight, SpotLight
File "/usr/local/lib/python3.5/dist-packages/pyrender/light.py", line 11, in
from .texture import Texture
File "/usr/local/lib/python3.5/dist-packages/pyrender/texture.py", line 8, in
from OpenGL.GL import *
File "/usr/local/lib/python3.5/dist-packages/OpenGL/GL/init.py", line 4, in
from OpenGL.GL.VERSION.GL_1_1 import *
File "/usr/local/lib/python3.5/dist-packages/OpenGL/GL/VERSION/GL_1_1.py", line 14, in
from OpenGL.raw.GL.VERSION.GL_1_1 import *
File "/usr/local/lib/python3.5/dist-packages/OpenGL/raw/GL/VERSION/GL_1_1.py", line 7, in
from OpenGL.raw.GL import _errors
File "/usr/local/lib/python3.5/dist-packages/OpenGL/raw/GL/_errors.py", line 4, in
_error_checker = _ErrorChecker( _p, _p.GL.glGetError )
AttributeError: 'NoneType' object has no attribute 'glGetError'

I have no idea how to solve this.

thanks for the nice tool, who know render a scence with a transparent bgcolor

Hi,

I've found that the command Mesh.from_trimesh() seems to use all of my CPU cores, i.e. it uses some sort of multithreading trick to run faster. Problem is, I have a giant computer with 32 cores. So when I run this command repeatedly it uses all the cores to near 100% but doesn't run much faster, presumably because all the processing is being used to transfer information between cores rather than do something useful. I'd like to reduce this to 4 or 8 cores. Is there an easy way to do that?

Thanks!
Henry C.

Thanks!
Henry C.

Hi,
First of all thanks for this tool.
I tried to follow offscreen renderering guide on a Google Colab notebook, but it failed, you can try it yourself live here (google account needed):

---------------------------------------------------------------------------
NoSuchDisplayException                    Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/pyrender/platforms.py in init_context(self)
    209                                                 resizable=False,
--> 210                                                 width=1, height=1)
    211         except Exception as e:

/usr/local/lib/python3.6/dist-packages/pyglet/window/xlib/__init__.py in __init__(self, *args, **kwargs)
    169 
--> 170         super(XlibWindow, self).__init__(*args, **kwargs)
    171 

/usr/local/lib/python3.6/dist-packages/pyglet/window/__init__.py in __init__(self, width, height, caption, resizable, style, fullscreen, visible, vsync, display, screen, config, context, mode)
    572         if not display:
--> 573             display = pyglet.canvas.get_display()
    574 

/usr/local/lib/python3.6/dist-packages/pyglet/canvas/__init__.py in get_display()
     94     # Otherwise, create a new display and return it.
---> 95     return Display()
     96 

/usr/local/lib/python3.6/dist-packages/pyglet/canvas/xlib.py in __init__(self, name, x_screen)
    118         if not self._display:
--> 119             raise NoSuchDisplayException('Cannot connect to "%s"' % name)
    120 

NoSuchDisplayException: Cannot connect to "None"

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-11-9ac112cbad03> in <module>()
     20                                innerConeAngle=np.pi/16.0)
     21 scene.add(light, pose=camera_pose)
---> 22 r = pyrender.OffscreenRenderer(400, 400)
     23 color, depth = r.render(scene)
     24 (PIL.Image.fromarray(color)).save('color.png')

/usr/local/lib/python3.6/dist-packages/pyrender/offscreen.py in __init__(self, viewport_width, viewport_height, point_size)
     30         self._platform = None
     31         self._renderer = None
---> 32         self._create()
     33 
     34     @property

/usr/local/lib/python3.6/dist-packages/pyrender/offscreen.py in _create(self)
    127                 os.environ['PYOPENGL_PLATFORM']
    128             ))
--> 129         self._platform.init_context()
    130         self._platform.make_current()
    131         self._renderer = Renderer(self.viewport_width, self.viewport_height)

/usr/local/lib/python3.6/dist-packages/pyrender/platforms.py in init_context(self)
    214                 'context. If you\'re logged in via SSH, ensure that you\'re '
    215                 'running your script with vglrun (i.e. VirtualGL). The '
--> 216                 'internal error message was "{}"'.format(e.message)
    217             )
    218 

AttributeError: 'NoSuchDisplayException' object has no attribute 'message'

What can I do to overpass this limitation? Thanks.

Hi,
Thanks for contributing this library.

I tried to render with objects in ply file format with following codes:

obj1_trimesh = trimesh.load('./models/obj_01.ply')
obj1_mesh = Mesh.from_trimesh(obj1_trimesh)
scene.add(obj1_mesh)

However, nothing appeared in the rendered images.
The zip format of ply file is attached.
obj_01.ply.zip

Could you please give some advice?
Thanks in advance!

Best,
Stacey

Hi
I am using this awesome repo alongside trimesh, but the run_in_thread does not seem to work on macos.
It seems to not work on the sample provided in the docs, and I get something like below.

My environment is
10.14.2 Mojave
xcode 10.2
just in case.

2019-05-16 17:18:54.912 python[10096:8443809] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff2abf0ded __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff56cbc720 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff2ac0a87d -[NSException raise] + 9
	3   AppKit                              0x00007fff2810bc8e -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 324
	4   AppKit                              0x00007fff2810907c -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1488
	5   AppKit                              0x00007fff28108aa6 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 45
	6   _ctypes.cpython-37m-darwin.so       0x000000010ebb8377 ffi_call_unix64 + 79
	7   ???                                 0x000070000c8c8460 0x0 + 123145512846432
)
libc++abi.dylib: terminating with uncaught exception of type NSException

Hi, I'm getting a TypeError: Expected a Trimesh or a list, got a <class 'trimesh.scene.scene.Scene'> when importing a .glb file. Is there any way to go around this?

import trimesh
import pyrender
fuz_trimesh = trimesh.load("donut_calibrate2_forgltf.glb")
mesh = pyrender.Mesh.from_trimesh(fuz_trimesh)

I'm trying to render the Khronos GLTF example https://github.com/KhronosGroup/glTF-Sample-Models/blob/master/2.0/Duck/glTF-Binary/Duck.glb with pyrender:

1. I'm using trimesh.load on the glb file
2. Iterate over mesh.geometry.values() and add them to the scene

Resulting image:

Note: This is the color image, not the depth image.

I added several lights to the scene and all they do is to make the black a little lighter. I think some material or texture information not loaded.

Is there anything I missed?

A tutorial/example for loading Duck.glb would be helpful for beginners.

pip list:

PyOpenGL (3.1.0)
pyrender (0.1.24)
scipy (1.3.0)
Shapely (1.6.4.post2)
six (1.12.0)
trimesh (2.38.24)

I have a obj file from http://cvl-demos.cs.nott.ac.uk/vrn/

You can download and try this tool by yourself, after that you can download the .obj file
Source: https://specials-images.forbesimg.com/imageserve/5c76b4b84bbe6f24ad99c370/416x416.jpg?background=000000&cropX1=0&cropX2=4000&cropY1=0&cropY2=4000

bill-gates.obj (with VERTEX COLORS) contains something like that:

v 61.00 117.00 22.00 0.49 0.31 0.16
v 60.00 118.00 22.00 0.44 0.28 0.13
v 62.00 118.00 22.00 0.49 0.31 0.18
v 61.00 119.00 22.00 0.45 0.28 0.15
v 61.00 117.00 22.00 0.49 0.31 0.16
v 61.00 116.00 22.50 0.52 0.33 0.17
v 60.00 117.00 22.50 0.46 0.29 0.14
.... ....
.... ....
f 3630 2580 3628
f 2582 2580 3630
f 3632 2582 3630
f 3632 3635 2582
f 3635 2584 2582
f 3636 2585 2584
f 3636 2584 3637
f 3637 2584 3635
f 3637 3639 3638
f 3636 3637 3638
f 3639 3641 3640
f 3638 3639 3640
f 3641 3643 3642
f 3640 3641 3642

I have tried this code:

import trimesh
import pyrender

bill_trimesh = trimesh.load('bill-gates.obj')
mesh = pyrender.Mesh.from_trimesh(bill_trimesh, smooth=False, wireframe=True)

scene = pyrender.Scene()
scene.add(mesh)

pyrender.Viewer(scene, use_raymond_lighting=True)

The face color is always gray.

1. How to extract vertex colors?
2. How to apply this vertex colors?

Hi! @mmatl , I am trying to load my own mesh, but I don't know how to choose a proper camera for pyrender. I use the camera_pose in example.py and find nothing in the rendered depth image.

What's more, I find the pyrender.Viewer shows with a good viewpoint, can I get its corresponding camera pose?

Thanks a lot!

I tried to run pyrender on the Leonard cluster at ETH Zurich.
https://scicomp.ethz.ch/wiki/Leonhard_applications_and_libraries

I get the following error:

File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 351, in getattr
return getattr(self._module, name)
AttributeError: 'NoneType' object has no attribute 'Window'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 351, in getattr
return getattr(self._module, name)
AttributeError: 'NoneType' object has no attribute '_create_shadow_window'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "full_pipeline.py", line 354, in
import pyrender
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyrender/init.py", line 12, in
from .viewer import Viewer
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyrender/viewer.py", line 36, in
class Viewer(pyglet.window.Window):
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 357, in getattr
import(import_name)
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/window/init.py", line 1872, in
gl._create_shadow_window()
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/init.py", line 357, in getattr
import(import_name)
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/gl/init.py", line 100, in
from pyglet.gl.lib import GLException
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/gl/lib.py", line 143, in
from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/gl/lib_glx.py", line 51, in
glu_lib = pyglet.lib.load_library('GLU')
File "/cluster/home/lehnerj/.local/share/virtualenvs/wearable3dhandreconstruction--fu_gjdO/lib/python3.6/site-packages/pyglet/lib.py", line 168, in load_library
raise ImportError('Library "%s" not found.' % names[0])
ImportError: Library "GLU" not found.

Hi @mmatl , I am using the pyrender to render some point clouds in off-screen mode.

I am wondering if it is possible to render images with alpha channel?

Thanks!

If you are interested in EGL headless check https://bitbucket.org/pyglet/pyglet/issues/219/egl-support-headless-rendering

The documentation of IntrinsicsCamera states that the fx and fy input parameters are in units of meters. However, it is difficult for me to understand what that means, since to understand focal length in meter units means that you also know the size of the sensor (or image plane) in meters as well... instead, it seems from my tests that fx and fy are actually meant to be specified in pixel units. Is that correct?

I tried to render a high resolution depth map but get really coarse result (among 100000 pixels there are only 9 different values).
I noticed that pyrender gets depth from the z-buffer, and performs multiple divisions, that will lead to a lot of numerical loss.
A better solution is to record exact depth using another texture in shaders.

Hi, thanks for this very useful library.

I was wondering if it is possible to use SpecularGlossinessMaterial?

I have tried editing example.py with:

bottle_material = pyrender.material.SpecularGlossinessMaterial()
bottle_mesh = Mesh.from_trimesh(bottle_trimesh, material=bottle_material)

but I receive an error:

Traceback (most recent call last):
  File "example.py", line 131, in <module>
    v = Viewer(scene, shadows=True)
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 347, in __init__
    self._init_and_start_app()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 1003, in _init_and_start_app
    pyglet.app.run()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/app/__init__.py", line 142, in run
    event_loop.run()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/app/base.py", line 164, in run
    self._legacy_setup()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/app/base.py", line 253, in _legacy_setup
    window.dispatch_pending_events()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/window/xlib/__init__.py", line 877, in dispatch_pending_events
    EventDispatcher.dispatch_event(self, *self._event_queue.pop(0))
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyglet/event.py", line 370, in dispatch_event
    if handler(*args):
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 606, in on_resize
    self.on_draw()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 566, in on_draw
    self._render()
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/viewer.py", line 985, in _render
    self._renderer.render(self.scene, flags)
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/renderer.py", line 140, in render
    retval = self._forward_pass(scene, flags)
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/renderer.py", line 345, in _forward_pass
    primitive, flags, ProgramFlags.USE_MATERIAL
  File "/home/damien/venv/intrinsic_behav/lib/python3.7/site-packages/pyrender/renderer.py", line 952, in _get_primitive_program
    elif isinstance(material, SpecularGlossinessMaterial):
NameError: name 'material' is not defined

I am getting this error in the created thread when setting run_in_thread=True in pyrender.Viewer

Exception in thread Thread-6:
Traceback (most recent call last):
File "\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "\lib\site-packages\pyrender\viewer.py", line 1003, in init_and_start_app
pyglet.app.run()
File "\lib\site-packages\pyglet\app_init.py", line 142, in run
event_loop.run()
File "\lib\site-packages\pyglet\app\base.py", line 167, in run
platform_event_loop.start()
File "\lib\site-packages\pyglet\app\win32.py", line 96, in start
'thread that imports pyglet.app')
RuntimeError: EventLoop.run() must be called from the same thread that imports pyglet.app

This might have something to do with it https://stackoverflow.com/questions/27071801/can-pyglet-run-in-new-thread

I have not seen any other problems except not being able to set os.environ['PYOPENGL_PLATFORM'] = 'egl' for offline rendering.

My environment:
Windows 10 v1809
Python 3.6.8 :: Anaconda, Inc.
imageio 2.5.0
networkx 2.2
numpy 1.16.2
Pillow 5.4.1
pyglet 1.4.0b1
PyOpenGL 3.1.0
PyOpenGL-accelerate 3.1.0
pyrender 0.1.16
trimesh 2.37.5

Hi @mmatl , after I finished the steps needed for offscreen render, I ran the example script, and I got this, do you have any clue for the reason? Thanks!

FYI, I am using docker, ubuntu 16, a headless server connected over SSH.

root@dffaaaba9cf1:/workspace/mesh_render# PYOPENGL_PLATFORM=osmesa python meshrender_example.py
No handlers could be found for logger "OpenGL.acceleratesupport"
Traceback (most recent call last):
File "meshrender_example.py", line 136, in
r = OffscreenRenderer(viewport_width=6402, viewport_height=4802)
File "/usr/local/lib/python2.7/dist-packages/pyrender/offscreen.py", line 28, in init
self._create()
File "/usr/local/lib/python2.7/dist-packages/pyrender/offscreen.py", line 102, in _create
self._platform.make_current()
File "/usr/local/lib/python2.7/dist-packages/pyrender/platforms.py", line 216, in make_current
self.viewport_width, self.viewport_height))
AssertionError

I have a mesh stored in .obj format, the relative positions of different objects in the mesh are shown big difference, who can give me some advice? Thanks in advance~

Hello,

I'm trying to render the depth image from my point cloud. Below is the outline of code snippet i have tried.

fuze_trimesh = trimesh.load('.../teddyUpright2.ply')

#camera pose
transform_mat = np.linalg.inv((np.loadtxt(".../frame-000000.pose.txt", delimiter=' ')))

cam_intrinsics = np.loadtxt(.../camera-intrinsics.txt", delimiter=' ')

fuze_trimesh.apply_transform(transform_mat)

mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)

scene = pyrender.Scene()

scene.add(mesh, pose=np.eye(4))

camera = pyrender.IntrinsicsCamera(cam_intrinsics[0, 0], cam_intrinsics[1, 1], cam_intrinsics[0, 2], cam_intrinsics[1, 2], 0.0001, 10000) 

camera_pose = np.eye(4)
camera_pose[2, 3] = 0.6

scene.add(camera, pose=camera_pose)

pyrender.Viewer(scene)

r = pyrender.OffscreenRenderer(640, 480)

color, depth = r.render(scene)

Now the actual issue:

The size of the object in the rendered depth image is very small as compared to the size of the object in image and even in imported mesh.

When i check the scale of the fuze_trimesh it is around 0.3045. i'm not able to figure out why the rendered depth image has small size?

any suggestions on why this is happening and how to solve this?

Hi,
in Scene, bg_color has the following docstring:

bg_color ((4,) float, optional) – Background color of scene.

However later in the documentation is estipulated that it should be:

Type: | (3,) float

Should it be 3 or 4?

P.S.
Great project :)!

Hey,

I am quiet new to pyrender but I get an error that I do not know what is causing it when I run the tutorial code. The error I get is for the command pyrender.Viewer(scene), and it says

"Require WGL_ARB_create_context extension to create OpenGL 3 contexts". I am not sure how to proceed with this as I do not know what to do to fix this.

Hi, thanks for setting up this amazing library!! I was wondering if its possible to render a scene without the need for adding lights. For example, since a mesh has an associated set of face colors and vertex colors, we should be able to directly render those RGB colors.

I noticed that OpenGL supported disabling lighting glDisable(GL_LIGHTING); however, I couldn't figure out how to add this functionality to pyrender -- in the even that you don't already support this :)

Hi, thanks for the great project.

Is there currently an easy way to select which device to render on?

I tried changing the value passwd to EGL.eglGetDisplay() from EGL_DEFAULT DEVICE to display_id = cast(1, EGL.NativeDisplayType) but I get a segmentation fault instead.

I've also been trying to get device information via eglQueryDevicesEXT but haven't had much luck.

Hi,
Given a mesh and camera in a given pose and location; is there a function that I can use to compute which faces/vertices of the mesh are visible from the specified camera view?
I understand that this should anyway be computed under the hood during the rendering process.

Thanks in advance.

Hi,

Suppose I have a mesh (e.g. a sphere), which I want to render while performing a series of (non-rigid) deformations (as an animation). I would like to do this as a series of snapshots of an OffscreenRenderer.
However, as soon as I render the first image, I cannot modify the mesh's positions without destroying the renderer and creating a new one. Is there a way to do this with the same renderer?

Here's what I have been doing:

# Create my scene, Create my mesh, and add to scene

renderer = OffscreenRenderer(viewport_width=640*2, viewport_height=480*2)
image_1, _ = renderer.render(self.scene)

# deform in some way the positions to animate
node.mesh.primitives[-1].positions = np.sin(node.primitives[-1].positions)
#node.primitives[-1].normals = # re-calculate normals

del renderer;
renderer = OffscreenRenderer(viewport_width=640*2, viewport_height=480*2)
image_2, _ = self.renderer.render(self.scene)

Without re-instancing the renderer, image 2 is no different from image 1. What should I do to properly update the positions?

Hello,
I tried your example code to render the object in color and depth but can't get the correct result (see screenshot). In the depth image, the background is white (Value 0) and the Object has values between 1.13 and 1.2 - although it should be a float32 image. Do I have to do a normalization or something to get the correct result?
Thank you in advance!

Hi, I'm trying to run the following code:

import trimesh
import pyrender
tm = trimesh.load('/home/henry/Downloads/fuze.obj')
pymesh = pyrender.Mesh.from_trimesh(tm)

scene = pyrender.Scene()
scene.add(pymesh)
pyrender.Viewer(scene, use_raymond_lighting=True)

And I get the following error:

File "generate_pose_dataset.py", line 172, in mesh_render
    pyrender.Viewer(scene, use_raymond_lighting=True)
  File "/usr/local/lib/python2.7/dist-packages/pyrender/viewer.py", line 347, in __init__
    self._init_and_start_app()
  File "/usr/local/lib/python2.7/dist-packages/pyrender/viewer.py", line 995, in _init_and_start_app
    height=self._viewport_size[1])
  File "/usr/local/lib/python2.7/dist-packages/pyglet/window/xlib/__init__.py", line 170, in __init__
    super(XlibWindow, self).__init__(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyglet/window/__init__.py", line 595, in __init__
    context = config.create_context(gl.current_context)
  File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/xlib.py", line 216, in create_context
    return XlibContextARB(self, share)
  File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/xlib.py", line 322, in __init__
    super(XlibContext13, self).__init__(config, share)
  File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/xlib.py", line 230, in __init__
    raise gl.ContextException('Could not create GL context')
pyglet.gl.ContextException: Could not create GL context

My computer shows that PyOpenGL 3.1.0, PyOpenGL_accelerate 3.1.0, and pyglet appear to be correctly installed.

To test pyglet externally from pyrender, I ran the following code:

import pyglet
pyglet.window.Window()

and a window opens as expected.

Any ideas why this error is occurring?

I'm running 16.04 with 64 bit.

Thanks!
Henry C.

I am running the following basic example:

import trimesh
import pyrender
fuze_trimesh = trimesh.load(OBJ FILE)
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)

and I encounter this error on a Mac:

No handlers could be found for logger "trimesh"
Traceback (most recent call last):
  File "scripts/pyrender-trail.py", line 41, in <module>
    pyrender.Viewer(scene, use_raymond_lighting=True)
  File "/anaconda2/lib/python2.7/site-packages/pyrender/viewer.py", line 347, in __init__
    self._init_and_start_app()
  File "/anaconda2/lib/python2.7/site-packages/pyrender/viewer.py", line 997, in _init_and_start_app
    raise ValueError('Unable to initialize an OpenGL 3+ context')
ValueError: Unable to initialize an OpenGL 3+ context

when I try to render 3d file using the same code as in the example:

import trimesh
import pyrender
fuze_trimesh = trimesh.load('examples/models/fuze.obj')
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)
The 3d object is shown without the texture. But when I try it on the model 'fuze.obj' it works just fine.
What could be the problem?

Hi， I follow the step to install.But when I run the example , I got the bug "pyglet.window.NoSuchConfigException".
If you know why, please tell me how to solve it .
Thx your contribute.

Thanks for the great tool.

I am looking to capture a screenshot of my loaded model, ideally headless.

The goal is to use this to generate model thumbnails.

This is my simple code for loading:

import trimesh
import pyrender
fuze_trimesh = trimesh.load('cup.stl')
mesh = pyrender.Mesh.from_trimesh(fuze_trimesh)
scene = pyrender.Scene()
scene.add(mesh)
pyrender.Viewer(scene, use_raymond_lighting=True)

I am on Windows 10.
Many thanks.

Using the simple offscreen rendering example, my models have incorrect normals (see screenshot on the right).
If I display the mesh with trimesh's mesh.show(), it looks correct (left part of the screenshot).
This happens with all models that I tested and even if I supply a mesh without normals.

It's not clear how to register and trigger events in the Viewer class
e.g. on_mouse_release, on_mouse_press, etc

Hi @mmatl , I am trying to render 2d image with a given perspective using a given camera pose. The camera pose that I am using is :

camera pose : [[ 0.69210445 0.03735558 0.72083007 -0.48671459]
[ 0.009216 0.99812114 -0.06057437 -0.02976532]
[-0.72173852 0.04856696 0.69045982 0.42914029]
[ 0. 0. 0. 1. ]]

I have also attached the .ply file for your reference.
surface-L2-clean.ply.zip

Thanks.

Hi! I have multiple meshes forming a single scene. From these meshes, I want to render an instance mask containing the index of the mesh visible.

Here's an example of what I mean:

The scene is composed of different meshes (one for each object). The image on the left is an RGB image rendered from the scene, and the one on the right is an instance mask I where I(x,y) tells which object/mesh is visible at pixel (x,y). Here, the bed and guitar have different instance values.

I feel this should be possible by creating a custom texture for each object/mesh, but I'm not sure how exactly to do this. I'm currently rendering each object/mesh individually and comparing depth to create the instance mask but that's suboptimal.

Questions:

1. Can pyrender let me create such instance masks directly?
2. If yes, how do I do it?

Such masks are fairly common for many computer vision tasks and I'm sure there are others wanting to do the same. I'd appreciate any help you can offer!

Thanks,
Shubham