Comments (13)
I recently had to generate a normal buffer with pyrender, so here is what I did:
Define a custom vertex shader in shaders/mesh.vert
:
#version 330 core
// Vertex Attributes
layout(location = 0) in vec3 position;
layout(location = NORMAL_LOC) in vec3 normal;
layout(location = INST_M_LOC) in mat4 inst_m;
// Uniforms
uniform mat4 M;
uniform mat4 V;
uniform mat4 P;
// Outputs
out vec3 frag_position;
out vec3 frag_normal;
void main()
{
gl_Position = P * V * M * inst_m * vec4(position, 1);
frag_position = vec3(M * inst_m * vec4(position, 1.0));
mat4 N = transpose(inverse(M * inst_m));
frag_normal = normalize(vec3(N * vec4(normal, 0.0)));
}
Define a custom fragment shader in shaders/mesh.frag
:
#version 330 core
in vec3 frag_position;
in vec3 frag_normal;
out vec4 frag_color;
void main()
{
vec3 normal = normalize(frag_normal);
frag_color = vec4(normal * 0.5 + 0.5, 1.0);
}
Now you can generate a normal buffer like this:
import trimesh
import pyrender
import numpy as np
from PIL import Image
class CustomShaderCache():
def __init__(self):
self.program = None
def get_program(self, vertex_shader, fragment_shader, geometry_shader=None, defines=None):
if self.program is None:
self.program = pyrender.shader_program.ShaderProgram("shaders/mesh.vert", "shaders/mesh.frag", defines=defines)
return self.program
camera_pose = np.array(
[[ 1, 0, 0, 0],
[ 0, 0, -1, -4],
[ 0, 1, 0, 0],
[ 0, 0, 0, 1]]
)
scene = pyrender.Scene(bg_color=(0, 0, 0))
scene.add(pyrender.Mesh.from_trimesh(trimesh.primitives.Capsule(), smooth = False))
camera = pyrender.PerspectiveCamera(yfov=np.pi / 3.0, aspectRatio=1.0, znear = 0.5, zfar = 40)
scene.add(camera, pose=camera_pose)
renderer = pyrender.OffscreenRenderer(512, 512)
renderer._renderer._program_cache = CustomShaderCache()
normals, depth = renderer.render(scene)
world_space_normals = normals / 255 * 2 - 1
image = Image.fromarray(normals, 'RGB')
image.show()
Now if you want a normal buffer it might be useful to disable antialiasing. To do that, I changed renderer.py
of pyrender, which in my case was in /usr/local/lib/python3.5/dist-packages/pyrender/renderer.py
. Here are my changes:
# Clear it
glClearColor(*scene.bg_color)
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
glDisable(GL_MULTISAMPLE) ###### change here
# .....
# Generate multisample buffer
self._main_cb_ms, self._main_db_ms = glGenRenderbuffers(2)
glBindRenderbuffer(GL_RENDERBUFFER, self._main_cb_ms)
glRenderbufferStorage( ###### change here
GL_RENDERBUFFER, GL_RGBA, ###### change here
self.viewport_width, self.viewport_height
)
glBindRenderbuffer(GL_RENDERBUFFER, self._main_db_ms)
glRenderbufferStorage( ###### change here
GL_RENDERBUFFER, GL_DEPTH_COMPONENT24, ###### change here
self.viewport_width, self.viewport_height
)
Maybe @mmatl could add an option to pyrender to disable antialiasing.
from pyrender.
Thanks for your detailed description @marian42 ! Works like a charm
from pyrender.
Thanks @marian42, still works out of the box! One small addition: The CustomShaderCache
might not (no longer?) be necessary. Simply point shader_dirs
of the pyrender ShaderProgramCache
class to the directory where you have created your shader files:
import pyrender
from pyrender.shader_program import ShaderProgramCache
renderer = pyrender.OffscreenRenderer(640, 480)
renderer._renderer._program_cache = ShaderProgramCache(shader_dir="shaders")
from pyrender.
Ok, I figured out that I need to supply the smooth
option:
pyrender.Mesh.from_trimesh(mesh, smooth = False)
.
You might want to change the default for that since most users probably don't want to have their model smoothened.
from pyrender.
@marian42, may I ask how to get normal image using pyrender?
Thanks.
from pyrender.
Are you looking for a normal buffer of the rendered frame?
from pyrender.
Thanks for your reply @marian42.
Both normal buffer and normal map.
from pyrender.
I'll have a try.
Thank you very much for help @marian42.
from pyrender.
@LittleYuer map points to it's norms, and then render color image
from pyrender.
One more addition: if you're on Mac, you'll also need to change this in render.py
:
# Resize for macos if needed
if sys.platform == 'darwin':
color_im = self._resize_image(color_im, False) # Change last parameter (anti-alias) to False.
from pyrender.
This is a very helpful thread. This is the second time I am referring to this thread, this time for a different project. It will be very useful if the developers can bring this as a feature.
Maybe we can add a RenderFlag called "DISABLE_ANTI_ALIASING" and then use that to condition multi-sample in a render call. (E,g,: #175). But, this mechanism cannot handle the macOS specific cases mentioned above by @ndahlquist.
from pyrender.
Thanks for your scripts @marian42 ! Would you pls tell me the coordinate of your output normal map? world space or camera space?
from pyrender.
I cannot compile the shader:
RuntimeError: ("Shader compile failure (0): b'0:5(8): error: compile-time constant expressions require GLSL 4.40 or ARB_enhanced_layouts\\n0:6(8): error: compile-time constant expressions require GLSL 4.40 or ARB_enhanced_layouts\\n'", [b'#version 330 core\n\n// Vertex Attributes\nlayout(location = 0) in vec3 position;\nlayout(location = NORMAL_LOC) in vec3 normal;\nlayout(location = INST_M_LOC) in mat4 inst_m;\n\n// Uniforms\nuniform mat4 M;\nuniform mat4 V;\nuniform mat4 P;\n\n// Outputs\nout vec3 frag_position;\nout vec3 frag_normal;\n\nvoid main()\n{\n gl_Position = P * V * M * inst_m * vec4(position, 1);\n frag_position = vec3(M * inst_m * vec4(position, 1.0));\n\n mat4 N = transpose(inverse(M * inst_m));\n frag_normal = normalize(vec3(N * vec4(normal, 0.0)));\n}'], GL_VERTEX_SHADER)
Any ideas?
from pyrender.
Related Issues (20)
- BROKEN LINK: Google colab notebooks HOT 1
- Invalid device ID (0), Pyrender NOT working on Python 3.10 HOT 5
- Inverted triangles?
- How to retrieve intrinsic camera parameters from PerspectiveCamera
- Textures rendering slanted
- Sphere Rendered with Non-reflective Material Not Appearing Black
- offscreen error
- 'Viewer' object has no attribute 'render'. Did you mean: '_render'?
- Interpolation artifacts in segmentation masks.
- Unable to get color on running the bottle example code HOT 1
- Wrong depth rendering in WSL-ubuntu environment
- Path3D type error when loading trimesh scene
- GLB is not rendered correctly HOT 1
- How to properly merge the Scenes from multiple trimeshes?
- Request for Release of Tag 0.1.46 with auto_start Option
- Cannot render SpecularGlossinessMaterial HOT 1
- jupyter notebook support?
- Attempt to call an undefined function glGenVertexArrays, check for bool(glGenVertexArrays) before calling HOT 3
- OpenGL.error.NullFunctionError: Attempt to call an undefined function glGenVertexArrays, check for bool(glGenVertexArrays) before calling
- Change camera pose to identity matrix (np.eye(4)) and move object to get same result HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyrender.