yuphin / lumen Goto Github PK

A Vulkan Raytracing framework for various bidirectional path tracing techniques

License: MIT License

C++ 64.84% GLSL 2.71% C 30.37% Objective-C 1.79% Python 0.03% CMake 0.26%

bdpt mlt path-tracing raytracing restir vcm vulkan

lumen's Introduction

Lumen

Lumen is a Vulkan ray tracing framework that contains a variety of variance reduction techniques. Its aim is to accelerate some of the well-known rendering techniques in computer graphics literature with the help of the ray tracing hardware in modern GPUs. All the techniques Lumen implements run fully on the GPU.

Features

Rendering Techniques

Unidirectional Path Tracer with Multiple Importance Sampling (Path)
Bidirectional Path Tracer (BDPT)
Stochastic Progressive Photon Mapping (SPPM)
Vertex Connection and Merging (VCM)
Primary Sample Space Metropolis Light Transport (PSSMLT)
Combined VCM + MLT Integrator (VCMMLT)
ReSTIR
ReSTIR GI
ReSTIR PT / GRIS
DDGI (Real time)
FFT Convolution Bloom

Engine

Diffuse, mirror, glass, conductor, dielectric and Disney Principled BSDF
Spot (point), polygonal and directional lights
JSON-based scene system
Modified Mitsuba parser
A lightweight Vulkan abstraction layer from scratch
EXR output (F10)
On-the-fly RMSE computation
SPIRV reflection
Render graph support with experimental Vulkan features
- Automatic resource and synchronization management
- Binding inference based on shader reflection results
- Simple builder pattern

About experimental features

With the recently integrated render graph, Lumen uses some of the more experimental Vulkan features. These are namely,

Templated push descriptors (may be problematic with AMD)
Event-based syncronization (via syncronization2 API, available from Vulkan 1.3)
- Experimental feature, may not work depending on your driver (May need Vulkan beta drivers on Nvidia)
- Enabled via use_events flag in the Render Graph settings. (See RenderGraphSettings in RenderGraphTypes.h and RayTracer.cpp)

Showcase

Caustics Glass (VCM)

Caustics Zoomed (VCM)

Mitsuba Torus (VCM)

Japanese Classroom (Courtesy of Benedikt Bitterli)

ReSTIR 1 sample per pixel (Video comparison)

ReSTIR GI 1 sample per pixel (Video comparison)

Unbiased ReSTIR PT (aka GRIS) 1 sample per pixel (Indirect lighting only, on/off comparison)

Dynamic Diffuse Global Illumination (DDGI)

Building

To build Lumen, start cloning the repository with

git clone --recursive https://github.com/yuphin/Lumen.git

Requirements

Vulkan SDK
Turing+ or RDNA2 GPU
CMake 3.4 or above (Credits to @Lachei for porting the project to CMake and testing it on Linux)

Usage

Some of the sample scenes can be found in the scenes/ directory. Sample scene files with various integrators can be found in the scenes/cornell_box/ directory.

To load a scene file simply run:

Lumen.exe <scene_file>

Getting started with Lumen

The best way to get started is to take a look at the unidirectional path tracer implemented in src/Raytracer/Path.cpp and gradually explore the other integrators. From there, you can focus on the related shaders that are located in the src/shaders folder.

References

lumen's People

Contributors

Stargazers

Watchers

lumen's Issues

Bloom explosion : )

Enough time : )

This does not really affect me.

Missing Lumen.vcxproj.filters

Without this file, there's no organization in the Solution Explorer.

Strange behavour if #define ENABLE_DISNEY 1 and #define ENABLE_DISNEY 0

#define ENABLE_DISNEY 1
SPPM, small radius at start

Path

SPPM, small radius at start

Path

#define ENABLE_DISNEY 0
SPPM, bigger radius at start, reddish material

Path, reddish materail

SPPM, bigger radius at start, different material than with ENABLE_DISNEY 1

Path, brighter material

Add CMakeLists.txt

Most modern C/C++ projects switched to cmake based build system.

absense of GLASS material in ENABLE_DISNEY mode. and wrong SPPM after add it

I try to render torus scene in ENABLE_DISNEY mode with SPPM integrator and I see this

After investigation I add GLASS material and SPPM renders this

fixed with https://github.com/tigrazone/Lumen/commit/bb23059cbb4311e4277be0fca2f28ca953ecdf95

I just change to in torus.xml scene file
with VCM renders ok

SPPM integrator with ENABLE_DISNEY 0 mode gives me

different SPPM, BDPT, Restir GI, Restir PT vs VCM/Path

Hello.
In new bsdf branch I have different result when I change integrator.
VCM

Path

VCM and Path is almost same.

SPPM, darker

BDPT is broken

My camera settings for this view is

    "camera": {
        "fov": 45,
        "position": [
            -4.404,
            -0.887,
            10.913
        ],
        "dir" : [
            0,0.09, 2
        ]
    },

Crashes and errors

The release version (1 from git releases) crashes without any warnings. It says Logger initialized and window closes.

I build a release with VS 2022 and got this.

Console full of everything, like https://pastebin.com/DF19sUaZ . Never ending scroll.
And at last:

Unable to parse built-ins
ERROR: 0:362: 'double' : not supported with this profile: none
ERROR: 0:362: '' : compilation terminated
INTERNAL ERROR: Unable to parse built-ins
ERROR: 0:3: 'arrays of arrays' : not supported with this profile: none
ERROR: 0:3: 'arrays of arrays' : not supported with this profile: none
ERROR: 0:3: '' : compilation terminated
INTERNAL ERROR: Unable to parse built-ins

void subgroupMemoryBarrierShared();
void barrier();void memoryBarrierShared();void groupMemoryBarrier();void writePackedPrimitiveIndices4x8NV(uint, uint);
out gl_MeshPerVertexNV {vec4 gl_Position;float gl_PointSize;float gl_ClipDistance[];float gl_CullDistance[];perviewNV vec4 gl_PositionPerViewNV[];perviewNV float gl_ClipDistancePerViewNV[][];perviewNV float gl_CullDistancePerViewNV[][];} gl_MeshVerticesNV[];perprimitiveNV out gl_MeshPerPrimitiveNV {int gl_PrimitiveID;int gl_Layer;int gl_ViewportIndex;int gl_ViewportMask[];perviewNV int gl_LayerPerViewNV[];perviewNV int gl_ViewportMaskPerViewNV[][];} gl_MeshPrimitivesNV[];out uint gl_PrimitiveCountNV;out uint gl_PrimitiveIndicesNV[];in uint gl_MeshViewCountNV;in uint gl_MeshViewIndicesNV[4];const highp uvec3 gl_WorkGroupSize = uvec3(1,1,1);in highp uvec3 gl_WorkGroupID;in highp uvec3 gl_LocalInvocationID;in highp uvec3 gl_GlobalInvocationID;in highp uint gl_LocalInvocationIndex;
in highp int gl_DeviceIndex;in int gl_DrawIDARB;
in int gl_DrawID;
uniform uint gl_SubGroupSizeARB;in uint     gl_SubGroupInvocationARB;in uint64_t gl_SubGroupEqMaskARB;in uint64_t gl_SubGroupGeMaskARB;in uint64_t gl_SubGroupGtMaskARB;in uint64_t gl_SubGroupLeMaskARB;in uint64_t gl_SubGroupLtMaskARB;
in mediump uint  gl_SubgroupSize;in mediump uint  gl_SubgroupInvocationID;in highp   uvec4 gl_SubgroupEqMask;in highp   uvec4 gl_SubgroupGeMask;in highp   uvec4 gl_SubgroupGtMask;in highp   uvec4 gl_SubgroupLeMask;in highp   uvec4 gl_SubgroupLtMask;in highp   uint  gl_WarpsPerSMNV;in highp   uint  gl_SMCountNV;in highp   uint  gl_WarpIDNV;in highp   uint  gl_SMIDNV;
in highp   uint  gl_NumSubgroups;in highp   uint  gl_SubgroupID;
const highp int gl_ShadingRateFlag2VerticalPixelsEXT       = 1;
const highp int gl_ShadingRateFlag4VerticalPixelsEXT       = 2;
const highp int gl_ShadingRateFlag2HorizontalPixelsEXT     = 4;
const highp int gl_ShadingRateFlag4HorizontalPixelsEXT     = 8;

src/shaders/integrators/restir/gi/restir.rgen: error: #version: ray tracing shaders require non-es profile with version 460 or above
src/shaders/integrators/restir/gi/spatial_reuse.rgen: error: #version: ray tracing shaders require non-es profile with version 460 or above
src/shaders/ray.rahit: error: #version: ray tracing shaders require non-es profile with version 460 or above
src/shaders/ray.rchit: error: #version: ray tracing shaders require non-es profile with version 460 or above
src/shaders/integrators/restir/gi/temporal_reuse.rgen: error: #version: ray tracing shaders require non-es profile with version 460 or above
src/shaders/ray.rmiss: error: #version: ray tracing shaders require non-es profile with version 460 or above
src/shaders/post.frag: error: #version: Desktop shaders for Vulkan SPIR-V require version 140 or higher
src/shaders/post.vert: error: #version: Desktop shaders for Vulkan SPIR-V require version 140 or higher
src/shaders/ray_shadow.rmiss: error: #version: ray tracing shaders require non-es profile with version 460 or above
Failed to create shader module!
Failed to create shader module!
Failed to create shader module!
Failed to create shader module!
Failed to create shader module!
Unhandled image layout case

[Question] about restirgi

I am lookin at src\shaders\integrators\restir\gi\temporal_reuse.rgen and buffer old_samples, and I can not understand where its being written. Only accessed at line 97.

Error in glossy_f()

when we read https://www.pbr-book.org/3ed-2018/Reflection_Models/Fresnel_Incidence_Effects

Spectrum FresnelBlend::f(const Vector3f &wo, const Vector3f &wi) const {
    auto pow5 = [](Float v) { return (v * v) * (v * v) * v; };
    Spectrum diffuse = (28.f/(23.f*Pi)) * Rd *
        (Spectrum(1.f) - Rs) * 
        (1 - pow5(1 - .5f * AbsCosTheta(wi))) *
        (1 - pow5(1 - .5f * AbsCosTheta(wo)));
    Vector3f wh = wi + wo;
    if (wh.x == 0 && wh.y == 0 && wh.z == 0) return Spectrum(0);
    wh = Normalize(wh);
    Spectrum specular = distribution->D(wh) /
        (4 * AbsDot(wi, wh) *
         std::max(AbsCosTheta(wi), AbsCosTheta(wo))) *
         SchlickFresnel(Dot(wi, wh));
    return diffuse + specular;
}

and if we compare with glossy_f():

vec3 glossy_f(const Material mat, const vec3 wo, const vec3 wi, const vec3 n_s,
              float hl, float nl, float nv, float beckmann_term) {
    const vec3 h = normalize(wo + wi);
    const vec3 f_diffuse =
        (28. / (23 * PI)) * vec3(mat.albedo) * (1 - mat.metalness) *
        (1 - pow5(1 - 0.5 * dot(wi, n_s))) * (1 - pow5(1 - 0.5 * dot(wo, n_s)));
    const vec3 f_specular = beckmann_term *
                            fresnel_schlick(vec3(mat.metalness), hl) /
                            (4 * hl * max(nl, nv));
    return f_specular + f_diffuse;
}

(1 - mat.metalness) is Rd
mat.metalness is Rs
and then
right version of calc f_diffuse is

    const vec3 f_diffuse =
        (28. / (23 * PI)) * vec3(mat.albedo) * (1 - mat.metalness) * mat.metalness *
        (1 - pow5(1 - 0.5 * dot(wi, n_s))) * (1 - pow5(1 - 0.5 * dot(wo, n_s)));

Real working #define pow5(x) ((x)(x)(x)(x)(x)) can show you this error and my code is fixes it

Unknown constant

Hello.
I readed sources and I understand
const float cos_width = cos(PI / 6);
const float cos_faloff = cos(25 * PI / 180);
And PI / 6 is 30 * PI / 180 and 30 and 25 is hardcoded angles of light. Is this correct?
But I dont understand what is

Lumen/src/shaders/bsdf_commons.glsl

Line 102 in 051b10f

(28. / (23 * PI)) * vec3(mat.albedo) * (1 - mat.metalness) *

vcm resampled

please shortly explain how you use restir in vcm integrator

Investigate the slowdown with VCM

Compared to 2022 drivers, the current version of the VCM algorithm seems a lot slower (~%50% !). Investigate why.

Error in disney material rendering

With all integrators left sphere is black

White window border and VCMMLT issues

ENABLE_DISNEY mode
scenes\occluded2.json

Sometimes when integrator switch then window have white border and metallic sphere have white mirroring

same white border also if in scene file i switch from 'path' to 'vcm'
and when I switch integrator by select I have black window border and black 'backgroud light'

scenes\cornell_box\cornell_box_path.json

No sky with some integrators

Lumen.exe scenes/bedroom/scene.xml

Path: Bright sky through the window behind the curtain
BDPT/VCM/ReSTIRGI: Black sky

Thanks.

what BSDF_FLAG_GLOSSY do?

in previous bsdf implementation GLOSSY was used but in new implementation BSDF_FLAG_GLOSSY used only in
src\RayTracer\LumenScene.cpp
but nowhere in shaders.

is it issue?
how to fix it?

VCM is broken in rework branch

bsdf branch: strange radial pattern on checkerboard with vcm and bdpt integrators

vcm

bdpt

[Question] What atmosphere calculate algorithm is used?

How do I load complex scenes?

I can load some scenes, for example:
Lumen.exe scenes/occluded2.json

I can't load (the Tungsten file):
Lumen.exe scenes/bedroom/scene.json

Thanks.

[Question] MLT integrators

Hello.
Your renderer is impressive.
I have questions about MLT integrators.

WHY rendered too slow and calculate something too much time for every frame
SMLT is a "Selectively Metropolised Monte Carlo light transport simulation" by BENEDIKT BITTERLI and WOJCIECH JAROSZ, 2019?
VCMMLT is done?
Sometimes I have "Fence wait error" on my nvidia 3060

two-sided not processed or so

I do import of materials(maybe I do some errors - fix me) for bsdf branch
in a2b4d0c

bedroom scene

classroom

torus

computer shader version err

How do I add lights to an XML scene?

Yeah, it might be a Mitsuba question.

I took scenes/classroom/scene.xml and added the following:

	<emitter type="point">
		<point name="position" value="-3.7, 5.0, 1.0"/>
		<rgb name="intensity" value="123456789.0"/>
	</emitter>
	<shape type="rectangle">
		<transform name="toWorld">
			<matrix value="0.53241 0 0 -1.47544 0 0.907794 1.78252e-008 1.18968 0 1.78252e-008 0.5 -1.26735 0 0 0 1"/>
		</transform>
		<bsdf type="twosided">
			<bsdf type="diffuse">
				<rgb name="reflectance" value="0, 0, 0"/>
			</bsdf>
		</bsdf>
		<emitter type="area">
			<rgb name="radiance" value="123456789.0, 123456789.0, 123456789.0"/>
		</emitter>
	</shape>

There didn't seem to be new lights in the rendering.

scenes\bedroom\scene.xml contains 2 area emitters, which do not seem to appear in renderings.

Also didn't see sphere:

	<shape type="sphere">
		<float name="radius" value="10"/>
		<transform name="toWorld">
			<translate x="-3.6" y="5.0" z="1.4"/>
		</transform>
		<ref id="Walls" />
	</shape>

Thanks.

Disney material shader divergence

Currently enabling Disney materials (ENABLE_DISNEY macro in src/shaders/commons.h) results in significant performance loss compared to the non-Disney counterpart. This issue needs to be investigated.

SPPM fixed radius

Hello!
Please explain, why pc_ray.radius is not used in shaders but used fixed pc_ray.ppm_base_radius?
I see in code, radius is calculated

        pc_ray.ppm_base_radius = config->base_radius;
	const glm::vec3 diam = pc_ray.max_bounds - pc_ray.min_bounds;
	const float max_comp = glm::max(diam.x, glm::max(diam.y, diam.z));
	const int base_grid_res = int(max_comp / pc_ray.radius);
	pc_ray.grid_res = glm::max(ivec3(diam * float(base_grid_res) / max_comp), ivec3(1));
	if (pc_ray.radius < 1e-7f) {
		pc_ray.radius = 1e-7f;
	}
	auto op_reduce = [&](const std::string& op_name, const std::string& op_shader_name, const std::string& reduce_name,
						 const std::string& reduce_shader_name) {
		uint32_t num_wgs = uint32_t((instance->width * instance->height + 1023) / 1024);
		instance->vkb.rg->add_compute(op_name, {.shader = Shader(op_shader_name), .dims = {num_wgs, 1, 1}})
			.push_constants(&pc_ray)
			.bind(scene_desc_buffer)
			.zero({residual_buffer, counter_buffer});
		while (num_wgs != 1) {
			instance->vkb.rg->add_compute(reduce_name, {.shader = Shader(reduce_shader_name), .dims = {num_wgs, 1, 1}})
				.push_constants(&pc_ray)
				.bind(scene_desc_buffer);
			num_wgs = (uint32_t)(num_wgs + 1023) / 1024;
		}

code from src/RayTracer/SPPM.cpp lines 78-98.
Is possible to use calculated radius as start radius, not fixed ppm_base_radius?