Giter VIP home page Giter VIP logo

amber's Introduction

Amber

Amber is a multi-API shader test framework.

Amber lets you capture and communicate shader bugs with the fluidity and ease of a scripting flow:

  • No graphics API programming is required.
    • Supports Vulkan graphics API
    • WIP: Supports Dawn graphics API
  • A single text string (or file) maps to a single graphics API pipeline test case. The text includes:
    • Input data, including buffers and images.
    • Shaders.
    • Expectations for the result of running the pipeline.
  • Shaders can be expressed in binary form (as hex), in SPIR-V assembly, or in a higher level shader language.
  • After executing the pipeline, result buffers and images can be saved to output files.

Amber is influenced by Talvos and VkRunner. The VkScript syntax matches the format used by VkRunner.

This is not an officially supported Google product.

Writing Amber Tests

Working with Amber involves writing input test files. Some example files can be see in the tests/cases folder.

The main input format is Amberscript. New features are added to AmberScript as Amber is enhanced. This is the preferred format in which new script files are written.

Clear test as AmberScript

SHADER vertex vtex_shader PASSTHROUGH
SHADER fragment frag_shader GLSL
#version 430

layout(location = 0) in vec4 color_in;
layout(location = 0) out vec4 color_out;

void main() {
  color_out = color_in;
}
END

BUFFER img_buf FORMAT B8G8R8A8_UNORM

PIPELINE graphics my_pipeline
  ATTACH vtex_shader
  ATTACH frag_shader

  FRAMEBUFFER_SIZE 256 256
  BIND BUFFER img_buf AS color LOCATION 0
END

CLEAR my_pipeline
EXPECT img_buf IDX 0 0 SIZE 256 256 EQ_RGBA 0 0 0 0

The VkScript format is supported for historic reasons. It is based off, and very closely matches, the format accepted by VkRunner. There are no new features being added to VkScript, it is for historical use.

Clear test as VkScript

[require]
VK_KHR_get_physical_device_properties2

[vertex shader passthrough]

[fragment shader]
#version 430

layout(location = 0) in vec4 color_in;
layout(location = 0) out vec4 color_out;

void main() {
  color_out = color_in;
}

[test]
clear
relative probe rect rgba (0.0, 0.0, 1.0, 1.0) (0, 0, 0, 0)

Requirements

  • Recommended: Configure at least one backend. See Backends below.
  • Git
  • CMake (version 3.7+ enables automatic discovery of an installed Vulkan SDK)
  • Ninja (or other build tool)
  • Python, for fetching dependencies and building Vulkan wrappers

Building

git clone https://github.com/google/amber.git
cd amber
./tools/git-sync-deps
mkdir -p out/Debug
cd out/Debug
cmake -GNinja ../..
ninja

Alternatives:

  • On Windows, Amber normally statically links against the C runtime library. To override this and link against a shared C runtime, CMake option -DAMBER_ENABLE_SHARED_CRT. This will cause Amber to be built with /MD for release builds or /MDd for debug builds.

Android

  • Android build needs Android SDK 28, Android NDK r25b, Java 8. If you prefer other versions of Android SDK, Android NDK, Java, then you can change ANDROID_PLATFORM and ANDROID_BUILD_TOOL_VERSION in tools/build-amber-sample.sh.
  • Set up Android SDK path by running export ANDROID_SDK_HOME=path/to/Android/SDK in your shell.
  • Set up Android NDK path by running export ANDROID_NDK_HOME=path/to/Android/NDK in your shell.
  • Generate a KeyStore using keytool command and set up KEY_STORE_PATH env variable for the KeyStore file path.
  • Run ./tools/build-amber-sample.sh [build output directory path].

Android plain executable

It is possible to obtain a plain executable for Android, as opposed to an APK, with the following:

git clone https://github.com/google/amber.git
cd amber
./tools/git-sync-deps

./tools/update_build_version.py . samples/ third_party/
./tools/update_vk_wrappers.py . .

mkdir build
cd build
mkdir app
mkdir libs
${ANDROID_NDK_HOME}/ndk-build -C ../samples NDK_PROJECT_PATH=. NDK_LIBS_OUT=`pwd`/libs NDK_APP_OUT=`pwd`/app

The list of target ABIs can be configured in samples/jni/Application.mk by editing the APP_ABI entry:

APP_ABI := arm64-v8a armeabi-v7a x86 x86_64

The resulting executable will be produced as build/app/local/<abi>/amber_ndk. This executable can be run via the adb shell on your device, e.g. under /data/local/tmp (/sdcard is generally not suitable because it is mounted with a non-executable flag). Also, vulkan layers may not be available to this executable as it is not an app, so make sure to use the -d flag to disable Vulkan layers:

adb push build/app/local/<abi>/amber_ndk /data/local/tmp
adb shell
# Now on device shell
cd /data/local/tmp
./amber_ndk -d <shader-test-files>

Optional Components

The components which build up Amber can be enabled or disabled as needed. Any option with _SKIP_ in the name is on by default, any with _USE_ is off by default.

The available flags which can be defined are:

  • AMBER_SKIP_TESTS -- Skip building Amber unit tests
  • AMBER_SKIP_SAMPLES -- Skip building the Amber sample applications
  • AMBER_SKIP_SPIRV_TOOLS -- Disable the SPIRV-Tools integration
  • AMBER_SKIP_SHADERC -- Disable the ShaderC integration
  • AMBER_SKIP_LODEPNG -- Disable the LodePNG integration
  • AMBER_USE_DXC -- Enables DXC as a shader compiler
  • AMBER_USE_LOCAL_VULKAN -- Does not try to find the Vulkan SDK, builds needed components locally
  • AMBER_USE_CLSPV -- Enables CLSPV as a shader compiler
  • AMBER_USE_SWIFTSHADER -- Builds Swiftshader so it can be used as a Vulkan ICD
cmake -DAMBER_SKIP_TESTS=True -DAMBER_SKIP_SPIRV_TOOLS=True -GNinja ../..

DXC

DXC can be enabled in Amber by adding the -DAMBER_USE_DXC=true flag when running cmake.

Build Bots

There are a number of build bots to verify Amber continues to compile and run on the various targets. Due to bot limitations, the integration tests are not being run on the bots, just the unit tests.

Backends

Amber is designed to run against different graphics APIs. Amber will build if no graphics API is found, but will only allow verifying the syntax of the amber script files.

Currently the Vulkan and Dawn graphics APIs are supported.

Using Vulkan as a backend

A Vulkan implementation is found by CMake in the following priority order:

  • If AMBER_USE_LOCAL_VULKAN is enable the headers, loader and layers will be built locally and not found on the system.

  • If an enclosing CMake project includes the Vulkan-Headers CMake project, then headers will be picked up from there.

    In this case the CMake variable Vulkan_LIBRARIES can name the Vulkan library, or a default of vulkan will be used.

  • If you have CMake 3.7 or later, then the Vulkan implementation will be found from a Vulkan SDK as published by LunarG.

    Environment variables:

    • VULKAN_SDK should point to the platform-specific SDK directory that contains the include and lib directories.
    • VK_ICD_FILENAMES should point to the ICD JSON file.
    • VK_LAYER_PATH should point to the explicit_layer.d folder.
    • LD_LIBRARY_PATH must contain the $VULKAN_SDK/lib/ folder for the validation libraries.
    export VULKAN_SDK=$HOME/vulkan-macos-1.1.85.0/macOS
    export VK_ICD_FILENAMES=$VULKAN_SDK/etc/vulkan/icd.d/MoltenVK_icd.json
    export VK_LAYER_PATH=$VULKAN_SDK/etc/vulkan/explicit_layer.d
    export LD_LIBRARY_PATH=$VULKAN_SDK/lib:$LD_LIBRARY_PATH
    

Using Dawn as a backend

We assume you have built Dawn from source, and have access to both the source and build trees. To build a Dawn backend for Amber, set the following CMake variables when configuring Amber:

  • Dawn_INCLUDE_DIR: The directory containing dawn/dawn_export.h (in the source tree).
  • Dawn_GEN_INCLUDE_DIR: The directory containing generated header dawn/dawncpp.h (in the build output tree).
  • Dawn_LIBRARY_DIR: The directory containing the dawn_native library (in the build output tree).

Using SwiftShader as a backend

SwiftShader, if available, can be used by by exporting the VK_ICD_FILENAMES environment variable and using it directly. If SwiftShader is not installed it can be built with Amber by setting AMBER_ENABLE_SWIFTSHADER during the configure step of CMake.

mkdir out/sw
cd out/sw
cmake -GNinja -DAMBER_ENABLE_SWIFTSHADER=TRUE ../..
ninja
export VK_ICD_FILENAMES=$PWD/Linux/vk_swiftshader_icd.json
./amber -d -V    # Should see SwiftShader listed as device
./amber -d ../../tests/cases/clear.amber

Amber Samples

The build will generate an out/Debug/amber executable which can be used to run amber scripts. The script can be used as out/Debug/amber <path to amber file>.

out/Debug/amber tests/cases/clear.amber

The sample app returns a value of 0 on success or non-zero on error. Any issues encountered should be displayed on the console.

Run out/Debug/amber -h to see a description of the program's command line options.

Example AmberScript files can be found in the tests/cases directory in this repository. Also the Vulkan Conformance Test Suite contains many real-world examples in its external/vulkancts/data/vulkan/amber subdirectory.

By default, out/Debug/amber supports saving the output image into '.png' file. You can disable this by passing -DAMBER_SKIP_LODEPNG=true to cmake.

The image_diff program will also be created. This allows comparing two images using the Amber buffer comparison methods.

Contributing

Please see the CONTRIBUTING and CODE_OF_CONDUCT files on how to contribute to Amber.

References

amber's People

Contributors

afd avatar alan-baker avatar antiagainst avatar asuonpaa avatar ben-clayton avatar cdotstout avatar dj2 avatar dneto0 avatar gfxstrand avatar gnl21 avatar hevrard avatar ilkkasaa avatar jaebaek avatar jarisiru avatar jeremyg-lunarg avatar jrprice avatar k3wlbuddy avatar karthikrajatp avatar kraj avatar mdagois avatar mmerecki avatar mvainola avatar paulthomson avatar pdaniell-nv avatar phoenixxxx avatar rg3igalia avatar s-perron avatar sarahm0 avatar vihanakangas avatar zoddicus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

amber's Issues

NDK Build

Add support to building Amber with the NDK and executing on Android.

AmberScript: Add support for DXC

We currently link in shaderc to compile GLSL shaders. We should add the option to build in DXC as well to compile HLSL shaders.

Clarify semantics of the Shutdown() commands

Does it mean:

  • I can never use this object again? If so then I'd call it Destroy()
  • Reset() . -- I can revive this object by re-initializing it?

It means never use again. For anything in Shutdown() that doesn't require error checking we can probably move it to the Destructor. Shutdown is there so we can error check the closing of our resources (similar to the Initialize() for startup).

ComputeCommand does not need PipelineData

As far as I understand, PipelineData will be used for only graphics pipeline because it only has setups for graphics pipeline e.g., topology, polygon mode, cull mode.

I think we can remove it from ComputeCommand.

Redefined buffer types

AmberScript's buffer type is confusing.

I feel that we should have the following buffer types:

  • color for color attachment of framebuffer
  • depth stencil for depth/stencil attachment of framebuffer
  • vertex for vertex buffer
  • index for index buffer
  • storage image
  • sampled image
  • combined image sampler
  • uniform texel buffer
  • storage texel buffer
  • storage buffer
  • uniform buffer
  • input attachment

Note that here we also uses the term 'buffer' for image because in the view of Amber script, the image is also a sequence of bytes.

Since storage image, sampled image, combined image sampler, uniform texel buffer, storage texel buffer, storage buffer, uniform buffer, and input attachment are Vulkan specific descriptor types and I am not sure WebGPU will define the same types or not, we would better consider how to support buffers of WebGPU efficiently.

Vulkan: Remove asserts from the codebase

The vulkan engine currently asserts in places where things should never happen. We should probably return in these cases so that amber can be used as a library if desired.

Windows build

Make sure everything builds and works on Windows. (This does not include the Dawn backend at this time).

VkScript: Data Ordering

According to the VkRunner README the draw commands are only executed when a probe command is encountered. So if you have something like:

ssbo 0 subdata vec3 0 1 2 3
draw rect
ssbo 0 subdata vec3 0 5 6 7
probe

The draw rect command will use the 5 6 7 values and not the 1 2 3 values.

We have to decide if we want to follow this behaviour or go with a more expected draw executes when it's seen so draw would use 1 2 3 not 5 6 7.

Dump image data

There is an un-implemented command line option to the amber script to dump image data, this should be implemented.

VkScript: Buffer Sizing

The buffer size for a given ssbo, uniform or uniform ubo is defined to be the largest size seen in the script. This isn't currently possible in Amber as you won't have the last SSBO command when you see the first one.

So, I think we need to add a pre-pass over the commands in Amber to sum the size of each type of buffer and send that data to the engine to declare the various buffers of the correct sizes.

Add shader oracle for Android CTS.

When working with the CTS the preference is to pre-compile all of the shaders, and send that blob over to the device. We need to support this in Amber.

There are two sides to this 1- generating the oracle 2- consuming the oracle.

For 1, we add a --oracle option to the amber application which will loop over all of the test cases, parse them, pull out the pipelines and compile the shaders as needed. It will generate a SHA of the shader source + spirv-opt options used during the compilation. This will then be written out to a file as SPIRV-HEX in a format like:

SHADER <sha>
 ... hex bytes ...
END

one block for each shader we compiled. (This also has the benefit of de-duplicating any shaders which use the same options to spirv-opt like the passthrough shader).

For 2, we update the amber application such that you can pass in the path to an Oracle, it will get read and when ShaderCompiler runs it will pull the shader from the Oracle instead of compiling.

Oracle class would be something like

Oracle:
  GenerateID(shader, options) ... outputs sha
  FindShader(shader, options) ... outputs hex bytes
  Load(string data)   ... loads oracle data from given string
  string Record()     ... writes data the string

We then put something into the API to allow you to Load and Record oracle data.

I actually think for the (shader, options) flags we want to pass in Pipeline and define an actual pipeline object like AmberScript will have. We force the creation for VkScript as needed.

Vulkan: Add memory barrier

Draw after sending vertex data to vertex buffer is not guaranteed currently, as well as probe after copying frame buffer to host accessible buffer.
We must add memory barrier for safety.

Dump buffer data

There is an un-implemented flag to the amber script to dump buffer data, this should be implemented.

Using system include directory causes build errors

If some libraries in third_party/ are already installed in the system, current Amber build adopts the system library instead of the one in third_party/ and it can lead to build errors.

The following log shows arguments for src/amberscript/parser_test.cc:

[6/7] /usr/bin/c++  -DAMBER_ENGINE_DAWN=0 -DAMBER_ENGINE_VULKAN=1 -I../../include
-I../../ -I../../third_party/spirv-tools/include -isystem /usr/local/include -isystem
../../third_party/googletest/googletest/include -I../../third_party/shaderc/libshaderc/include
-I../../third_party/glslang/SPIRV/.. -I../../third_party/glslang/SPIRV/../External
-I../../third_party/glslang/glslang/.. -I../../third_party/spirv-headers/include -isystem
../../third_party/googletest/googletest -std=c++11
-Wall -Werror -Wextra -Wno-unknown-pragmas -Wpedantic -pedantic-errors -g
-Wno-global-constructors -fno-exceptions -fno-rtti -pthread -MD -MT 
src/CMakeFiles/amber_unittests.dir/amberscript/parser_test.cc.o -MF 
src/CMakeFiles/amber_unittests.dir/amberscript/parser_test.cc.o.d -o 
src/CMakeFiles/amber_unittests.dir/amberscript/parser_test.cc.o -c ../../src/amberscript/parser_test.cc

-isystem /usr/local/include is shown before -isystem ../../third_party/googletest/googletest/include.

If gtest/gtest.h exists in /usr/local/include, it will be used instead of third_party/googletest/googletest/include.

However, the build will try to link it with the one in third_party/ and it can lead to unreferenced symbols.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.