Giter VIP home page Giter VIP logo

blinkjs's Introduction

blink.js

Easy GPGPU on the web, powered by WebGL 2.0.

Latest NPM release License Dependencies Dev Dependencies

blink.js (not to be confused with Blink, the Chromium render engine) is a small, easy to use GPGPU library for the web, exploiting the power of WebGL 2.0.

Please note: blink.js uses its own WebGL 2.0 context. Which means it's not pluggable with other WebGL frameworks. Though, theoretically, you could use blink.js' context as your main WebGL context.

Table of contents

Installation

Download the blink.min.js file from the dist folder. Then reference it in the HTML using the <script> tag.

<script src="blink.min.js"></script>

Or use the unpkg CDN:

<script src="https://unpkg.com/blink.js/dist/blink.min.js"></script>

Quickstart

blink.js works with two types of objects: Buffers and Kernels. A Buffer is an (large) array of values that can be read from and/or written to. A Kernel contains the shader code that will be executed on the device.

Counting up

In the following example we will initialize a Buffer, allocating space for 1,048,576 integers (4 MB). The Kernel will set all values to their corresponding location in the buffer.

let buffer = new blink.Buffer({
    alloc: 1024 ** 2,
    type: blink.UINT32,
    vector: 1
}) // 4 MB

let kernel = new blink.Kernel({
    output: { buffer }
}, `void main() {
        buffer = bl_Id();
    }`)

kernel.exec()

for (let a = 0; a < 10; a++) {
    console.log(`Value at ${a} is ${buffer.data[a]}.`)
}

Greyscale image

In this example we will use blink.js to convert the image data of a canvas context to black and white.

const ctx = canvas.getContext('2d')
/* Perform any drawing in the context here. */

let imageData = ctx.getImageData(0, 0, ctx.canvas.width, ctx.canvas.height)
let buffer = new blink.Buffer({
    data: imageData.data,
    vector: 4
})

let kernel = new blink.Kernel({
    input: { in_rgba: buffer },
    output: { out_rgba: buffer }
}, `void main() {
        lowp uvec4 color = texture(in_rgba, bl_UV);
        mediump uint grey = (color.r + color.g + color.b) / 3u;
        out_rgba = uvec4(uvec3(grey), color.a);
    }`)

kernel.exec()

ctx.putImageData(imageData, 0, 0)

Usage

Classes

Buffer

let buffer = new blink.Buffer({
    alloc: 1024 ** 2,
    type: blink.UINT8,
    vector: 1,
    wrap: blink.CLAMP
}) 
buffer.data[0] = 1

The Buffer class represents an array of values that can be read from and written to on the device. A Buffer's data is copied to the device the moment it's required. After a Kernel is done executing all its steps, the data on the device is copied back to the host, the data on the device is destroyed immediately.

  • new Buffer({ alloc|data, type, vector, wrap })

    Initialize a new buffer using the given Object containing the following parameters:

    • alloc: Initialize an (0 filled) ArrayBuffer with this size. Note: The given number represents the number of elements of type. Not the size in bytes.

    • data: Opposed to having blink.js initialize the data, you can parse a TypedArray. The Buffer will hold a reference to this TypedArray. Note: If both alloc and data are present in the Object, alloc is chosen.

    • type: The type of primitives of the Buffer. See Types. Default is FLOAT.

    • vector: Number of elements in the vector. Can be 1, 2 or 4. Default is 1.

    • wrap: Texture wrap mode. Can either be an array for S and T or a single constant. See Wrap modes. Default is CLAMP.

  • Buffer.prototype.data

    Reference to the TypedArray.

  • Buffer.prototype.copy()

    Returns a copy of the Buffer. The new instance will also hold a copy of the data allocated on the host.

DeviceBuffer

const size = 512 ** 2 * 4
let d_buffer = new blink.DeviceBuffer({
    alloc: size * 4,
    type: blink.UINT32,
})

d_buffer.toDevice(new Uint32Array(size).fill(1))
const array = d_buffer.toHost()

d_buffer.delete()

Unlike a Buffer, a DeviceBuffer keeps its memory allocated only on the device. This greatly increases performance when memory is not required to be copied back to the host after a Kernel is done executing.

Memory is allocated (or copied) the moment the DeviceBuffer is initialized. Memory is retained on the device until the DeviceBuffer's delete method is called. (Or until the browser garbage collects the DeviceBuffer. But it is strongly advised to manually maintain the memory on the device.)

Data can be downloaded to the host and uploaded to the device using the toHost and toDevice methods respectively.

  • new DeviceBuffer({ alloc|data, type, vector })

    See new Buffer(). Only major difference is that no data is allocated nor referenced on the host.

  • DeviceBuffer.prototype.copy()

    Returns a copy of the DeviceBuffer. The data on the device is also copied.

  • DeviceBuffer.prototype.delete()

    Delete the data on the device, and, essentially, turn the DeviceBuffer's instance unusable.

  • DeviceBuffer.prototype.toDevice(data)

    • data: A TypedArray (of the same type and size the DeviceBuffer was initialized with) whose data will be uploaded to the device.
  • DeviceBuffer.prototype.toHost([data])

    Download the data on the device back to the host.

    • data: (Optional) If given, it should be of the same type and size the DeviceBuffer was initialized with. If not given, blink.js will initialize and return the correct TypedArray.
  • DeviceBuffer.prototype.toHostAsync([data])

    Download the data on the device back to the host, asynchronously. Unlike toHost(), this method returns a Promise.

    • data: (Optional) See DeviceBuffer.prototype.toHost. If not given, blink.js will initialize the correct TypedArray, and pass it through the returned Promise's thenable.

NOTE: Only available if the WEBGL_get_buffer_sub_data_async extension is supported.

Kernel

let buffer = new blink.Buffer(/* ... */)

let input = { in_buffer: buffer }
let output = { out_buffer: buffer }
let kernel = new blink.Kernel({ input, output }, `
    uniform float multiplier;

    void main() {
        float val = texture(in_buffer, bl_UV).r;
        out_buffer = val * multiplier;
    }`)

kernel.exec({ multiplier: 2 })
kernel.delete()
  • new Kernel({ input, output }, shaderSource)

    Initialize a new Kernel with the given inputs, outputs and (fragment) shader source.

    • input: (Optional) A key-value Object, where keys are the names of the inputs and the values a reference to either a Buffer or DeviceBuffer. The input names become available in the shader to read from their respective buffer.

    • output: Same as input, except for writing to buffers.

    • shaderSource: Source of the shader as a String.

  • Kernel.prototype.exec([uniforms])

    Execute the Kernel.

    • uniforms: (Optional) If given, it should be a key-value Object with the keys being the uniforms' names and values their value.
  • Kernel.prototype.delete()

    Delete all associated shaders and programs on the device. Essentially rendering the Kernel's instance as unusable.

Types

  • blink.FLOAT (Float32Array)
  • blink.UINT8 (Uint8Array) (Uint8ClampedArray will be casted to Uint8Array)
  • blink.UINT16 (Uint16Array)
  • blink.UINT32 (Uint32Array)
  • blink.INT8 (Int8Array)
  • blink.INT16 (Int16Array)
  • blink.INT32 (Int32Array)

Wrap modes

  • blink.CLAMP
  • blink.REPEAT
  • blink.MIRROR

Device

blink.device contains information gathered from the WebGL context.

  • glslVersion: version of GLSL.
  • maxColorAttachments: Maximum number of outputs a Kernel can render to in a single step.
  • maxTextureSize: Maximum dimension of an input/output buffer.
  • maxTextureUnits: Maximum number of input buffers.
  • renderer: Renderer name.
  • vendor: Vendor name.
  • unmaskedRenderer: Unmasked renderer name.[1]
  • unmaskedVendor: Unmasked vendor name.[1]

[1] Only available if the WEBGL_debug_renderer_info extension is supported.

GLSL

WebGL 2.0 supports GLSL ES 3.00, which includes (but not limited to) the following significant new features compared to GLSL ES 1.30 in WebGL 1.0:

  • Unsigned integer types.
  • Bitwise operators.
  • for and while loops with variable lengths.
  • Non-square matrix types.
  • A butt-load of matrix functions.

Built-in variables

highp vec2 bl_UV;    // Normalized coordinate of the current fragment.
highp ivec2 bl_Size; // Dimensions of the output buffer(s).
highp uint bl_Id();  // Returns the id of the current fragment.

Type compatibility

Different combinations of operating systems, browsers and hardware may not support the entire list of type and vector size combinations. See COMPATIBILITY.md for a personally tested list of compatibility. Or open test/report.html in your browser to test yourself.

Built for today

Both Firefox 59 and Chrome 65 (on desktop) support WebGL 2.0 now. As of writing this readme, the status of WebGL 2.0 for Safari is declared as Supported in preview (Safari Technology Preview). However, shaders are unable to successfully compile, thus rendering WebGL 2.0 support as pretty much unusable. There is no concrete answer whether Safari will fully support WebGL 2.0 in the future.

blink.js uses ES6 syntax/features that are not transpiled or polyfilled for ES5. Babel is only used to minify the final code, using the babili preset.

See also

headless-gl - create WebGL contexts in Node.js, powered by ANGLE.

turbo.js - a similar GPGPU library for the browser, using WebGL 1.0.

WebGP.js - another similar GPGPU library for the browser.

gpu.js - transpile and run Javascript code on the GPU.

NOOOCL - OpenCL bindings for Node.js.

The Book of Shaders - Great introduction to GLSL by Patricio Gonzalez Vivo.

WebGL 2.0 Reference card - PDF containing all GLSL ES 3.00 functions and variables.

blinkjs's People

Contributors

dependabot[bot] avatar frzi avatar minujeong avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

blinkjs's Issues

can't use vector: 1 or vector: 2 on core i7 laptop

I am running the very first example of blink.js (the one using vector: 1 and assigning to each output fragment its id). In console, I get a list of zeroes. On anoter machine, or using vector: 4 and manually setting each field of the vector, I get the expected result. In the javascript console, I get a warning:

Error: WebGL warning: readPixels: Incompatible format or type.

This is at line 840 in file dist/blink.js:

gl.readPixels(0, 0, size[0], size[1], gl[format], gl[type], buffer.data, 0);

I then added a line before that one:

console.log(gl[format], gl[type]);

and here is the result:

33319 5126

That's all I know. I have a fairly standard 2013 laptop, intel core i7 4th generation with an intel integrated gpu. I am using chrome 62.0.3202.94 on ubuntu 17.10.

gl.FLOAT_VEC_2 should be gl.FLOAT_VEC2

const uniformsFnTable = {
[gl.FLOAT]: 'uniform1f',
[gl.FLOAT_VEC_2]: 'uniform2f',
[gl.FLOAT_VEC_3]: 'uniform3f',
[gl.FLOAT_VEC_4]: 'uniform4f',

gl.FLOAT_VEC_2 should be gl.FLOAT_VEC2
gl.FLOAT_VEC_3 should be gl.FLOAT_VEC3
gl.FLOAT_VEC_4 should be gl.FLOAT_VEC4

Errors Running in a React environment installed using npm.

When including blink.js as -

import * as blink from 'blink.js';

I get the Error -

Error: Unable to compile vertex shader. Info log: ERROR: 0:3: '/' : syntax error

▼ 2 stack frames were expanded.
compileShader
     node_modules/blink.js/src/WebGL/Program.js:86

Module../node_modules/blink.js/src/WebGL/Program.js
     node_modules/blink.js/src/WebGL/Program.js:5

After further looking into i found this error is because of these lines -

import vertexSource from './../shaders/quad.vert'

import fragTemplate from './shaders/template.frag'

The imported shader is passed into -

function compileShader(type, source) {
// Check if the shader defines glsl version.
if (!(/^#version 300 es/g).test(source)) {
source = '#version 300 es\n\r' + source
}
let shader = gl.createShader(type)
gl.shaderSource(shader, source)
gl.compileShader(shader)
if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
// TODO: Better error logging.
const typeName = type == gl.VERTEX_SHADER ? 'vertex' : 'fragment'
const infoLog = gl.getShaderInfoLog(shader)
throw new Error(`Unable to compile ${typeName} shader. Info log:
${infoLog}`)
}
return shader
}

it seems like importing the shaders in this way results in the shader becoming -

#version 300 es

/static/media/quad.75d70ea6.vert

This is fixed by exporting the shader as a string like -

export default`#version 300 es

precision highp float;

out vec2 bl_UV;

void main() {
	float x = -1.0 + float((gl_VertexID & 1) << 2);
    float y = -1.0 + float((gl_VertexID & 2) << 1);
    gl_Position = vec4(x, y, 0, 1);
	bl_UV = gl_Position.xy * 0.5 + 0.5;
}`

this results in the correct behavior of the initial shaders being compiled as -

#version 300 es

precision highp float;

out vec2 bl_UV;

void main() {
	float x = -1.0 + float((gl_VertexID & 1) << 2);
    float y = -1.0 + float((gl_VertexID & 2) << 1);
    gl_Position = vec4(x, y, 0, 1);
	bl_UV = gl_Position.xy * 0.5 + 0.5;
}

I will submit a PR once i perform some more tests.

2d_coordinates

Hello, thank you for your work. When running a shader on the imagedata object from a 2d canvas context, I cannot find a reference to the x and y coordinate of the current fragment/pixel on the canvas. gl_FragCoord.y seems to be working but gl_FragCoord.x seems not correct. bl_UV behaves similar but in the range 0-1. Is there a way to get the x and y coordinate for example within the grayscale example? I might be missing something. Thanks!

Different sized buffer than the output size causing problem

Hi,

Thanks for releasing such a great library. It's the most flexible GPGPU library I have found. But I am getting an error. When I am doing as following.

let a = new blink.Buffer({ data: new Int32Array([1, 2, 3, 4, 5, 6]), type: blink.INT32, vector: 1 }); let b = new blink.Buffer({ alloc: 5, type: blink.INT32, vector: 1 }); let kernel = new blink.Kernel({ input: { 'a': a }, output: { 'b': b } }, 'void main() { b = texture(a, bl_UV).r; }'); kernel.exec(); for (let i = 0; i < 5; i++) { console.log('Value at ${i} is ${b.data[i]}.') }

Means, I am sending output buffer size as 5 but the size my input buffer 6 or might be some larger value, it results very weird. Sometime we need to pass larger input buffer. Also, I am not sure, if I can pass a buffer through uniforms using "exec()" function key-value pair. I need to pass a few very large buffers as input and I need to get comparatively smaller output buffer. Is it possible?

Can you please help me to sort out this issue? Thanks in advance.

Outputs without consistent sizes

Is there a particular reason for requiring consistent output buffer sizes? Handling the case for inconsistent sizes could be supported with an impact to the API of only one additional option.

On that note, what do you think of the allowing the kernel to be "sized by", not the consistent size of all output buffers, but by any of the input / output buffers or by an integer value? The generated vert shader, that ultimately sets up the iterable execution of the kernel, appears to be agnostic to the particulars of the bl_size.

I can see this being useful for a variety of usecases, mine in particular is based on a derivative of this GPU-based Sparse Voxel Octree, where an input voxel_fragment_list is iterated over and outputs an updated node_pool where nodes that need subdividing are flagged. In this case, the output buffer size is unknown before exec() because multiple entries in the voxel_fragment_list will flag the same nodes and thus the iteration is based on the size of the input buffer... I could preallocate a large node_pool, but given the potential sparseness of voxelized data this would waste a lot of cycles.

I may have time to proto an implementation, hopefully before you get to responding to this...

DeviceBuffer._finish()

_finish() {
		// Swap.
		this._getReadable().delete()
		readablesMap.set(this, this._getWritable())
		writablesMap.delete(this)
	}

The following is my modification:

_finish() {
        // Swap.
        const wr_copy = this._getWritable();
        if (wr_copy) {
            const rd_copy = this._getReadable();
            if (rd_copy) {
                rd_copy.delete();
            }
            readablesMap.set(this, wr_copy);
            writablesMap.delete(this);
        }
    }

Copy data to host takes time proportional to previous iterations of the same kernel

I have an odd behaviour that I can't explain. I observed this with other frameworks (e.g. turbo.js, or gpu.js). I run a kernel which takes very few time (0.1 milliseconds on my laptop) using a DeviceBuffer (no intermediate data copy). The kernel uses the same buffer for input and output. I iterate it k times. After that, I copy the data to host for rendering. Data copy to host (just that call) takes time proportional to k whereas it should take constant time if I understand things correctly.

Here is a .zip file with a quasi-minimal working example. There is a constant named iters in line 4 of main.js. Increasing the value of iters will make the call to dbuffer.toHost in line 60 take more and more. Open console first, to be able to see the logging of execution times, then click on the canvas to see timing measurements, then try to increase k (by very small amounts!), and click again on the canvas to see the bug.

I'm running chrome on linux / intel card. In Firefox I observe also odd behaviour and huge cpu/gpu load, but I am unsure if it's only the call to toHost which is affected.

blinkjs-bug.zip

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.