Giter VIP home page Giter VIP logo

indirectbuffer's Introduction

IndirectBuffer

This library provides a way for emscripten-compiled C++ code to store large amounts of data outside of the main heap. Moving large allocations out of the main heap reduces memory fragmentation issues for long-running sessions, allows your code to use more of the limited address space in 32-bit browsers, and allows your code to break past the 31-bit typed array size limitation in 64-bit browsers.

This library isn't exactly what we use internally at Figma but it's the same general idea. We're providing this code because we think indirect memory storage is a big win and it's not described in the emscripten documentation. The closest analogy is the emscripten file system API but this IndirectBuffer API is simpler and easier to work with both from C++ and from JavaScript.

To use it, just copy IndirectBuffer.h and IndirectBuffer.cpp into your project. The main.cpp and catch.hpp files included here are for unit testing (run "make test" assuming you have emscripten installed and activated).

Example usage (out-of-heap image decode):

// Compile using: emcc example.cpp IndirectBuffer.cpp -std=c++11 -O3 -s NO_EXIT_RUNTIME=1 -s EXPORTED_FUNCTIONS="['_main','_DecodeImage_resize','_DecodeImage_finish']"

#include <emscripten.h>
#include <functional>
#include <stdio.h>
#include <string>
#include "IndirectBuffer.h"

struct DecodeImage {
  using Callback = std::function<void (int, int, IndirectBuffer)>;

  DecodeImage(const std::string &url, const Callback &callback) : _callback(callback) {
    EM_ASM_ARGS({
      var image = new Image;
      image.onload = function() {
        var width = image.width;
        var height = image.height;
        var canvas = document.createElement('canvas');
        canvas.width = width;
        canvas.height = height;
        var context = canvas.getContext('2d');
        context.drawImage(image, 0, 0);
        var pixels = context.getImageData(0, 0, width, height);
        var handle = Module._DecodeImage_resize($0, pixels.data.length);
        Module._IB_[handle].set(new Uint8Array(pixels.data.buffer));
        Module._DecodeImage_finish($0, width, height);
      };
      image.src = Module.Pointer_stringify($1);
    }, this, url.c_str());
  }

  int resize(size_t size) {
    _buffer.resize(size);
    return _buffer.handleForEmscripten();
  }

  void finish(int width, int height) {
    _callback(width, height, std::move(_buffer));
  }

private:
  Callback _callback;
  IndirectBuffer _buffer;
};

extern "C" int DecodeImage_resize(DecodeImage *self, size_t size) {
  return self->resize(size);
}

extern "C" void DecodeImage_finish(DecodeImage *self, int width, int height) {
  self->finish(width, height);
}

static bool isImageOpaque(const IndirectBuffer &buffer) {
  return EM_ASM_INT({
    var array = Module._IB_[$0];
    for (var i = 3, n = array.length; i < n; i += 4) {
      if (array[i] < 255) {
        return false;
      }
    }
    return true;
  }, buffer.handleForEmscripten());
}

int main() {
  static DecodeImage async("image.png", [](int width, int height, IndirectBuffer buffer) {
    printf("loaded %dx%d image outside main heap\n", width, height);
    printf("image is opaque: %d\n", isImageOpaque(buffer));
  });
  return 0;
}

indirectbuffer's People

Contributors

evanw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

indirectbuffer's Issues

Can you provide a full working example ?

Hi, thanks for this promising library.
I'm new to WASM so my request may look naive.
Can you please provide a full example of how to use this library and call it from JS.
I understand from the example you provided in the readme that I need to deal with everything from C++ (because you added the main function) but what if I was building a library that'll be called from JS directly?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.