• Website: https://www.greenfield.app

Greenfield is a Wayland compositor written entirely in TypeScript while utilizing WebAssembly and WebGL for the performance critical parts. It can run native Wayland applications remotely and display them directly in your browser.

Greenfield running DOOM3 remotely 1920x1080@60FPS (16 Mar 2023):

For more information, visit the website, or have a look on how it all began

Greenfield consists of 3 separate parts.

• Westfield A Wayland protocols implementation in TypeScript.
• Greenfield Compositor Proxy A Wayland compositor proxy. Forwards the local Wayland applications to the browser.
• Greenfield Compositor Module A bare-bones Wayland compositor library for the browser. Receives the forwarded signals of the compositor-proxy.

To run, you will need 2 parts to work together.

• The Greenfield Compositor Proxy that runs remotely and forwards the native Wayland applications to the browser. You need at least one, but can run as many as you require e.g. to access different machines.
• An implementation of the Greenfield Compositor Module that runs in your browser and receives the forwarded signals of potential multiple Greenfield Compositor Proxies.

The Greenfield Compositor Module comes with a simple demo implementation.

In the compositor-module directory run

• yarn install
• yarn generate
• yarn start

to start the compositor demo module.

Note that the compositor-module built uses Open-API to generate client code to talk to the compositor-proxy and requires java to be present on your PATH during build.

Go to http://localhost:8080 and be greeted with a nice white compositor. It has 2 URL input fields that can be used. The first input field connects to the provided remote compositor and accepts input in the form of host:port. The second input field launches and runs Wayland applications in an iframe (experimental) and accepts input in the form of http(s)://host:port/path.

To build, you need a set of native dependencies. You can look at the Docker image to see which ones you need to build and run respectively, or if you're running a Debian based distro you can run:

sudo apt install cmake build-essential ninja-build pkg-config libffi-dev libudev-dev libgbm-dev libdrm-dev libegl-dev \ 
 libwayland-dev libglib2.0-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgraphene-1.0-dev gstreamer1.0-plugins-base \ 
 gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-gl xwayland

More dependencies may be required depending on your GPU eg. for nvidia based cards.

Next, inside compositor-proxy, run:

• yarn install
• yarn generate
• yarn build:native
• cp dist/encoding/proxy-encoding-addon.node src/encoding/proxy-encoding-addon.node
• cp dist/proxy-poll-addon.node src/proxy-poll-addon.node

For XWayland support a few extra steps may be needed, this is optional and only required if you don't already hava an X server running eg. when running on a server:

• export XAUTHORITY=.Xauthority
• touch "$HOME/$XAUTHORITY"
• xauth add "${HOST}":1 . "$(xxd -l 16 -p /dev/urandom)"

and finally

• yarn start

NOTE: Firefox needs to be at least at version 113 and dom.workers.modules.enabled preference needs to be set to true. To change preferences in Firefox, visit about:config.

This will start a development build+run. You should now see something that says Compositor proxy started. Listening on port 8081. You can also adjust some things in src/config.yaml.

In our demo compositor we can now input the url localhost:8081 to make a connection to the Greenfield Compositor Proxy. You should see a message appear in the log output of the compositor-proxy that we started earlier: New websocket connected..

You should now have a Wayland compositor running on your system, so let's start some applications. Most recent GTK3/4 applications (like gnome-terminal) should auto-detect the compositor-proxy and simply connect without issues or extra setup. QT applications often require an extra -platform wayland parameter. If your application can't connect, try setting the WAYLAND_DISPLAY environment variable to the value that was printed by compositor-proxy. ie if you see Listening on: WAYLAND_DISPLAY=\"wayland-0\". then set the environment variable export WAYLAND_DISPLAY=wayland-0.

It's also possible to build a distributable release.

• yarn install
• yarn generate
• yarn build
• yarn package

This creates a set of files in the package directory. The run.sh script accepts several options:

        Usage
          $ compositor-proxy <options>

        Options
          --help, Print this help text.
          --static-session-id=...,  Mandatory. Only use and accept this session id when communicating.
          --config-path=...,  Use a custom configuration file located at this path.

        Examples
          $ compositor-proxy --static-session-id=test123 --config-path=./config.yaml

Below is an example config file (the default config). It can be copy-pasted and used with the --config-path=... option. You will at least need to set the public.baseURL when not running locally.

server:
  http:
    # Hostname argument.
    bindIP: 0.0.0.0
    # Port argument.
    bindPort: 8081
    # CORS allowed origins, used when doing cross-origin requests. Value can be * or comma seperated domains.
    allowOrigin: '*'
public:
  # The base url to use when connecting to this endpoint. This is the publicly reachable address of the compositor proxy.
  baseURL: http://localhost:8081
encoder:
  # Path of the render device that should be used for hardware acceleration. e.g. /dev/dri/renderD128
  renderDevice: /dev/dri/renderD128
  # The gstreamer h264 encoder to use. 'x264' and 'nvh264' are supported ('vaapih264' is currently broken). 'x264'
  # is a pure software encoder. While 'nvh264' is a hw accelerated encoder for Nvidia based GPUs.
  # see https://gstreamer.freedesktop.org/documentation/x264/index.html
  # see https://gstreamer.freedesktop.org/documentation/nvenc/nvh264enc.html
  h264Encoder: x264
logging:
  # "fatal" | "error" | "warn" | "info" | "debug" | "trace"
  level: info

The packaged binary expects the following set of dependencies to be available for mesa & nvidia support, if you're running a Debian based distro you can run:

apt-get install \
    libffi8 \
    libudev1 \
    libgbm1 \
    libgraphene-1.0-0 \
    gstreamer1.0-plugins-base \
    gstreamer1.0-plugins-good \
    gstreamer1.0-plugins-bad \
    gstreamer1.0-plugins-ugly \
    gstreamer1.0-gl \
    libosmesa6 \
    libdrm2 \
    libdrm-intel1 \
    libopengl0 \
    libglvnd0 \
    libglx0 \
    libglapi-mesa \
    libegl1-mesa \
    libglx-mesa0 \
    libnvidia-egl-wayland1 \
    libnvidia-egl-gbm1 \
    xwayland \
    xauth \
    xxd \
    inotify-tools \
    libnode108

Running the Greenfield Compositor Proxy can also be done using docker-compose (see docker-compose.yml in the compositor-proxy directory), but you will be limited to the applications specified in the docker-compose file. Beware that this docker compose file only provides the Greenfield Compositor Proxy, so you will still need to run a Greenfield Compositor Module implementation yourself.

There is also a short screen-cast, if you're unsure on how to get started:

The compositor-proxy is also available as a public docker image docker.io/udevbe/compositor-proxy but does not include any config.yaml. This means you'll have to include it yourself using a mount. Have a look at the docker-compose file for inspiration.

A Greenfield browser compositor uses a native compositor-proxy to talk to native applications. This proxy compositor accepts native Wayland client connections and assigns them to a WebSocket connection as soon as one becomes available. A native client and it's WebSocket connection are bound to each other until either one is closed.

A compositor-proxy proxy can request additional WebSocket connections from an already connected Greenfield browser compositor. This is needed in case a Wayland client spawns a new Wayland client process. If no WebSocket connections already exists, the compositor-proxy will wait until a new WebSocket connection is available. In other words, the first WebSocket connection is always initiated from the browser.

Each application's content is encoded to video frames using GStreamer and send to the browser for decoding. In the browser the application is realised by a WebGL texture inside a HTML5 canvas. This canvas is basically what you would call an 'output' in Wayland terminology. The browser compositor is asynchronous, meaning a slow client will not block the processing of another client.

This setup has some performance constraints as a lot of parts still use the CPU to do image encoding and decoding. The proxy compositor can't deal with OpenGL texture coming from the client either, meaning that a client (ie a game) needs to download its entire rendering to RAM before passing it to the proxy-compositor. Another drawback is that we have to work with a dual encoding/decoding solution as the H.264 codec does not support transparency (alpha).

To make this performant, all image processing should be done on the GPU and an image codec that supports alpha (transparency) should be used.

Such a hypothetical perfect pipeline would look something like this:

Here, all heavy operations are done by the GPU. H.264 has been replaced with H.265 which supports transparency, so we can use a single encoding/decoding step. The compositor-proxy supports the Wayland DRM protocol, so it can pass OpenGL applications directly to the encoder without making any copies.

There is however one major problem to this solution: the combination of WebCodecs API and the H.265 codec simply does not exist in any browser.

Here we've extended the old solution with support for the Wayland DRM protocol (+ DMA_BUF protocol). This allows for a zero-copy transfer of the application pixels to the encoding pipeline. We've also added support for the H.264 WebCodecs API, which allows us to do decoding on the GPU of the receiving browser client if supported. If no usable GPU is available in the Compositor-Proxy, the pipeline falls back to slower software rendering.

The end result is a near ideal solution that is fast enough to support gaming.

There is one drawback that currently still remains, and that's the use of WebSockets to deliver data to the browser. WebSockets operate over TCP which is ill-suited for real-time applications like Greenfield. Instead, a UDP based protocol is needed. Browsers today unfortunately have no support for UDP based protocols aside from WebRTC DataChannels. However, we can not use WebRTC DataChannels as the build-in SCTP congestion algorithm is unacceptably slow. A more low level UDP protocol is required and is currently in the works in the form of the WebTransport protocol. Once WebTransport becomes more widely available, we can operate in UDP mode and assure fast end reliable transfers using KCP in combination with forward-error-correction.

If both clients are connected to separate compositor-proxy, copy-paste will use a direct peer to peer transfer between compositor-proxies. This avoids the round trip and massive overhead of transferring all content to the browser and back. How this works is illustrated in the image below:

Very much beta. Most things are implemented except for fullscreen applications. Please report any bugs or annoyances you find.

Fosdem presentation + demo (2 Feb 2019):

Early tech preview demo (23 Nov 2017):