Giter VIP home page Giter VIP logo

modsurfer's Introduction

Modsurfer

Modsurfer

Modsurfer provides ops & dev teams with a system of record + diagnostics application to search, browse, and investigate modules.

For developers, SRE, DevOps, and engineering leaders: understand what the WebAssembly in your system is all about. Modsurfer provides critical information about WebAssembly code through a handy GUI, or directly at your fingertips via our CLI.

Use Modsurfer for:

  • at-a-glance insights into WebAssembly module data (code size & complexity, imports/exports & more)
  • search for details about modules (hash, ID, function names, strings, namespaces, errors & more)
  • off-the-shelf System of Record: easily audit and track all the WebAssembly code in your stack
  • debug & triage issues otherwise difficult to pinpoint in opaque WebAssembly format

Modsurfer Desktop Application

The desktop application is free and available to download from Dylibso, and is a useful debugging, visibility, and diagnostics tool for anyone working with WebAssembly.

Modsurfer

Enterprise

If you're running WebAssmebly code in production, you may be interested in using the Enterprise version of Modsurfer. Please reach out to us at [email protected] and we'll be happy to get you more information about licensing and self-managed deployment requirements.


In this repository

This is a collection of Rust crates and other code that is used for a variety of Modsurfer-related purposes. You can find a list and description of these below.

cli

The Modsurfer CLI provides two primary functions:

  1. HTTP Client to interace with Modsurfer (either the desktop app, or your Enterprise deployment)
  2. Validation to ensure that WebAssembly binaries:
  • are compatible with various host ABIs
  • have no known security issues / comply with your policy
  • meet the runtime requirements in terms of size and code complexity (RAM/CPU limits, etc)

Download the CLI

Modsurfer CLI can be downloaded via:

  • the latest GitHub Release
  • the GitHub container registry (for Docker environments)
    • docker pull ghcr.io/dylibso/modsurfer:latest

Validate WebAssembly Modules

Modsurfer CLI provides a novel feature to process and validate a WebAssembly module based on policy/properties of the module that you define. In a "checkfile" (see the mod.yaml below), declare certain properties of a WebAssembly module and Modsurfer will verify that they are true, or report which properties of your binary are invalid.

validate:
  # simply require that a module can have WASI functionality or not
  allow_wasi: false
  
  # ensure that various imports and exports are included/exlcuded such that a module
  # will run properly in any host environment
  imports:
    include:
      # ensure these named functions are in the imports of this module  
      - log_message
      - proc_exit
      
      # further specify the function beyond its name 
      - namespace: env
        name: http_get
        params: [I32, I32]
        results: [I32]
    exclude: 
      - fd_write
    namespace:
      include:
        - env
      exclude:
        # phasing out old APIs? exclude these from acceptable namespaces/module names
        - some_future_deprecated_module_name
        - wasi_snapshot_preview1

  exports: 
    # only want exactly 2 functions exported: `_start` and `bar` for the host to call:
    max: 2
    # secure your modules by ensuring that there is no superfluous functionality hidden inside a binary
    include:
      - _start
      - name: bar
        params: []
        results: [I32, I32, I32, I32]
    # and/or ensuring no unwanted functions to be exported.
    exclude:
      - name: init
        results: []
      - foo

  # use a human-readable module size to prevent overly large binaries from running in your environment
  size:
    max: 4MB

  # our Cyclomatic Complexity analysis can help prevent risk of CPU exhaustion from deteriorating 
  # your user experience and slowing down your system
  # (override these low, medium, high optional values with environment variables $MODSURFER_RISK_{LOW,MEDIUM,HIGH})
  complexity:
    max_risk: low

You can also point to a remote check file to track up-to-date requirements:

validate:
  url: https://raw.githubusercontent.com/fermyon/spin/main/tools/modsurfer/http/mod.yaml

Usage

Modsurfer runs validation tests on compiled .wasm binaries. It uses a "checkfile" to compare with the contents of a .wasm binary. The modsurfer CLI provides a number of commands to create and use a checkfile, as well as to interact as a client to a remote service that stores and organizes your WebAssembly modules available over HTTP (via this protobuf API).

To create a "checkfile" (passed to -c), run the generate command:
modsurfer generate -p path/to/my.wasm -o mod.yaml

NOTE: this checkfile will be very restrictive, and you likely want to edit it to fit less (or more) restricted environments.

To run validation, you can use our GitHub Action, or call the validate command directly:
modsurfer validate -p path/to/my.wasm -c path/to/mod.yaml

If any of the restrictions or expectations declared in your checkfile are not satisfied, Modsurfer will report them:

┌────────┬──────────────────────────────────────────────────┬──────────┬──────────┬───────────────────┬────────────┐
│ Status │ Property                                         │ Expected │ Actual   │ Classification    │ Severity   │
╞════════╪══════════════════════════════════════════════════╪══════════╪══════════╪═══════════════════╪════════════╡
│ FAIL   │ allow_wasi                                       │ false    │ true     │ ABI Compatibility │ |||||||||| │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ complexity.max_risk                              │ <= low   │ medium   │ Resource Limit    │ |          │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ exports.exclude.main                             │ excluded │ included │ Security          │ |||||      │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ exports.include.bar                              │ included │ excluded │ ABI Compatibility │ |||||||||| │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ exports.max                                      │ <= 100   │ 151      │ Security          │ ||||||     │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ imports.include.http_get                         │ included │ excluded │ ABI Compatibility │ ||||||||   │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ imports.include.log_message                      │ included │ excluded │ ABI Compatibility │ ||||||||   │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ imports.namespace.exclude.wasi_snapshot_preview1 │ excluded │ included │ ABI Compatibility │ |||||||||| │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ imports.namespace.include.env                    │ included │ excluded │ ABI Compatibility │ ||||||||   │
├────────┼──────────────────────────────────────────────────┼──────────┼──────────┼───────────────────┼────────────┤
│ FAIL   │ size.max                                         │ <= 4MB   │ 4.4 MiB  │ Resource Limit    │ |          │
└────────┴──────────────────────────────────────────────────┴──────────┴──────────┴───────────────────┴────────────┘

NOTE: convert this table into JSON with the --output-format json option, supported by the validate command and many others.

Find more information about the CLI in its dedicated README, or download the tool and run modsurfer -h.

Validating platform-specific compatibility

Before running or integrating a WebAssembly module on your platform (Emscripten, Extism, Fastly, Shopify, Spin, Suborbital, wasmCloud, Workers)

Contributing

Testing the CLI

From the root of the repo, run the following to see a basic validation report:

  • make test-cli
  • make empty-cli
  • make unknown-cli

test/ contains a mod.yaml, which declares expected properties of a WebAssembly module, as well as a spidermonkey.wasm file to use as example input to use for the validation. wasm/ contains a set of WebAssembly binaries downloaded from the wapm package manager used for analysis and testing.


proto Protobuf definitions and libraries

This directory contains the Protobuf definitions for the types used in the API. Messages have various levels of documentation as well as endpoints if they are request types. Use the api.proto to generate a language client if you'd like to interact with Modsurfer API programmatically from your application.

modsurfer's People

Contributors

bhelx avatar g4vi avatar neuronicnobody avatar nilslice avatar shivaylamba avatar wikiwong avatar zshipko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

modsurfer's Issues

Resource Limit check fail on size.max.

Command to reproduce :

  cargo run generate -p /tmp/tfjs-backend-wasm.wasm -o /tmp/tfjs-backend-wasm.yaml
  cargo run validate -p /tmp/tfjs-backend-wasm.wasm -c /tmp/tfjs-backend-wasm.yaml

(* tfjs-backend-wasm.wasm can be found here)

Result :

┌────────┬──────────┬──────────────┬───────────┬────────────────┬──────────┐
│ Status │ Property │ Expected     │ Actual    │ Classification │ Severity │
╞════════╪══════════╪══════════════╪═══════════╪════════════════╪══════════╡
│ FAIL   │ size.max │ <= 303.8 KiB │ 303.8 KiB │ Resource Limit │ |        │
└────────┴──────────┴──────────────┴───────────┴────────────────┴──────────┘

Expected Result :
Validation passes successfully.

looks like a Serialization/Deserialization issue :
Manual serialization in :

size.max = Some(human_bytes(module.size as f64));

Default Deserialization in :
let mut validation: Validation = serde_yaml::from_slice(&buf)?;

ci: add checks against known checkfiles for pushed code

Currently, the CI workflows only "test" that the checked in checkfiles still work against the latest published GitHub Action. Not that the actual code in this repo works against any tests when PRs are created.

We need to at minimum run cargo run -- validate ... against these checkfiles, so that we're doing a test against the changed code in the PR.

consider changing imports `include` name

After the recent change in #45, we broke a Tezos example: https://github.com/dylibso/modsurfer/actions/runs/4928214424/jobs/8806287884

This is because the checkfile only has a single import, given that the past semantics of imports.include were such that a module had to have at least the specified imports. Now that we've updated this to mean "the module must have a subset of these imports, but no others", the validation fails for this example.

A consideration would be to change the name from includes here to something less restrictive... maybe allow ?

Undefined behaivour in dependency extism 0.4.0 : `extism::Plugin.call_map`

I am using the last version available on github.

Command to reproduce :
cargo run -- generate -p ~/apps/rust/llama2.rs.wasm/www/pkg/llama2_rs_wasm_bg.wasm -o /tmp/mod.yml
the wasm file is available here

Result :
���MzU;-�A����enabled for shared memories (at offset 0x80d)

Expected Result :
threads must be enabled for shared memories (at offset 0x80d)

My first guess was related to non UTF-8 characters, I checked the Error String and its UTF-8 compliant.
Most certain this related to lifetime and FFI bounds on 'Plugin::Internal' and 'Plugin::Internal::last_error'.
A chain of calls pass around a reference of a owned string and eventually display it to the user, except the owner of that string (Internal) gets freed before.

My naive fix would be to allocate a string for the error as soon we have it, it doesn't cost that much and it should have an impact only on non-happy paths (Errors).
In extism 0.4.0

diff --git a/src/plugin.rs b/src/plugin.rs
index 150ce69..3289e49 100644
--- a/src/plugin.rs
+++ b/src/plugin.rs
@@ -255,7 +255,7 @@ impl<'a> Plugin<'a> {
             let err = unsafe { bindings::extism_error(context, self.id) };
             if !err.is_null() {
                 let s = unsafe { std::ffi::CStr::from_ptr(err) };
-                return Err(Error::msg(s.to_str().unwrap()));
+                return Err(Error::msg(s.to_str().unwrap().to_string()));
             }

             return Err(Error::msg("extism_call failed"));

N.B : I tried to upgrade extism to the last version but it seems that there is a lot of differences. (I imagine it's a WIP)

Trying to use ModSurfer with GitLab CI

👋 I'm trying to use ModSurfer with GitLab CI. I did several ways.

Use the official image (distroless images are not usable with GitLab CI)

1-analyze-go-plugin:
  stage: analyze-plugins
  allow_failure: true
  image: ghcr.io/dylibso/modsurfer:latest
  variables:
    WASM_MODULE: "hello-go.wasm" 
  artifacts:
    paths:
      - ${WASM_MODULE}    
  script: |
    ./cli/modesurfer validate -p ${WASM_MODULE} -c ${WASM_MODULE}.yaml

But it failed with error: unrecognized subcommand 'sh'

And it's probably because of the distroless image: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26501

Create my image (but I'm doing my baby steps with docker)

So I try to build my image (adding a shell)

FROM rust:latest as builder

RUN update-ca-certificates
RUN git clone https://github.com/dylibso/modsurfer.git

WORKDIR /modsurfer
COPY . .

RUN cargo build --release

FROM busybox:1.36.0-uclibc as busybox

FROM gcr.io/distroless/cc

LABEL org.opencontainers.image.source=https://github.com/dylibso/modsurfer
LABEL org.opencontainers.image.description="Modsurfer CLI image, learn more at https://dev.dylib.so/docs/modsurfer/cli"
LABEL org.opencontainers.image.licenses=Apache-2.0

# Copy the static shell into base image.
COPY --from=busybox /bin/sh /bin/sh

WORKDIR /cli

COPY --from=builder /modsurfer/target/release/modsurfer ./
#ENTRYPOINT ["/cli/modsurfer"]

And I can run docker run ${IMAGE_NAME} ./modsurfer --version with success, but I need to digg more because, when I run the new CI Job, the CLI cannot "see" my file (I probably forget something with the docker file).

2-analyze-go-plugin:
  stage: analyze-plugins
  allow_failure: true
  image: registry.gitlab.com/k33g_org/modsurfer-experiments/modsurfer-image/modsurfer-for-gitlab:0.0.0
  variables:
    WASM_MODULE: "hello-go.wasm" 
  artifacts:
    paths:
      - ${WASM_MODULE}    
  script: |
    ./cli/modesurfer validate -p ${WASM_MODULE} -c ${WASM_MODULE}.yaml

Download the release file (dependency issue)

3-analyze-go-plugin:
  stage: analyze-plugins
  allow_failure: true
  image: gitpod/workspace-full:latest
  artifacts:
    paths:
      - ${WASM_MODULE}
  variables:
    WASM_MODULE: "hello-go.wasm" 
  before_script:
    - curl -L https://github.com/dylibso/modsurfer/releases/download/v0.0.7/modsurfer-x86_64-unknown-linux-gnu.tar.gz > modsurfer-x86_64-unknown-linux-gnu.tar.gz
    - tar -C ./ -xzf modsurfer-x86_64-unknown-linux-gnu.tar.gz
  script: |
    ./modsurfer validate -p ${WASM_MODULE} -c ${WASM_MODULE}.yaml

When I try to execute modsurfer I get this message:

./modsurfer: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

Using cargo run (it works 🎉 but it's little bit long)

4-analyze-go-plugin:
  stage: analyze-plugins
  image: gitpod/workspace-full:latest
  allow_failure: true
  artifacts:
    paths:
      - ${WASM_MODULE}
  variables:
    WASM_MODULE: "hello-go.wasm" 
  script: |
    git clone https://github.com/dylibso/modsurfer.git
    cd modsurfer/cli
    cargo run -- validate -p ../../${WASM_MODULE} -c ../../${WASM_MODULE}.yaml

If you have any idea about the dependency issue:

./modsurfer: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

It will solve my problem

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.