istio-ecosystem / wasm-extensions Goto Github PK

View Code? Open in Web Editor NEW

105.0 10.0 34.0 243 KB

Contains miscellaneous Wasm extensions for Istio

License: Apache License 2.0

Starlark 5.01% C++ 76.79% Go 6.33% Zig 3.33% Shell 3.64% C 0.16% HTML 4.75%

wasm-extensions's Introduction

Istio Ecosystem Wasm Extensions

This repository contains several canonical Wasm extensions, which intend to demonstrate:

Wasm extension development patterns.
Best practice to test, build, and release a Wasm extension.

Extensions

Basic auth enforces basic auth based on request host, path, and methods. In this extension, you can find how to perform local auth decision based on headers and local reply, as well as JSON configuration string parsing and base64 decoding.
C++ scaffold provides an empty C++ extension, which can be used as a starting point to write a C++ Wasm extension.
gRPC access logging makes a logging request to a gRPC service with various kinds of request and workload attributes. In this extension, you can find how to perform asynchronous telemetry reporting, fetch various request attributes and proxy properties, use protobuf and make gRPC callout.
JWT based routing (WIP) reads JWT token information from Envoy dynamic metadata written by JWT auth filter, update host header accordingly, and trigger routing recomputation. In this extension, you can find how to read dynamic metadata, manipulate headers, and affect request routing.
Local rate limit applies a token bucket rate limit to incoming requests. Each request processed by the filter utilizes a single token, and if no tokens are available, the request is denied. In this extension you can find how to share data across all plugin VMs and deny request with local reply.
Open Policy Agent client makes HTTP callout to an Open Policy Agent (OPA) server and based on OPA server response decides whether to allow or deny an incoming request. A result cache is also included to avoid expensive callout on every request. In this extension, you can find how to perform HTTP callout, and asynchronously continue or stop an incoming request based on the response of HTTP call. You will also find how to record stats, which can be scraped in the same way as Istio standard metrics.
Zig scaffold provides an empty Zig extension, which can be used as a starting point to write a Zig Wasm extension.

Development Guides

Write a Wasm Extension with C++

Integration Test

Write integration test with Istio proxy for Wasm extension

Tips & Tricks

Wasm development tips and tricks

wasm-extensions's People

Contributors

Stargazers

Watchers

wasm-extensions's Issues

Publish precompiled Wasm modules

Currently the build pipeline only publishes non-precompiled modules. It needs to be extended to also publish precompiled modules.

Add open policy engine Wasm extension

This is to track adding an OPA Wasm extension, which extracts a small set of request attributes and sends out to an OPA policy engine. It will also have result cache built in. This goal of this extension is to demonstrate Wasm extension as a call out policy check extension point. The extensions needs to have unit test converage, integration test, and documentation about how to customize it to fit bespoke needs.

Add wiki guide on how to test an Wasm extension

Add wiki pages about how to add unit test and integration test to a Wasm extension. This will be used by an Istio blog post.

Is there a problem with the attributegen match logic?

When I use the following code, I found that as long as the first condition does not match successfully, then my second match will never go. Is this a bug in the code? ? ?

patch:
      operation: INSERT_BEFORE
      value:
        name: istio.attributegen
        typed_config:
          "@type": type.googleapis.com/udpa.type.v1.TypedStruct
          type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          value:
            config:
              configuration:
                "@type": type.googleapis.com/google.protobuf.StringValue
                value: |
                  {
                    "attributes": [
                      {
                        "output_attribute": "x_route",
                        "match": [
                          {
                            "value": "x-gray",
                            "condition": "request.headers['end-user'] == 'mark'"
                          },
                          {
                            "value": "x-normal"
                          }
                        ]
                      }
                    ]
                  }
              vm_config:
                runtime: envoy.wasm.runtime.null
                code:
                  local: { inline_string: "envoy.wasm.attributegen" }

source code:

// class Match
// Returns the result of evaluation or nothing in case of an error.
std::optional<bool> Match::evaluate() const {
  if (condition_.empty()) {
    return true;
  }

  std::optional<bool> ret = {};

  const std::string function = "expr_evaluate";
  char* out = nullptr;
  size_t out_size = 0;
  auto result = proxy_call_foreign_function(function.data(), function.size(),
                                            reinterpret_cast<const char*>(&condition_token_),
                                            sizeof(uint32_t), &out, &out_size);

  if (result != WasmResult::Ok) {
    LOG_TRACE(absl::StrCat("Failed to evaluate expression:[", condition_token_, "] ", condition_,
                           " result: ", toString(result)));
  } else if (out_size != sizeof(bool)) {
    LOG_TRACE(absl::StrCat("Expression:[", condition_token_, "] ", condition_,
                           " did not return a bool, size:", out_size));
  } else {
    // we have a bool.
    bool matched = *reinterpret_cast<bool*>(out);
    ret = std::optional<bool>{matched};
  }

  if (out != nullptr) {
    free(out);
  }

  return ret;
}

istio-proxy log:

Support basicAuth per hostname

In our setup we have multiple hostnames routed against one istio ingess gateway pod so we have to setup basic auth for each hostname differently.

How to get upstream ip address from a WASM filter

I am trying to get the upstream host ip address from a WASM filter using getValue() method. I tried ,

std::string remote_ip;
auto buffer = getValue({"upstream", "address"}, &remote_ip);
LOG_INFO("remote_ip >>>>>>>>>>>>>>>"+remote_ip);

but it's not working. It returns env.proxy_get_property return: 1 which I assume should be 0 if everything's correct. I had a look at this method https://github.com/envoyproxy/envoy/blob/c93d486c2d303dbf2cc9f88717de0bfd66e1afdb/source/extensions/common/wasm/context.cc#L490 from envoy source code and they have implemented it. Anything wrong with the way i'm using it ?

Add a helper script to generate proxy sdk http_archive based on Istio version

Add WAF extension

This is the track the effort to get #2 in. The remaining work includes:

Add unit test and integration test
Use bazel for its build
Add doc about how to customize the extension

cc @fpesce I will add guide on how to use bazel and add tests coverage to a Wasm extension.

wasm extension with gRPC bidi streams

Is there any example of a wasm extension that uses gRPC bidi streaming ? For gRPC unary, there are some examples. But for gRPC bidi streams couldn't find any examples. I would really appreciate if someone can provide a sample code snippet of using grpc bidi streams from a filter since there are no resources currently available for that. Thanks

Add local rate limit Wasm extension

This is to track adding a local rate limiting extension, which make local rate limit decision based on request attributes extracted from the request. The extension should have unit test coverage, integration test coverage, and guide on how to customize for bespoke needs.

cc @gargnupur

Use WasmPlugin in examples

Examples like https://github.com/istio-ecosystem/wasm-extensions/tree/master/extensions/basic_auth use envoyfilter. Should move to WasmPlugin

Add wasm-oidc-plugin

Hey there,

i figured it might be useful to include https://github.com/antonengelhardt/wasm-oidc-plugin/ inside this repo. Or are only Wasm Extension written in C++ featured?

Cheers

Update basic auth filter to consume base64 encoded password

Currently basic auth filter only consumes plain text username:passwd. This is to track the work to make it read base64 encoded username:passwd instead.

Create example of Wasm Singleton

Wasm singletons are used to build state centrally, and example that shows how to do the following

Populate shared state that can be used from normal filters.
Show how this state can be updated, either thru XDS or some other means.

cc @bianpengyuan

@mdhume since you use bootstrap extension, do you mind contributing an example plugin?

Add JWT header mutation Wasm extension

This is to track the work to add Wasm extension which mutates request header based on Jwt Header and recompute route based on the new header. The extension should have unit test coverage, integration test coverage, and guide on how to customize for bespoke needs.

cc @mandarjog

grpc_logging example get request body

hi! how to get a request body in the grpc_logging example .

std::string request_body;
getValue({"response", "body"}, &request_body);

Can I get it this way?

How to reference Kubernetes secrets in WasmPlugins?

I am new to Istio and Kubernetes

Is there a way to reference Kubernetes secret for the basic-auth plugin instead of hard-coding values in CRD object?

apiVersion: v1
kind: Secret
metadata:
  name: basic-auth-creds
type: Opaque
data:
  user: ok
  password: test
  user_password: "ok:test"
---
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: basic-auth
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  url: oci://ghcr.io/istio-ecosystem/wasm-extensions/basic_auth:1.12.0
  phase: AUTHN
  pluginConfig:
    basic_auth_rules:
      - prefix: "/productpage"
        request_methods:
          - "GET"
          - "POST"
        credentials:
          - "ok:test"
          - "YWRtaW4zOmFkbWluMw=="

Could you provide an example how to reference the basic-auth-creds secret in pluginConfig?

Add guide on how to build a Wasm extension

This is to track adding guide about how to build a wasm extension, including:

minimum dep requirement
How to make a bazel workspace for Wasm extension

Verify C++ Wasm extension walkthrough guide

We recently added a walk through guide for C++ Wasm extension development: https://github.com/istio-ecosystem/wasm-extensions/blob/master/doc/write-a-wasm-extension-with-cpp.md. Before sharing it more broadly, it would be great if we can get some feedback about it internally from folks who do not work on writing Wasm extensions day by day.

@douglas-reid @richardwxn It would be really nice if you could please take a look it. Thanks!

Help required for deploying basic auth wasm extension

Hi there maintainers,

I was trying to deploy the basic auth wasm module on Istio 1.12.0-beta with book info workload but failed to do so. Can anyone point me to where I am making a mistake? I did the following:

create kind cluster

kind create cluster

install istio components to istio-system namespace

istioctl install -y

label default namespace for automatic injection of sidecar proxies

kubectl label namespace default istio-injection=enabled

kubectl apply -f [bookinfo.yaml](https://raw.githubusercontent.com/istio/istio/release-1.11/samples/bookinfo/platform/kube/bookinfo.yaml)

kubectl apply -f [bookinfo-gateway.yaml](https://raw.githubusercontent.com/istio/istio/release-1.11/samples/bookinfo/networking/bookinfo-gateway.yaml)

kubectl apply -f [gateway-filter.yaml](https://raw.githubusercontent.com/istio-ecosystem/wasm-extensions/master/extensions/basic_auth/config/gateway-filter.yaml)

curl to /productpage seems to be same before and after applying the filter. What am I missing here?

Add a telemetry extension

This is to track adding a telemetry extension, which extracts some request attributes and make a log HTTP call to an external telemetry service. The goal of this extension is to demonstrate telemetry use case with Wasm extension. The extension should have unit test coverage, integration test coverage, and guide on how to customize for bespoke needs.

Can basic_auth wasm only be used with gateway resources?

Hi everyone, I have a wasm problem and I hope someone can help me.

I used the community's basic_wasm, its address is: https://github.com/istio-ecosystem/wasm-extensions/tree/master/extensions/basic_auth

But I changed some configurations, I want to use it in the following way, my purpose is to do interface permission verification.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  labels:
    app: reviews
  name: notice-reviews
  namespace: demo
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      listener:
        filterChain:
          filter:
            name: envoy.http_connection_manager
      proxy:
        proxyVersion: ^1\.11.*
    patch:
      operation: INSERT_BEFORE
      value:
        config_discovery:
          config_source:
            ads: {}
            initial_fetch_timeout: 0s
          type_urls:
          - type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
        name: notice-reviews
  - applyTo: EXTENSION_CONFIG
    match: {}
    patch:
      operation: ADD
      value:
        name: notice-reviews
        typed_config:
          '@type': type.googleapis.com/udpa.type.v1.TypedStruct
          type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          value:
            config:
              configuration:
                '@type': type.googleapis.com/google.protobuf.StringValue
                value: |
                  {
                     "basic_auth_rules": [
                       {
                         "prefix": "/reviews",
                         "request_methods":[ "GET", "POST" ]
                       }
                     ]
                   }
              vm_config:
                code:
                  remote:
                    http_uri:
                      uri: http://xxxx/wasm/basic_auth.wasm
                runtime: envoy.wasm.runtime.v8
                vm_id: notice-reviews
  workloadSelector:
    labels:
      app: reviews

When I applied the envoyfilter to the workload, it reported an error:

2023-03-07T07:42:14.653677Z	warning	envoy wasm	wasm log notice-reviews: [extensions/basic_auth/plugin.cc:306]::configure() cannot parse plugin configuration JSON string: {
   "basic_auth_rules": [
     {
       "prefix": "/reviews",
       "request_methods":[ "GET", "POST" ]
     }
   ]
 }

I don't understand what's going on here, is basic_auth wasm only enabled with gateway ? Can someone help me, it's urgent. .

Add release automation to keep in sync with Istio release

Wasm extensions in this repo will be released out of band, but keep in sync with Istio release so that they can be easily used in Istio docs without version incompatibility concern. This issue is to track the work to automate release process which

Watches Istio new branch/tag, and create the same branch/tag correspondingly.
Push Wasm modules with the same Istio tag.

The same release process should be reusable for users who fork or clone this repo for their own extensions.

Add a WAF extension

This is the track the effort to get #2 in. The remaining work includes:

Add unit test and integration test
Use bazel for its build
Add doc about how to customize the extension

cc @fpesce I will add guide on how to use bazel and add tests coverage to a Wasm extension.

Build a Wasm Extension with C++ is always failed

Build according to Develop a Wasm extension with C++，commond bazel build //:example.wasm is failed：
ERROR: C:/users/administrator/_bazel_administrator/e2vfoios/external/proxy_wasm_cpp_sdk/toolchain/BUILD:21:10: no such package '@emscripten_toolchain//': The repository '@emscripten_toolchain' could not be resolved and referenced by '@proxy_wasm_cpp_sdk//toolchain:all' ERROR: Analysis of target '//:example.wasm' failed; build aborted: Analysis failed

Package built binary as OCI images, push to GitHub Container Registry

Then we can leverage these built OCI images in the Istio tests (in addition to the fake ones as I did in the unit tests https://github.com/istio/istio/blob/master/pkg/wasm/imagefetcher_test.go#L80-L103). I've already done this in Proxy-Wasm Go SKD repository (https://github.com/orgs/tetratelabs/packages/container/package/proxy-wasm-go-sdk-examples).

Note that GHCR is free for OSS projects as long as the registry is public.

I can take care of this if it's ok. wdyt? @bianpengyuan

Issue with grpc_logging example

Hi - I tried to compile the grpc logging example and received the following error:
wasm_compile_log_1 | In file included from examples/wasm-cc/plugin.cc:1: wasm_compile_log_1 | examples/wasm-cc/plugin.h:1:10: fatal error: 'extensions/grpc_logging/config.pb.h' file not found wasm_compile_log_1 | #include "extensions/grpc_logging/config.pb.h" wasm_compile_log_1 | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ wasm_compile_log_1 | 1 error generated.

Are there files missing? or am I doing something wrong? Thanks!