Giter VIP home page Giter VIP logo

go-pdk's Introduction

Overview

Extism is a lightweight framework for building with WebAssembly (Wasm). It supports running Wasm code on servers, the edge, CLIs, IoT, browsers and everything in between. Extism is designed to be "universal" in that it supports a common interface, no matter where it runs.

Note: One of the primary use cases for Extism is building extensible software & plugins. You want to be able to execute arbitrary, untrusted code from your users? Extism makes this safe and practical to do.

Additionally, Extism adds some extra utilities on top of standard Wasm runtimes. For example, we support persistent memory/module-scope variables, secure & host-controlled HTTP without WASI, runtime limiters & timers, simpler host function linking, and more. Extism users build:

  • plug-in systems
  • FaaS platforms
  • code generators
  • web applications
  • & much more...

Run WebAssembly In Your App

Pick a SDK to import into your program, and refer to the documentation to get started:

Type Language Source Code Package
Rust SDK Rust SDK https://github.com/extism/extism/tree/main/runtime Crates.io
JS SDK JS SDK https://github.com/extism/js-sdk
(supports Web, Node, Deno & Bun!)
NPM
Elixir SDK Elixir SDK https://github.com/extism/elixir-sdk Hex
Go SDK Go SDK https://github.com/extism/go-sdk Go mod
Haskell SDK Haskell SDK https://github.com/extism/haskell-sdk Hackage
Java SDK Java SDK https://github.com/extism/java-sdk Sonatype
.NET SDK .NET SDK https://github.com/extism/dotnet-sdk
(supports C# & F#!)
Nuget
OCaml SDK OCaml SDK https://github.com/extism/ocaml-sdk opam
Perl SDK Perl SDK https://github.com/extism/perl-sdk N/A
PHP SDK PHP SDK https://github.com/extism/php-sdk Packagist
Python SDK Python SDK https://github.com/extism/python-sdk PyPi
Ruby SDK Ruby SDK https://github.com/extism/ruby-sdk RubyGems
Zig SDK Zig SDK https://github.com/extism/zig-sdk N/A
C SDK C SDK https://github.com/extism/extism/tree/main/libextism N/A
C++ SDK C++ SDK https://github.com/extism/cpp-sdk N/A

Compile WebAssembly to run in Extism Hosts

Extism Hosts (running the SDK) must execute WebAssembly code that has a PDK library compiled in to the .wasm binary. PDKs make it easy for plug-in / extension code authors to read input from the host and return data back, read provided configuration, set/get variables, make outbound HTTP calls if allowed, and more.

Pick a PDK to import into your Wasm program, and refer to the documentation to get started:

Type Language Source Code Package
Rust PDK Rust PDK https://github.com/extism/rust-pdk Crates.io
JS PDK JS PDK https://github.com/extism/js-pdk N/A
Go PDK Go PDK https://github.com/extism/go-pdk Go mod
Haskell PDK Haskell PDK https://github.com/extism/haskell-pdk Hackage
AssemblyScript PDK AssemblyScript PDK https://github.com/extism/assemblyscript-pdk NPM
.NET PDK .NET PDK https://github.com/extism/dotnet-pdk
(supports C# & F#!)
https://www.nuget.org/packages/Extism.Pdk
C PDK C PDK https://github.com/extism/c-pdk N/A
Zig PDK Zig PDK https://github.com/extism/zig-pdk N/A

Support

Discord

If you experience any problems or have any questions, please join our Discord and let us know. Our community is very responsive and happy to help get you started.

Usage

Head to the project website for more information and docs. Also, consider reading an overview of Extism and its goals & approach.

Contribution

Thank you for considering a contribution to Extism, we are happy to help you make a PR or find something to work on!

The easiest way to start would be to join the Discord or open an issue on the extism/proposals issue tracker, which can eventually become an Extism Improvement Proposal (EIP).

For more information, please read the Contributing guide.


Who's behind this?

Extism is an open-source product from the team at:

Reach out and tell us what you're building! We'd love to help: [email protected]

go-pdk's People

Contributors

bhelx avatar chrisdickinson avatar egonelbre avatar mhmd-azeez avatar nilslice avatar syke99 avatar zshipko avatar zwiterrion avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

go-pdk's Issues

Imports (Host Functions) example not functioning

I have having trouble getting the "Imports (Host Functions)" example to function.

main.go (using go v1.21.6 and go-pdk v1.0.1)

package main

import (
	"github.com/extism/go-pdk"
)

func main() {}

//go:wasmimport extism:host/user a_python_func
func aPythonFunc(uint64) uint64

//export hello_from_python
func helloFromPython() int32 {
	msg := "An argument to send to Python"
	mem := pdk.AllocateString(msg)
	defer mem.Free()
	ptr := aPythonFunc(mem.Offset())
	rmem := pdk.FindMemory(ptr)
	response := string(rmem.ReadBytes())
	pdk.OutputString(response)
	return 0
}

compiled with tinygo (0.30.0):
tinygo build -target=wasi -o plugin.wasm main.go

app.py (extism 1.0.1 installed via pip)

from extism import host_fn, Function, ValType, Plugin

@host_fn
def a_python_func(plugin, input_, output, _user_data):
    # The plug-in is passing us a string
    input_str = plugin.input_string(input_[0])

    # just printing this out to prove we're in Python land
    print("Hello from Python!")

    # let's just add "!" to the input string
    # but you could imagine here we could add some
    # applicaiton code like query or manipulate the database
    # or our application APIs
    input_str += "!"
# 
    # set the new string as the return value to the plug-in
    plugin.return_string(output[0], input_str)

functions = [
    Function(
        "a_python_func",
        [ValType.I64],
        [ValType.I64],
        a_python_func,
        None
    )
]

manifest = {"wasm": [{"path": "./plugin.wasm"}]}
plugin = Plugin(manifest, functions=functions, wasi=True)
#result = plugin.call('hello_from_python').decode('utf-8') <- function name has a typo
result = plugin.call('helloFromPython').decode('utf-8')
print(result)

Executing results in the following error:

$ python app.py
Traceback (most recent call last):
  File "app.py", line 21, in <module>
    Function(
  File "/home/user/extism_wasm_test/lib/python3.8/site-packages/extism/extism.py", line 254, in __init__
    self.pointer = _lib.extism_function_new(
TypeError: initializer for ctype 'void(*)(struct ExtismCurrentPlugin *, ExtismVal *, unsigned long, ExtismVal *, unsigned long, void *)' must be a cdata pointer, not function

Add `Host.HttpRequest`

We should add a wrapper around extism_http_request to make it more usable from Go.

One blocker here is the lack of JSON support, I think we would either have to generate a JSON object manually or pull in a new dependency.

100kb larger size due to unicode tables

I'm not sure whether to consider this just unfortunate, anyways...

Currently the go-pdk depends on github.com/valyala/fastjson and also strings, which end up bringing in all of the unicode tables. However, ideally for the minimal use, you wouldn't want the unicode tables to be included.

Although, I agree, it might be difficult to achieve in practice.

Implement http.RoundTripper

To make the HTTP function more accessible to Go developers, it would be nice if go-pdk implemented http.RoundTripper to process standard HTTP requests. This way, a larger ecosystem of middleware can be used by plugins.

Ideally, this would allow a plugin like this:

package main

import (
    "net/http"

    "github.com/extism/go-pdk"
)

//export http_get_example1
func http_get_example1() int32 {
    req, _ := http.NewRequest("GET", "https://example.org", nil)
    resp, _ := pdk.HttpClient.Do(req)
    // Process the response

    return 0
}

//export http_get_example2
func http_get_example2() int32 {
    client := http.Client{
        RoundTripper: pdk.RoundTripper{},
    }

    req, _ := http.NewRequest("GET", "https://example.org", nil)
    resp, _ := client.Do(req)
    // Process the response

    return 0
}

There might be quite some things that the pdk.RoundTripper cannot support (streaming responses, for example). But since that's not supported anyways, we could just return an error explaining what isn't supported. For basic requests/responses this should be relatively doable.

The `http.RoundTripper interface is an easy one to implement:

https://github.com/golang/go/blob/a031f4ef83edc132d5f49382bfef491161de2476/src/net/http/client.go#L117-L143

Thoughts?

Bug: Can't compile the samples on Windows

I tried compiling the samples in example folder of this repo on Windows, but it seems like the C FFI is not working as expected. The examples build just fine on Windows Subsystem for Linux.

Versions:

Windows: Windows 11 - 22H2 (OS Build 22621.1848)
Go: go1.20.5 windows/amd64
TinyGo: 0.28.1 windows/amd64 (using go version go1.20.5 and LLVM version 15.0.0)

Logs

Mo in ~ λ go version
go version go1.20.5 windows/amd64
Mo in ~ λ tinygo version
tinygo version 0.28.1 windows/amd64 (using go version go1.20.5 and LLVM version 15.0.0)
Mo in D:\Explore\go\go-pdk\example on main ● ~1 λ tinygo build -o main.wasm .\main.go
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_load_u64
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_load_u8
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_alloc
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_store_u64
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_store_u8
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_var_get
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_length
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_input_length
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_input_load_u64
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_input_load_u8
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_var_set
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_config_get
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_length
wasm-ld: error: C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main.o: undefined symbol: extism_output_set
failed to run tool: wasm-ld
error: failed to link C:\Users\Mo\AppData\Local\Temp\tinygo3081171661\main: exit status 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.