Hello all, I'm cancelling this week's WASI Subgroup meeting, as some folks are travelling, and currently the import things to be focusing on are the various IDL discussions, which we're looking forward to building on top of once they're ready. See you on 2019-08-15!

__wasi_countcount_t is currently a uint32_t. We should change it to uint64_t in order to support hypothetical filesystems with very large link counts. For comparison, the nlink function in Rust returns a 64-bit value.

This will increase the size of __wasi_filestat_t, however that seems negligible.

For comparison, the userspace nlink_t is already a uint64_t: https://github.com/rust-lang/libc/blob/master/src/wasi.rs#L34.

The documentation for __wasi_environ_get says "The sizes of the buffers should match that returned by __wasi_environ_sizes_get" but the WASI implementations that I tested (wasmtime and wasmer) expect a buffer with space for additional null pointer at the end. The sysroot too implements it that way.

This is in contrast with __wasi_args_get and __wasi_args_sizes_get that say the same thing but do as documented.

I'm not sure if this is a bug or the expected behavior. If it is, it should be documented as such.

Hello,

Context

Been working on a fun little CLI project, to do matrix rain effect using the Wasi bindings, and using wasa.

I'm using Wapm / Wasmer to run my Wasm module.

What I think is the problem

I opened this issue on my project: torch2424/wasm-matrix#1 . Which Essentially comes down to, I can't figure out how to get the terminal size using the current available Wasi bindings.

I think what I would need from Wasi is someway to use a the underlying effect of C's ioctl or termios. Is this something you can do with the current WASI bindings? If so, how? If not, is this planned to be done at some point? I understand that Wasm / Wasi are super young, so no worries if it is planned, not looking for an ETA or anything.

cc @dcode , as I originally opened this at: AssemblyScript/assemblyscript#587 . And cc @binji because we chatted about it earlier today 😂

And thanks for all the hard work! Wasi is super cool / exciting. 😄 Thanks!

Most blockchain functions map well to WASI except for call/transact, which is an RPC augmented with the ability to transfer the platform's native token (e.g., Eth) from the caller to the callee. It's possible to do this out-of band, but not efficiently: it'd require making two transactions, passing around receipts, and handling rollbacks/refunds.

As long as we're going the WASI module route, it might be worth adding a single function to the "blockchain" module to support Ethereum-like blockchains:

__wasi_blockchain_transact(
	callee_addr: *const u8,
	value: u64,
	input: *const u8,
	input_len: u64,
	fd: *mut __wasi_fd_t
) -> __wasi_errno_t;

• callee_addr is a fixed-length byte sequence with length known by the platform. For instance, Ethereum addresses are 20 bytes long.
• value is the amount of native token to transfer from the calling account to the callee
• input/input_len - a C slice containing the input to the transaction. input_len = 0 for simple balance transfer.
• fd - a capability to read the output of the transaction

The errno result would be

• success: ESUCCESS
• no account at callee address: EADDRNOTAVAIL
• insufficient funds for value transfer: ENOBAL (this one would have to be assigned a number
• anything else: application specific
• (running out of computation resources, or gas, is unrecoverable and results in the process aborting)

Upon ESUCCESS, fd would be of type SOCKET_STREAM and have read-only pipe
semantics.

This proposal would support most existing smart contract platforms including those with a synchronous RPC model (e.g. Ethereum, EOS) and those with asynchronous cross-shard RPCs (e.g, Near).

Note: left out of this call function, but perhaps desirable, is the amount of gas allocated to the sub-transaction. The alternative is to simply give the callee the full amount of remaining gas. One would presumably not call untrusted code, so there should be no need to limit gas for security reasons.

The following is a very rough sketch for a WASI manifest design. This sketch takes the approach of encoding a manifest in WebAssembly custom sections. One of the goals here is just to get people thinking about what a manifest should be able to do and how it should work.

The basic idea here is to allow WASI-using modules to "request" certain capabilities (aka file descriptors) be pre-opened and provided to them. fd_prestat_get could then provide information about them.

The manifest section

This would be a WebAssembly custom section, which would be sequenced somewhere around the end of a module. For simplicity, this proposal encodes entries in the manifest as a sequence of varuint32 length + UTF-8 content strings. Requests consist of several strings, terminated by the string ".".

Resource requests

A resource request starts with an identifier of the type of resource being requested:

directory resources

For directory resources, the next string is the logical name, which is a UTF-8 string. Note that this name is just a key, rather than being an actual file path. Mapping the key to a path is implementation-specific and may require input from the end user. There are no significant path separators, such as '/', in this name.

The following names are recognized and implementations may choose to streamline the user experience for these paths:

This may be optionally followed by additional attribute strings:

write depends on list.

Examples:

socket resources

For socket resources, the next string is the name, which is a UTF-8 string, interpreted as a key, in the same way as directory resource names.

The next string specifies the domain, which is one of the following:

The next string specifies the type, which is one of the following:

This is followed by a mode, which is one of the following:

listen sockets

For listen socket resources, the next string is the arity, which is one of the following:

This is optionally followed by a port suggestion, which for ip sockets is : followed by either a number, an IANA port name, or "*".

Examples:

connect sockets

For connect socket resources, an optional destination suggestion may follow.

Destination description for ip sockets starts with an address set, which is either CIDR notation or a domain name in which components may be replaced by "*" to indicate that any name at that level is to be permitted. For IPV6 addresses, the CIDR notation is enclosed in brackets ("[" and "]"). It is followed by ":" and either a port number, an IANA port name, or "*".

Examples:

Note that WASI does not currently support sendto, so datagram sockets can't send packets to non-default addresses.

To make all this work:

The networking features here assume the addition of functions to create/bind/listen/connect sockets.

We've been talking about WASI APIs being provided by WASI "module"s. But a WebAssembly modules built against WASI APIs might also logically be called a WASI modules.

Since WASI APIs are not (in general) provided by WebAssembly modules but by the embedder directly i think it might make more sense to come up with a better term for a WASI API.

Ideas?

• WASI API?
• WASI sys-module?
• WASI component?

Importing a wasm module directly from another wasm module is currently mostly impossible. Using wasm modules together essentially requires it to be done from JS code.

This is slated to change with the esm integration proposal in the works.

That proposal is web-centered, though. Since WASI is positioned as the non-browser standard platform, it should define its relationship with the esm proposal. I'm also interested in how WASI positions itself with regards to WICG's import maps.

Speaking personally, I'm a big fan of npm's dependency tree model as a way to avoid dependency hell (although it can bring subtle problems). In the long run, I want to be able to use a wasm package manager to install a program/library, have the PM install all the dependencies and sub-dependencies in their own directory tree, and never have to worry about incompatible package versions and the like; and be able to just type rm -rf wasm_modules or something similar and be assured that every trace on earth of my dependencies have been erased.

Also, while I'm doing armchair design, I think having a subset of import maps could be pretty useful for serverless computing and the like, to provide easy-to-shim modules as capabilities without having to write your own WASI interpreter:

import api : "@cloudflare.workers";

func main() {
    api.startTask(...);
    // ...
}

In the above pseudocode example, the import statement would compile to an import instruction, which would point to a shim in a test environment, and to a library provided by cloudflare in production.

Thoughts?

As discussed at the end of the subgroup meeting today, we want to schedule an official adjunct meeting during the June CG meeting in A Coruña, Spain.

While an informal meeting was discussed amongst some members out-of-band, @sunfishcode indicated that an official meeting should occur during the CG meeting times, to avoid excluding those whose travel plans are already set in stone. However we still need to determine scheduling, which times are available for the subgroup to meet, and what, if any, conflicts may prevent some folks from attending.

It may be useful to discuss potential agenda topics here, but I'll defer to @sunfishcode and others who may wish to use the current PR process for agendas to manage that discussion.

My current understanding is that this is outside the scope of WASI and that WASI is more like the system call layer, dedicated only to communicating with the embedder. But I think there might be some level confusion (and I could be completely wrong).

If my assumption is correct then I think it might be useful to list inter-module interfaces as explicitly outside the scope of WASI. For example we might say that DLL-linkage schemes or "shared nothing" APIs don't make sense to be specified as WASI interfaces?

Note: this wouldn't prevent the capabilities model from functioning, since handles/capabilities received from WASI API calls could still be passed between modules. It would just mean that the interfaces used between modules (e.g. in a DLL, or shared nothing shceme) is not something that would be specified as a WASI interface.

Hello everyone, as a quick update, I'm working on setting up the first video-call meeting for the group, following the main CG process. It'll likely use zoom.us and follow the same overall format, at least to start with.

While the vote at last week's CG meeting was strongly in favor of launching this Subgroup, to do everything properly we need to officially select a chair and a charter, so we've now added agenda items to the next CG meeting here.

In the meantime, please continue to file issues with questions, comments, and suggestions!

Wasm doesn't have a "weak" import mechanism, which would allow one to (import "foo" "bar") where "bar" may or may not exist in "foo" (or perhaps where "foo" itself may not exist). This would be one way to allow feature detection in APIs.

As one possible solution, it's expected that the wasm GC proposal will eventually give wasm the concept of optional/nullable references, which would permit a kind of weak import where one imports global variables with reference type which can be either references to the desired entities or absent/null. However, GC may not be available in wasm for some time, and even when it is, it may not be implemented in all wasm engines, so it's not entirely obvious whether we should depend on it for something so fundamental to WASI.

Another is to essentially add a weak-import system on top of Wasm. For example, we could use a naming convention, such as a ".weak" suffix, to indicate a weak import, and a ".is_present" suffix on a global variable import to hold a boolean indicating whether a weak import was successfully resolved, like this:

  (func $foo (import "wasi_something" "foo.weak"))
  (global $do_we_have_foo (import "wasi_something" "foo.is_present") i32)

A similar scheme could enable detection of modules too:

   (func $foo (import "wasi_something.weak" "foo"))
   (global $do_we_have_wasi_something (import "wasi_something.is_present" "foo") i32)

or even both at the same time:

   (func $qux (import "wasi_something.weak" "bar.weak"))
   (global $do_we_have_it (import "wasi_something.is_present" "foo.is_present") i32)

Another option would be to have a dynamic API that takes a string, however while we can do that, passing around strings introduces some complexity and it seems like if have a naming convention for ".weak", we might as well have ".is_present" too.

I'm not attached to any of the details here, I'm mainly just looking to start a discussion.

Hey there, everyone!

As I very briefly described in WebAssembly/wasi-sdk#33, WASI at the moment seems to be lacking support for IPC communication. I think that this feature is essential for some applications, including drivers, daemons, window managers as well as applications talking to system-wide data buses.

It looks like there only about a dozen system calls required to make all IPC scenarios run correctly if we decide to follow the POSIX syscalls first.

So, is this possible? What do you think about the amount of effort it will take to implement support for it? The only things that sketch me out are mentions of synchronization primitives.

Thanks!

I think ideally we could represent file descriptors as an anyref once we have reference-types. For the preopened file descriptors we could designate an anyref table which would be exported to load the references into. fd_renumber could be deprecated and table.set/get could be used instead.

Using a table might also affect how preopened fd's are inherited by forking. Instead of forking there it might be nice to have a something that is tied more closely to wasm like module.instantiate -> modRef The parent processes then could populate the preopened file descriptors table before calling the entry point function.

@npmccallum @dumblob

Continuing the discussion from bytecodealliance/wasmtime#90, now that we have a dedicated repository (Github doesn't permit issues to be transferred between repositories).

As an example, let's say we have TCP-like sockets. We want to restrict the WASM to connect() only to a single address. If you're running on Linux (most all of us are), you just put the runtime in a cgroup and set iptables rules on that cgroup. The connect() command will fail on everything but the specified address. But this has nothing to do with the API itself. This is entirely enforced by the host OS. Where the host OS doesn't provide such capabilities, the runtime can add them. But it doesn't impact the API.

"Put the runtime in a cgroup" is a privileged operation on typical Linux systems. And, it requires a dedicated process, which many WebAssembly runtimes won't otherwise need -- WebAssembly's ability to be easily sandboxed without a process boundary is one of the key things many people are interested in it for (caveat: Spectre is a complex topic).

Lin Clark's blog post about WASI has an explanation of the capability model and why we're pursuing it for WASI. In many use cases, it's possible to set up cgroups or AppArmor or SELinux or other things, however these systems all have sufficient obstacles (configuration complexity, and the need for privileged operations), and in practice they aren't always used. And, they're all process-oriented, while the capability model allows WASI to provide more fine-grained protections. And they're all tied to Linux, while we expect WASI will be desirable in a very diverse set of environments.

(File descriptors are integers, and are therefore foregeable, but (a) even so they still provide some protections, and (b) in the future WASI will be able to represent capabilities as references which aren't forgeable.)

Assuming all of this is sensible, the next step is to apply these concepts to networking. We could create the analog of a directory for TCP-like sockets, which might look like a set of address/port/address/port/protocol tuples, possibly involving wildcards and/or netmasks or so, bundled up into a "capability", and presented to application code in the form of a file descriptor. That would then be the basis for something like a bindat system call, which would be to bind as openat is to open.

Something that prevents a web app from appearing as native is that it's an 'app within an app' and doesn't appear in OS-level app switchers.

A web app should be able to express its intent to be used natively and if the user permits, the OS would begin presenting it in its app switcher and other task management tooling separate from its hosting browser. Naturally this would require some form of cooperation between web devs, web standards, browsers, and OSs.

How should those pieces look and fit together, and what could WASI do, if anything, to contribute?

I'm confused on how preopened fds are numbered. The API doc currently says

As in POSIX, three file descriptor numbers are provided to instances on startup -- 0, 1, and 2, (a.k.a. STDIN_FILENO, STDOUT_FILENO, and STDERR_FILENO).

Other than these, WASI implementations are not required to allocate new file descriptors in ascending order.

So how do programs find the preopened fds? Wasmtime currently seems to monotonically increment the fds. Should this be considered part of the spec?

I'm working on Ewasm which tries to bring WebAssembly to Ethereum. We are evaluating how we could use WASI (core) or other modules.

As part of the design process we have realised that versioning is an important feature to have: ewasm/design#121

I wonder how WASI plans to solve versioning of its interfaces. I see some potential options:

• Would it be similar to #12 having a function to query the version?
• A system similar to glibc where either the namespace or the name has a pre/postfix? (This is option a) in the linked issue).
• A custom section listing interface version (This is option b) in the linked issue).
• Adding versioning to the wasm import sections themselves.

Probably there are other/better proposals. Was there any previous discussion about this topic?

A major problem we're seeing with using WASI, in contrast with Emscripten and unsandboxed code, for human-in-the-loop uses of WebAssembly is: how does the user know which capabilities a module will need?

Currently the flow is:

1. User runs WebAssembly module. Module does computation. Module encounters missing capability (preopened file descriptor).
2. Optional step: module causes intermediate state changes and can't run correctly anymore
3. Module terminates with error
4. User adds capability
5. Optional step: User reverse engineers what the module was doing and cleans up after it
6. Repeat steps 1-5 until module works

What we need is a way for the WASI module to ask for more permissions. We've considered adding something like this for WASI as it is now, but I think having it in the WASI standard is the way to go. And in the case of preopened file descriptors, a WASI-native solution will be much nicer than having the host edit the libpreopen data structures in WASI memory.

Alternative solutions:

• Drop the sandbox, grant all permissions by default. Here for completeness -- I think we don't need to discuss this.
• Set things up such that all capabilities can be statically inferred. Even if this were possible (and I don't think it is), having all possible permissions a module might need either on or off seems like a sub-optimal outcome.
• Have WASI modules declare/request their required permissions ahead of time. Very similar to the above with the same draw-backs.
• Have individual runtimes solve this problem independently. Not an unreasonable solution, but will increase fragmentation and the barrier to entry.

Known issues

• Existing code may, for example, look at a bunch of file paths to infer things about the system and having them fail rather than alert the user may be the best thing to do.
• Blocking on requesting permissions in situations where there is no one to grant permissions. We'd have to support both uses.
• Cognitive load on the user: in the case of multiple modules of various levels of trust, presenting the requests in a clear way to the user could be difficult. Having only the top level module able to request permissions and then sub-dividing them from there could help.
• The granularity of preopened directories is very large. If I'm running a CLI tool and I give it a file as a command line argument, I'd expect it to only access that file and maybe its own internal data files, not .ssh if I happen to run it in my home directory.[1]

I think this was mentioned in passing in one of the previous meetings, but I'd like to restart the conversation around this. Getting this right is critical to having WASI and sandboxed/secure computation used for user-facing applications.

Please share your ideas about this, suggestions for alternatives/known issues, and intermediate solutions that you all have been using if you've encountered this issue!

[1]: We've considered special casing uses like this (inferring permissions based on command line arguments) because we solved the "program relies on a pile of files to work correctly" problem for our uses with wapm package filesystems (statically declaring host file dependencies outside the WASI module). This is a band-aid though and is easily defeated, like in the case of a compiler which not only reads the file you gave it as an argument and its pile of files, it needs to output file(s).

This is likely a pretty far-future issue, but I was curious to start a conversation around it early on to see what others think!

As wasi continues to grow syscalls (in one way or another) it's inevitable that syscalls will be implemented in some implementations of wasi but not others. Furthermore some host environments may just simply choose to not provide wasi syscall functionality while others do indeed support it. A standard library compiled to wasm, however, needs some sort of policy about what it can and can't do.

For example would a standard library have to specify a minimum set of syscalls it needs to operate? Would it need a specific threshold saying "your implementation must be as new as X with these syscalls"? I ask about this in the context of implementing the Rust standard library for wasi in contrast with how some other platforms are implemented.

Some examples that have come up elsewhere in the past are:

• On Unix we've got syscalls like accept4 which allow atomically setting CLOEXEC, but it's not available on all kernels that the Rust standard library supports. As a result we dynamically detect at runtime whether accept4 is available and fall back to a "guaranteed to work" accept-then-CLOEXEC if it's not found.

• Some platforms support the idea of the "name of thread", primarily used for debugging when you attach gdb for example. We dynamically detect whether linux platforms have the right symbol we can call to configure this, but if it's not present it's just an opportunistic thing that we can safely ignore.

I'd be curious if we could develop something like this for wasi? For example if a syscall was added to aid in debugging (like the thread intrinsic) then the Rust standard library would want to immediately start using but wouldn't want to require all wasi implementations to provide it.

An example strawman API for this might be something like:

int __wasi_syscall_get(const char* module, const char* name, size_t *function_pointer);

where the wasi runtime would effectively attempt to dynamically import name from module (like a static wasm import). If it doens't find anything then the host would return an error, but otherwise a function would be appended to the instance's runtime table of functions and an index to that would be returned through function_pointer. It'd be up to the implementation to then ensure it has the right type signature of the syscall.

I suspect this is also related to #2 in that it could affect what APIs are used, although it's less so about statically requesting different behavior but rather dynamically requesting such behavior.

So once we have reference types we can use those types for fd but currently we can also enforce nearly the same behavior as if we had reference types by annotating functions. This would enable the host system to enforce the integrity of fd across module instances.

For example lets say we have to have 2 wasm instances (a, b) that share no memory or tables but instance a imports some exported functions from instance b. If those exported functions expected fd in its augments, those augments would be annotated in a custom section so that when a called b the host would enforce that values were valid fd.

The custom sections could looks something like the following

LANGUAGE_TYPES = {
  'i32': 0x7f,
  'i64': 0x7e,
  'f32': 0x7d,
  'f64': 0x7c,
  'fd': 0x7b,
  'anyFunc': 0x70,
  'func': 0x60,
  'block_type': 0x40,
}

The custom section has a name of "wasi.type".

The typeMap section maps the custom types to functions. And is encoded as follows

And a map is Encoded as follows and named "wasi.typeMap".

Lastly the custom type information for globals are encoded as follows. The section name is "wasi.globals"

@sunfishcode @hinzundcode

This is just to continue discussion from bytecodealliance/wasmtime#91 .

Apart from Drawbridge etc. we shall take into account also the surrounding environment of the "near future" (actually already current) at Microsoft. See Universal Windows Platform APIs and the environment to get a grasp of it. Put emphasis especially on the last chapter How the Universal Windows Platform relates to Windows Runtime APIs leading e.g. to CoreApplication Class.

WASI has no concept of a "process", and the current WASI Core has had most parts that depend on this concept removed already, but a few bits remain: __WASI_CLOCK_PROCESS_CPUTIME_ID, proc_exit, and proc_raise.

For proc_exit and __WASI_CLOCK_PROCESS_CPUTIME_ID, one option is to change them from "process" to "store". Implementations which don't wish to support __WASI_CLOCK_PROCESS_CPUTIME_ID may return an error code from __wasi_clock_time_get, __WASI_EINVAL, instead.

proc_raise probably makes sense to remove regardless, since there is no other support for signal handling.

For reference, in wasi-sysroot, __WASI_CLOCK_PROCESS_CPUTIME_ID is used in C functions clock (C standard), times (POSIX), and getrusage (POSIX).

I was curious if there has been any thought about a Cryptography module, or if this is a good place to start a conversation. As JF Bastien points out, one shouldn't implement crypto for Wasm targeting the browser due to security concerns, especially since the browser provides highly-optimized and more secure Web Crypto APIs that run outside the JS context. So with WASI and the lack of the browser, where should crypto live?

Coming from an Embedded electronics/IoT background, it's pretty common for an application to include a library like mbed TLS. It is optimized for the hardware architecture of the host, to be as resource-efficient as possible and leverage hardware-acceleration whenever possible (this is usually handled by build-time configuration.) I'm not familiar with the details of OpenSSL but I believe it is similar.

To that end, what would a Cryptographic module look like, and what concerns would need to be considered? Could it be based on something like mbed TLS?

Love to hear y'alls thoughts!

The current design has a hidden table of implementation-defined objects that are indexed by the integer "file descriptors" exposed through the API.

These "file descriptors" could be more generally called handles, and the table of implementation-defined objects generalized to include other kinds of objects: processes, threads, mutexes, clocks, etc.

This would allow, for example, WASI APIs that work with process objects without exposing a global process ID namespace, and without adding a separate "process descriptor" table that maps indices to process objects.

The impact on the current API would be pretty minimal:

typedef struct __wasi_fdstat_t
{
	__wasi_filetype_t fs_filetype;
	__wasi_fdflags_t fs_flags;
-	// __wasi_rights_t fs_rights_base;
-	// __wasi_rights_t fs_rights_inheriting;
} __wasi_fdstat_t;
-typedef int32_t __wasi_fd_t;
-__wasi_errno_t __wasi_fd_close(__wasi_fd_t fd);
-__wasi_errno_t __wasi_fd_renumber(__wasi_fd_t from,
-                                  __wasi_fd_t to);
-__wasi_errno_t __wasi_fd_fdstat_set_rights(__wasi_fd_t fd,
-                                           __wasi_rights_t base,
-                                           __wasi_rights_t inheriting);
-__wasi_errno_t __wasi_fd_prestat_get(__wasi_fd_t fd,
-                                     __wasi_prestat_t* out_prestat);
+typedef intptr_t __wasi_handle_t;
+typedef uint32_t __wasi_object_kind_t;
+#define __WASI_OBJECT_KIND_FD (UINT32_C(0))
+__wasi_errno_t __wasi_handle_close(__wasi_wasi_handle_t handle);
+__wasi_errno_t __wasi_handle_renumber(__wasi_handle_t from,
+                                      __wasi_handle_t to);
+__wasi_errno_t __wasi_handle_set_rights(__wasi_handle_t handle,
+                                        __wasi_rights_t base,
+                                        __wasi_rights_t inheriting);
+__wasi_errno_t __wasi_handle_get_rights(__wasi_handle_t handle,
+                                        __wasi_rights_t* out_base,
+                                        __wasi_rights_t* out_inheriting);
+__wasi_errno_t __wasi_handle_get_object_kind(__wasi_handle_t handle,
+                                             __wasi_object_kind_t* out_kind);
+__wasi_errno_t __wasi_handle_prestat_get(__wasi_handle_t handle,
+                                         __wasi_prestat_t* out_prestat);

And everywhere the API currently takes a __wasi_fd_t would be changed to take a __wasi_handle_t. EBADF could be aliased as EBADHANDLE, and returned when using an invalid handle. ENOTCAPABLE could returned when using an API function with a handle that references the wrong kind of object (as presumably the handle will not have the capabilities corresponding to the API function), or a distinct error code could be added.

This would apply the same if the integer handles are replaced by anyref:
typedef anyref __wasi_handle_t

Just throwing a bold title out there to look back in few years with satisfaction or disappointment 😝 We all know it needs to happen right? But now I just wonder how it would happen. 🤔

I'd like to hear opinions and crazy ideas on how an operating system around WASI would work, to reduce the overhead of a runtime and the translation to system calls there are projects like cervus that implement a WASM subsystem for Linux, if you take that further and strip out the Linux things you don't need, add package management, daemons, IPC and what not you are left kind of with a wasm linux distro no? But Linux world and its kernel might be too bloated for that WA-OS might need, perhaps a fresh micro kernel which it's only API is the WASI core sys calls and drivers and other stuff as user space modules? I see most WASM things are being build with Rust perhaps inspiration or parts from Redox could be a base of such system?

Another thing to take into account creating an OS is your target users, I guess it wont be the same targeting a desktop, a cloud environment or an embedded device, but maybe this hypothetical OS can fit all use cases? like be a good replacement for linux+containers on the cloud, perhaps have a hybrid scheduler to run real-time critical stuff as well as a GUI?

Talking about containers, I was curious about Docker co-founder post saying if WASI existed back then they wouldn't have needed to create Docker. Sure they overlap but not completely, so what can we do to make them overlap even more and conquer the cloud? 😈 As far as I understand the capability based isolation system is good for sandboxing processes but won't give you other things you get with containers like the network namespaces and cgroups to define custom networking for your app and limit the resources it gets, our OS and some WASI extensions could come in and fill the gaps right?

Let me hear your ideas! List things that such an OS would need and some clever person might see them and get the inspiration to put all pieces together 😉

Currently WASI program are launched by calling a function named _start. This is the name that C compilers have historically used. It's an option question whether we should just use the wasm start function rather than having an export with a designated name. However, as long as _start is not the wasm start function, it'd be good to use a different name to avoid confusion.

So far it seems the most advanced practical capability system is Capsicum. I think it would be nice to correlate their API with WASI API just to avoid some potential mistakes (see e.g. fine grained sub-filedescriptor capabilities, rough comparison of cap systems in 2010, and an example of Capsicum in tcpdump).

Also other capability systems shall be compared in detail with WASI for the same reasons.

In the end an overview of currently existing practical capability systems made of some high-level points from the thorough detailed comparison could help WASI in adoption.

There seem to be two distinct modes of program execution that applications broadly fit into: Commands and Reactors.

• A Command has a "main" function, and when this function returns the program terminates.
• A Reactor has a "setup" function, and when this function returns the program remains live, allowing functions registered as callbacks to be called. There is an event loop in the wasm runtime, rather than in the application.

Reactors could run in the main thread of a browser, but they may also have uses in a variety of settings where applications will primarily be responding to external events. Putting the event loop in the runtime gives the runtime the flexiblity to use a single event loop for multiple purposes.

(I briefly mentioned these ideas here, but I want to give them more visibility here.)

Emscripten provides a kind of hybrid approach, where instances can stick around after "main" exits to allow them to be called by callbacks. For WASI, it may be useful to make an explicit static distinction between Commands and Reactors, because:

• We could limit the APIs available to Reactors. For example, if we want to run Reactors in the main thread of a browser, we could prohibit them from using synchronous I/O APIs.
• We could limit the APIs available to Commands too, for example not allowing them to register for external event-loop APIs since there is no external event loop in a Command.
• Allowing runtimes to know that a program is a Reactor up front can enable some useful optimizations, like switching JIT tiers between turns of the event loop, in a very simple way.

Currently, WASI programs combined with clang fit the Command model, with _start being the "main" function. It might make sense to rename it to __wasi_command_main or something. Programs could declare themselves to be Reactors by exporting a function named something like __wasi_reactor_setup or so.

Some interesting questions:

• Is a static distinction between Command and Reactor too limiting?
• Are there programs which don't fit into either Command or Reactor models?
• Are the restrictions imposed by browser main thread execution too limiting for non-Web use cases?

Even with BitInt in many engines, chrome at least still doesn't support the use of BitInt for i64 WebAssembly bindings.

I know WASI is forward looking so this was most likely a deliberate design choice, but I thought at least mention it here for the record and see what solutions others have come up with.

To fully implement the current unstable syscalls in a JS engine today it seems to me that one would need to create a shim WASM file that split all the i64 values into a pair of i32 and route the problematic syscalls through this layer to JS.

Has anyone else found any other way to do this?

So I'm using wasm with wasmer + cranelift for a kind of scripting system for my application. So users can write little WebAssembly modules to write plugins. Over the weekend I've looked into integrating WASI to give them access to the operating system in a limited fashion (I'm planning to only let them read files and print to stdout / stderr initially). However it seems like so far (at least from Rust, not sure if this is an overall limitation) you can really only create WASI binaries, not libraries. The way my plugin system works is that my application expects certain exports to be there and I call into them, while providing WASI imports + other plugin specific imports to the wasm files. However at least the way it currently is in Rust and the wasi libc sysroot, WASI really only works for the duration of a main function that needs to be there. Calling the wasi functions from anywhere else (through the Rust std) segfaults because libpreopen isn't initialized (anymore). I'm not sure how much this is a design limitation or a just a bug, but overall I'd like my main application to have control over which functions to call. With a main function, the WebAssembly module takes control over the control flow, which isn't great for such an "extension" kind of use case.

It would be nice to have a standard testsuite for the WASI API

(Maybe this is two separate issues?)

It's common for runtimes (or apps) to have to bundle entire copies of collation tables, timezone data, etc with every app distributed. It adds a considerable amount of weight and for web deployment situations it could make things a lot worse - i.e. do we really want people to have to drag along 2-5mb worth of data with every little worker they want to run on edge nodes just because they need to sort strings?

It would make sense for host environments to be able to expose collation, timezone, etc data they have available to the application. Most environments that can host a wasm app have this data already - every browser has to ship ICU, etc.

AFAIK Mono currently ships its own timezone database on some platforms for this reason, and I see people ship ICU everywhere in all sorts of scenarios. It'd be cool if WASI could at least optionally eliminate the need to ship your own copies of ICU and tzdb.

My understanding is that WASI uses a calling convention that makes sense for direct use with languages like C. At the same time, there's ongoing work on WebIDL to make it into a practical, language-neutral API binding layer, with WebIDL bindings.

I'm curious why WebIDL doesn't meet the needs for WASI. When APIs for new capabilities are exposed, what is the advice for API designers, in terms of choosing between WebIDL and WASI-style APIs? What would be the relationship with WebIDL if WASI comes to the Web? If WASI is a path towards exposing new capabilities, like mmap, to the Web, how should those be accessible from JavaScript?

WASI currently has two pairs of functions which are similar to each other: sock_send/sock_recv and fd_write/fd_read. This PR describes a plan for merging them, in favor of fd_write/fd_read.

Background

The reason why send and recv are separate in POSIX is that they add flags arguments. POSIX says that send and recv are equivalent to write and read when no flags are set.

WASI's sock_send doesn't currently support any flags. WASI's sock_recv supports __WASI_SOCK_RECV_PEEK and __WASI_SOCK_RECV_WAITALL which correspond to MSG_PEEK and MSG_WAITALL in POSIX. Both of these operations conceptually could work on files, however typical operating systems only support them on sockets.

On Linux, there is a subtle difference between recv and read: "If a zero-length datagram is pending, read(2) and recv() with a flags argument of zero provide different behavior. In this circumstance, read(2) has no effect (the datagram remains pending), while recv() consumes the pending datagram." It is possible that applications could depend on this subtle difference, but the only reference to it I've been able to find is the git commit which added this line to the man page, which describes a bug where "[...] we would end up in a busy loop when we were using read(2). Changing to recv(2) fixed the issue [...]". The recv behavior, is what the code in that bug wanted, and is the more intuitive behavior.

The cause of this subtlety is that read special-cases a 0 return value to mean the end-of-file/stream has been reached. That creates an ambiguity when reading a zero-length datagram.

Proposal

• Remove sock_send and sock_recv.
• Add __wasi_siflags_t and __wasi_riflags_t arguments to fd_write and fd_read, respectively.
• Make fd_read return __WASI_EMSGSIZE when receiving a datagram which is larger than the provided buffer. And remove __WASI_SOCK_RECV_DATA_TRUNCATED, which is what sock_recv used in that case. WASI libc will check for this and to continue to implement the POSIX API (MSG_TRUNC).
• Make fd_read return __WASI_EEOS, a new errno code, when the end-of-file/stream is reached. This eliminates the ambiguity of the special case for 0. WASI libc will check for this and continue to implement the POSIX API with 0 being a special case.
• Add rights for __WASI_RIGHT_FD_READ_PEEK and __WASI_RIGHT_FD_READ_WAITALL, which are required to use the __WASI_SOCK_RECV_PEEK and __WASI_SOCK_RECV_WAITALL flags, respectively. These rights would not be granted for file-based file descriptors on OS's that don't support these features on files.
• Remove the fs_filetype field from the fdstat_t struct. This further hides unnecessary differences between sockets and files. fd_fdstat_get is an otherwise ambient authority, meaning anyone can do it on any open file descriptor. The file type is still accessible, via fd_filestat_get, but that requires (__WASI_RIGHT_FD_FILESTAT_GET).
• That happens to leave us with no easy way to implement isatty, so add a __WASI_RIGHT_FD_ISATTY right, to indicate whether a file descriptor is known to be a terminal. This is a little unusual as it's not a typical right, as it's not associated with an operation. However, this right makes it simple to implement isatty, which is used by libc to do line buffering for stdout when it's on a tty.

And some minor tidying:

• Rename sock_shutdown to fd_shutdown, and make it a file descriptor operation that happens to depend on the __WASI_RIGHT_SOCK_SHUTDOWN right, which on typical implementations will only get granted for sockets. This is the last remaining sock_* function.
• Rename __WASI_RIGHT_SOCK_SHUTDOWN to __WASI_RIGHT_FD_SHUTDOWN.
• Rename __WASI_SOCK_RECV_PEEK and __WASI_SOCK_RECV_WAITALL to say FD_READ instead of SOCK_RECV.

Miscellaneous notes

The change to make fd_read return __WASI_EEOS on end-of-file/stream also fixes an oddity in POSIX in which many applications do an extra read call after the EOF is encountered, in order to get a 0 return from read to confirm they've actually reached the end. That said, implementations on POSIX hosts won't be able to report __WASI_EEOS until they get a 0 from read themselves, so in practice there will still be an extra read on such systems.

Take this issue with a grain-of-salt.

I've noticed that the current (I understand it's a work in progress) wasi api for polling is similar to modern epoll in many ways. Perhaps, implementations that use wasi's polling api (which will presumably be many/most of them) would be simpler if we adopted a simpler, easier to write against api that mirrors that of rust's futures.

I propose that we export a function called register_callback to modules that has the following signature:

(type $t6 (func (param i32 i64 i32) (result i32)))

In rust, that would be:

extern {
    fn register_callback(handle: u32, key: u64, f: extern fn(u64)) -> u32;
}

There would be supporting machinery around it, but essentially, you'd register an event--on a socket for instance--to this handle value, and then register a callback associated with it. The key parameter is passed, unchanged, to the callback when the event occurs.

The callback would, at the wasm level, be the index of an exported function.

I think this would integrate quite directly with rust's Futures and other languages' async internals, though I haven't tried it of course. Plus, it would likely reduce the amount of async machinery a module would have to carry around with it by a very significant amount.

There's an important unanswered question: Would this callback be called on a separate thread by the runtime (sort of like a signal), meaning that it would have to use atomic instructions to interface with the module's memory, or would the module have to relinquish control to the runtime for the callbacks to start being called in a single-threaded manner?

We are able to retrieve environmental variables via environ_get, but there is no way to change them using Core. How about adding something like setenv/unsetenv (maybe merged into a single function)?

the description of __wasi_prestat_t from __wasi_fd_prestat_get seems to missing. Its defined here in wasmtime but I'm not sure why __wasi_preopentype_t pr_type; can't just be a __wasi_filetype_t

As discussed before (in the CG meeting, offline, and several threads), WASI should be designed to be split up into several modules/namespaces, each with a particular theme.

I'd like to discuss the granularity of the modules, a naming scheme, and versioning.

1. Granularity:

How far should syscalls be split up into modules? Would it work to have a single "fs" module, or would it make sense to split into an "fs" module that works with the filesystem and a "fd" module that works with file-descriptors? How much should the the "wasi core" module contain?

2. Naming:

Is there a preference on the naming scheme used? I'm personally most comfortable with modules being named wasi/<name> or wasi:<name>.

3. Versioning:

Should the WASI modules be versioned? Would we use semver, or something different? My opinion is that there are a lot of upsides to explicit versioning, but maybe it'd be better to keep things under the wasi_unstable umbrella until WASI is standardized.

Hey,
I'm planning to use WASI to compile my Rust code. But my question is if I use WASI, is it compatible to all rust crates?

Or is there any list to which and all crates WASI is supporting?

I think those crates which is having wasm32-unknown-unknown support will execute.

Thanks,
Achala

It might sound too early to be talking about such fancy high level APIs but I think a WASI runtime with hardware accelerated neural networks and related algorithms would have a huge immediate market since it's such a hot topic. Probably this is another of those times when copying the web or following it closely is a good thing, the Web Neural Network API(examples) is being defined to provide this APIs to JS developers but might work even better with WASI.

For a personal project related to the industry I work on there's a use case I'd like to try to make it work, a WASI run-time used as the main platform of an autonomous vehicle, developers could use this and other standardized APIs to program general purpose applications that make use of the hardware capabilities of the car like the vision system, they could even use it to develop more critical parts like the driver agent that they sell in some market place so users have the choice of whom will drive them and not the car manufacturer. All this applications require some brain power available in the platform and having access to just the GPU is not enough since now there is specialized hardware for neural network acceleration and would be important to make use of that hw if it's available.
I was thinking of prototyping the idea with my google coral board that has built-in TPU, also have some nvidia jetson xavier at hand that I could use but I have no idea how google or nvidia talk to their hardware at low level, also for starters the implementation could be CPU and GPU only, on the GPU side this feature could build on top of WebGPU for example #53

That's just my tiny use case but imagine the huge potential and amounts of hype that such a feature could bring, people deploying AI enabled applications in virtually any platform and hardware with ease 🤩

Sets of rights (e.g. __WASI_RIGHT_FD_READ) are currently encoded as a 64-bit bitmask:

typedef uint64_t __wasi_rights_t;
#define __WASI_RIGHT_FD_DATASYNC (UINT64_C(0x0000000000000001))
#define __WASI_RIGHT_FD_READ (UINT64_C(0x0000000000000002))
#define __WASI_RIGHT_FD_SEEK (UINT64_C(0x0000000000000004))
// etc..

This limits the API to 64 rights, of which 29 are currently used. It seems likely that WASI will exceed that limit at some point. The simple solution would be to just double or quadruple the width of __wasi_rights_t. Maybe 128 or 256 rights is enough?

A more complex solution would be to make everywhere that deals with rights take a variable length array of 64-bit masks. An API version could define a maximum length, so both sides of the API could use statically sized arrays. As long as the API functions take a dynamic length argument, their signature doesn't need to change if the maximum length increases.

There's also an issue with the current fd_fdstat_set_rights function (or the handle_set_rights of #62): it can only be used to remove rights, but it's easy to accidentally remove rights you don't know about! It should perhaps take a set of rights to remove, instead of a set of rights to keep.

Wasm has a phase process that describes how proposed features progress toward standardization. The process is designed to make it clear how to add to the wasm standard, both in terms of who should be involved and when, and the practical steps needed to ensure that we have all the artifacts we need before we declare things "done".

We'd like to have something similar for standardizing WASI, and it could be broadly similar.
Most of the steps in the process correspond fairly well to what we might want for WASI, and I expect to be uncontroversial but there are a couple of potentially-interesting bits to call out:

1. Aside from spec text and spec tests (which I assume we'll want too) the wasm spec has a reference interpreter, blessed as a single executable standard. Do we want to have an equivalent reference implementation? What should it be and/or what criteria do we want in a reference implementation?

2. To get to stage 4 wasm requires 2 or more "web VMs" to implement the feature. The requirement for multiple implementations helps ensure that a spec is implementable in different codebases with reasonable effort and avoids baking quirks of a particular VM into the standard (which is a good property that we want). Requiring that they be web VMs doesn't make sense for WASI though. So what should the requirement be? What kind of goal or property do we want in an implementation that we consider sufficient to allow a feature to be standardized?

3. At what point and how do we involve the broader wasm CG?

hi

I'm just had read lin's article. In the scope have input devices and host's 'other resources'. And in the future evolution have mentions:

One possibility is that this API could evolve into something like Fuchsia's low-level APIs, which are more complex and abstract, though also more capable.

I would like some clarification of yours goals on mobile app development. Is a wasi ambitions can put webassembly apps into google and apple store?
Have any web api outside the wasi's use case?
Phonepag nor nativescript supports webassembly, but will be an natural fit to use wasi for all OS comunication in new framworks?

thanks!

Hi. I would like to propose that all WASI functions require UTF-8 as text encoding and C and C++ bindings be defined using at least char8_t.

char and wchar_t are horrible mistakes that plague not just C but all other languages that take ideas from it. Let's not make this mistake again.

After looking at the spec, I propose this:
__wasi_string_t (struct)
Members:

• char8_t *buf and size_t buf_len
  The address of the first code unit and amount of code units.

Since char8_t has only been accepted to C++20 and not C (but it is expected in C2X), we can create a temporary workaround for C compilers:
__wasi_char8_t (uint8_t) in C (pre C2X) and __wasi_char8_t (char8_t) in C++.

Then change the following functions to use __wasi_string_t:

• __wasi_args_get
• __wasi_environ_get
• __wasi_fd_prestat_dir_name
• __wasi_path_create_directory
• __wasi_path_filestat_get
• __wasi_path_filestat_set_times
• __wasi_path_link
• __wasi_path_open
• __wasi_path_readlink
• __wasi_path_remove_directory
• __wasi_path_rename
• __wasi_path_symlink
• __wasi_path_unlink_file

This is perhaps dependent on resolving #1 and #4, but a large gap in the wasi's capabilities is that there is currently no way to create sockets (i.e. no analogue to accept or connect). There are currently ways to read and write to sockets (sock_send and sock_recv), but without a good way to get a socket fd, these are not terribly useful.

Given that a major early use case for wasi is in the FaaS and "edge computing" space where networking is pretty important, it makes sense for filling out the socket API to be high on the priority list.

It seems like a pretty fundamental function, which is currently missing from the Core. It could be as simple as sleep(uint64_t nanos).

Soon will come the time to decide how to pain something on a screen, are there discussions about this topic in the community? didn't see any. Seeing what WebGPU is supposed to do seems like a natural fit for WASI to use this API instead of defining its own or doing something WASIVulkan/WASIMetal ... I imagine on top of, next to it, or even bellow, other APIs could be defined right? like stuff to draw 2d graphics web 2d canvas style, or create windows, or not sure if it would need to go lower level like defining a compositor, a framebuffer, etc.
I imagine Rust WebGPU implementation should make it easy for wasi runtimes like wasmtime or wasmer to come up with a working prototype since they are rust based, there are idiomatic rust wrappers, works natively on windows, mac and linux and there are a WebIDL definitions.

From bytecodealliance/wasmtime#68:

__wasi_proc_raise in its current form is effectively just a variant of __wasi_proc_exit which produces an exit status indicating that the process was killed with a signal rather than exiting normally.

I agree that this is obscure, and it could create complications if real signal handling is added later, so we should consider removing it.

I noticed that WASI needed a logo, so I'm proposing this one:

I'd love to hear what people think and am happy to make alterations!
A zip file containing this image in SVG format is attached.

Slice.svg.zip