Repro of running touch a is reliable for me, it bricks the shell

Hi, I noticed a bug when trying the webshell feature

When pressing [tab], a autocomplete will happen, but the characters won't display:

After pressing [tab], the black space is presented, pressing enter will yield an error unknown name.

I am on version 0.13.0

Trying to implement shell integration to show exit status on the left, but some commands always exit with 1

See

Hi, I'd like to try Python execution on the web shell described in https://code.visualstudio.com/blogs/2023/06/05/vscode-wasm-wasi. I found the web shell extension on the market place, but couldn't find the Python testbed extension. Is it not publicly available yet?

The README does not describe the project, just an initial template. Would be nice to see a proper README.

Currently we only have support for Python so far. We should investigate to execute other languages like C# or a compiled language like C++.

There is now an early specification for threads in WASI and the WASI-SDK as a first implementation of this. We should explore supporting them in the VSCode WASM/WASI host since threads will enable a lot of additional use cases.

Explore to support a web shell that allows to execute wasm binaries for ls, .... and support mounting commands (like python, git, ..) into the web shell.

Are there any plans to consider supporting the WASIX API in vscode-wasm-wasi?

https://wasix.org/

This would offer further features for WASMs to leverage, but may be expanding the scope of the project too far?

I believe a lot could be simplified by leveraging the WASI preview2 + WIT work that is happening here. From the JavaScript binding generation perspective, you might have a look in this repo too: https://github.com/bytecodealliance/jco.

We need to come up with a testing strategy for the WASI implementation. Ideas:

• implement tests in TS and call the implementation methods directly
• implement tests in Rust/C++ and compile them to WASM-WASI and execute them through the web assembly execution.

Hi! I read here that there were plans to support hosting WASI LSP servers in web extensions. I did not find any example of this in the repo - is it still something that is being worked on?

We need to ensure that:

• common tests run under node and browser
• api tests run under desktop and web

If I create /pipe/input on the memory file system, calling int fd = open("/pipe/input", O_RDONLY); will succeed, but calling fstat(fd, &stat) will fail with EPERM

			const memFS = await wasm.createMemoryFileSystem();
			const rootFileSystem = await wasm.createRootFileSystem([
				{ kind: 'workspaceFolder' },
				{
					kind: 'memoryFileSystem',
					mountPoint: '/pipe',
					fileSystem: memFS
				},
			]);
			const pipeIn = memFS.createWritable('./input', 'utf-8');
			const seedBuf = new Uint8Array(1);
			seedBuf[0] = 42;
			pipeIn.write(seedBuf);

Repro: clone the "memfs" branch of https://github.com/lambdageek/big-buffer-write/tree/memfs build and run the extension and run it with "wasi: write big buffer". pick a small value from the Quick Pick, like 20.

Note the output in the output window:

2023-08-02 14:17:10.506 [error] stderr: the mode for /pipe is 040000

2023-08-02 14:17:10.506 [error] stderr: fstat input
2023-08-02 14:17:10.506 [error] stderr: :
2023-08-02 14:17:10.507 [error] stderr:  
2023-08-02 14:17:10.507 [error] stderr: Operation not permitted
2023-08-02 14:17:10.507 [error] stderr: 

2023-08-02 14:17:10.507 [error] stderr: that's EPERM

2023-08-02 14:17:10.509 [info] process terminated with exit code 1

Also, FileOnlyBaseRights looks like a typo:

This should probably be const FileOnlyBaseRights: rights = FileBaseRights & ~DirectoryBaseRights;

Why I care: .NET does an FStatCheckIO after opening a file that ends up failing
https://github.com/dotnet/runtime/blob/e2a2bfff005ea0620cf1d2b04a01dc90bbfe8ead/src/libraries/System.Private.CoreLib/src/Microsoft/Win32/SafeHandles/SafeFileHandle.Unix.cs#L334

So FileStream f = File.OpenRead("/pipe/input") always fails

As a follow up to #119, I discovered that the Readable stream's data is not always a Uint8Array, but is sometimes a SharedArrayBuffer. I believe this occurs when running an extension in the browser.

This is a problem because you can't TextDecoder#decode a SharedArrayBuffer, so my shims needed to copy the contents into a new Uint8Array before passing them onward.

The component model will add resource support. We need to understand this new feature and evaluate how it affects the VS Code WASI host.

Not sure if this is the right place, but I just want to mention a new VSCode extension I started last week which uses your extension to run the (quite ancient) asmx 8-bit assembler (http://xi6.com/projects/asmx/) as WASI blob as part of a "home computer IDE".

The extension is still heavily WIP:

https://marketplace.visualstudio.com/items?itemName=floooh.vscode-kcide

...and looks like this (the emulator is integrated as regular Emscripten WASM running in a webpanel):

...the WASI extension currently needs to be manually installed (automatic installation as dependency fails because the WASI extension doesn't yet have a release-version, only pre-release).

I also got the WASI execution working in the VSCode browser version, only the debug adapter is causing me some grief there (unrelated to WASI though).

So: thanks a lot and I'm looking forward to the first stable release!

PS: I was also missing a "cwd" arg for launching the WASI blob, but googling around it looks like WASI currently doesn't have the concept of a current-working-directory, so all good, I found a different solution.

With the component model tooling we now have an easy way to integrate WASM services into VS Code extensions. We should document this via samples and a blog post.

Hi,

First of all thank you for this extension, this is a great job!

I have few questions regarding MemoryFileSystem and Stdio.

First here is my current code i'm trying to get worked.

const pty = wasm.createPseudoterminal();
const terminal = vscode.window.createTerminal({ name: 'PIMO output', pty, isTransient: true });
terminal.show(true);
try {
            const wasmPath = path.resolve(__dirname, "wasm/pimo-wasi.wasm");
            const bits = await vscode.workspace.fs.readFile(vscode.Uri.file(wasmPath));
            const module = await WebAssembly.compile(bits);

            const fs = await wasm.createMemoryFileSystem();
            fs.createFile("masking.yml", new TextEncoder().encode(masking));

            const output = wasm.createReadable();
            const input = wasm.createWritable();
            await input.write(data);
            await input.write(Uint8Array.from([0x0A,0x04])); // end of input (EOF)

            const mountPoints: MountPointDescriptor[] = [
                { kind: 'workspaceFolder'},
                { kind: 'memoryFileSystem', fileSystem: fs, mountPoint: '/memory' }
            ];
            
            const rootfs = await wasm.createRootFileSystem(mountPoints);

            const stdio: Stdio = {
                in: {kind: "pipeIn", pipe: input},
                out: {kind: "pipeOut", pipe: output},
                err: {kind: "terminal", terminal: pty}
            };
            
            // Create a WASM process.
           const process = await wasm.createProcess('pimo', module, { stdio: stdio, args: ["-c", "/memory/masking.yml", "-v", "5"], rootFileSystem: rootfs, trace: true });
            process.stdout?.onData(data => {
                console.log(new TextDecoder().decode(data));
            });

            // Run the process and wait for its result.
            const result = await process.run();
            console.log("success " + result);
        } catch (error) {
            console.error(error);
        }

My questions are :

1 - the process reads an input of type pipeIn created with a wasm.createWritable() call, but it does not seem to catch the EOF and hang indefinitly waiting for more bytes. How can I indicates to the process that there nothing more and it should end ?
2 - at some point, the process reads a file /memory/masking.yml that is created by a call to fs.createFile("masking.yml", ...), the file exist I can get a stat (it is a regular file) but the process get an error Errno.perm ("Operation is not permitted") when trying to read it (call to path_open). Is it something I'm doing wrong ? or maybe the wasm file is not correct (i use the go compiler 1.21)

Thank you in advance ! Any help appreciated

I'm trying to run WASM in my extension, but the WebAssembly global is undefined, so I can't compile the module.

Version: 1.86.2
Commit: 903b1e9d8990623e3d7da1df3d33db3e42d80eda
Date: 2024-02-13T19:41:37.860Z
Electron: 27.2.3
ElectronBuildId: 26908389
Chromium: 118.0.5993.159
Node.js: 18.17.1
V8: 11.8.172.18-electron.0
OS: Linux x64 6.1.0-18-amd64

After seeing @dbaeumer's talk at WasmCon, I was inspired to compile the Noir LSP to WASI and hook it as the server component via the vscode-wasm harness. The current progress is available at noir-lang/vscode-noir#41

One of the workarounds I needed to do was stubbing the following interfaces:

interface _ReadableStream {
	onData(listener: (data: Uint8Array) => void): Disposable;
	onClose(listener: () => void): Disposable;
	onError(listener: (error: any) => void): Disposable;
	onEnd(listener: () => void): Disposable;
}

interface _WritableStream {
	onClose(listener: () => void): Disposable;
	onError(listener: (error: any) => void): Disposable;
	onEnd(listener: () => void): Disposable;
	write(data: Uint8Array): Promise<void>;
	write(data: string, encoding: _MessageBufferEncoding): Promise<void>;
	end(): void;
}

This allows me to use the Readable and Writable provided by this library as streams for ReadableStreamMessageReader and WriteableStreamMessageWriter. If this library implemented those interfaces, these streams could be used directly without any stubs.

I am trying to wire this up, but instead of just calling a main method once, I would like to wire this up using an LSP server (based on tower_lsp).

My biggest problem seems to be, how to wire up the I/O of the LSP with the client side in the addon. Is there any information on how to approach this? I wasn't able to find any, but maybe I just didn't spot it.

Hi There!

For new developers and early adopters we need a step by step guide for the RUST part.
This blog shows the python part only so we are not able to figure out how to use this for rust.

So please update docs either in vs code or provide a new blog for RUST implementation step by step guide.
Thank You

Wit and the component model has now support for resources. We need to support them in the VS Code generated interfaces as well since they are heavily used.

release version of 'ms-vscode.wasm-wasi-core' extension because it has no release version.

Summary:

The extension is installed as disabled

Expected results:

The extension is installed, activated and enabled

Symptoms

This extension is disabled in this workspace because it is defined to run in the Remote Extension Host. Please install the extension in 'Dev Container: sample-dotnet-wasi-vscode' to enable it.

Workaround

Manually install/enable the extension inside the devcontainer.
This is a tedious manual step and a bad user experience in general.

Steps to reproduce:

use devcontainer.json with the following content

{
	"name": "sample-dotnet-wasi-vscode",
	"image": "mcr.microsoft.com/devcontainers/base:bullseye",
	"features": {
		"ghcr.io/devcontainers/features/dotnet:1": {
			"version" : "8.0.100-preview.6.23330.14",
			"installUsingApt": false
		}
    	        ,"ghcr.io/devcontainers/features/node:1": {}
		,"ghcr.io/wxw-matt/devcontainer-features/command_runner:latest": {
			"command1": "wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk_20.0_amd64.deb",
			"command2": "sudo apt install ./wasi-sdk_20.0_amd64.deb",
			"command3": "rm ./wasi-sdk_20.0_amd64.deb"
                }
	}
	,"containerEnv": { "WASI_SDK_PATH": "/opt/wasi-sdk" }
	, "customizations": {
		"vscode": {
			"extensions": [
				"ms-vscode.wasm-wasi-core"
			]
		}
	}
	,"postCreateCommand": "dotnet workload install wasi-experimental"
	,"remoteUser": "vscode"
}

VSCODE version

➜ code --version
1.80.1
74f6148eb9ea00507ec113ec51c489d6ffb4b771
x64 

Shells don't normally do this:

See Stream.BufferSize and the write method here

What I'm trying to do:

I have an extension that communicates to a WASI module by sending JSON payloads over stdin/stdout. I configured the process with:

			const options: ProcessOptions = {
				stdio: {
					in: { 'kind': 'pipeIn' },
					out: { 'kind': 'pipeOut' },
					err: { 'kind': 'pipeOut' },
				},
				rootFileSystem,
			};

and I have callbacks set up for out and err to receive responses back from the WASI process.

Expected behavior:

The WASI process can call ssize_t written = write(STDOUT_FILENO, ptr, to_write) where to_write is any reasonable value. If necessary the host will buffer the write in whatever way is reasonable and written will be less than or equal to to_write.

Actual behavior:

For sizes of to_write <= 16384 the write succeeds. For sizes of to_write > 16384 the write hangs and the write syscall in C never returns

Same behavior in VS Code desktop and web shell.

Reproduction

Please clone this repo https://github.com/lambdageek/big-buffer-write/tree/main and run the wasi: write a big buffer command in VS and choose a value bigger than 16384 from the Quick Pick.

The extension will emit some notes to the big-buffer-write log output channel as it runs

Currently we can only plug-in filesystems on the VS Code side. To get pdb debugging work it would be helpful to support having a pluggable device driver on the WSAM-WASI side as well. This would allow to have special character devices to communicate with pdb.

When working with python running in WASM using the WASM API,
Listing contents on a mounted directory has an issue.

when os.listdir('/usr/folder') is called it works fine,
but doesn't work when I do so with os.listdir('/usr/folder/').

I just see an empty array in the latter case.

sample code to run to see output:

import os
print(os.listdir('/usr/lib/')

Suppose you have something like so:

Main Thread - Extension code
Worker 1 - Extension sub piece, which starts Worker 2
Worker 2 - Needs to use the sync api

Worker 2 cannot communicate with the Main Thread because its MessageChannel is only with Worker 1.

For CPython I have this exact situation when trying to run the debugger:

• Main thread - Extension code
• Worker 1 - Load of python.js
• Worker 2 - Reader thread for debugger
• Worker 3 - Writer thread for debugger

Worker 2 and 3 cannot send messages to the Main Thread.

• Note: Emscripten (by default proxies) all FS (or socket) API calls to its main thread (worker 1 in this case) but that causes deadlock issues because the Atomics.wait in the sync-api block the main worker thread, therefore no other fs API calls can be handled while waiting. Took me a while to figure this out as most of it was working until Worker 3 tried to acquire a lock.

First of all... great work on the project.

It came for my realization that you are using the same name vscode-wasm as another extension that is widely popular and used in VSCode: vscode-wasm, which has been around already for a few years.

https://marketplace.visualstudio.com/items?itemName=dtsvet.vscode-wasm
https://github.com/wasmerio/vscode-wasm

Having the same name might be confusing for users, so I want to suggest renaming this repo to something else (vscode-wasm-extensions could be a good name, but really anything other than vscode-wasm could work) so we minimize confusion between repos.

Note: this issue came after some discussion in HN. I think is fair to assume good intent, and give the opportunity for maintainers to work on a fix: https://news.ycombinator.com/item?id=35768888

Just a quick question, and since I haven't found a better place to ask (is there one?): are there plans already when the extension could move out of pre-release?

I have a VSCode extension here (https://marketplace.visualstudio.com/items?itemName=floooh.vscode-kcide) which is now pretty much ready to use. The extension runs a cmdline assembler tool via the WASI extension and it's working fine. The only downside is that users must install the extension as pre-release, otherwise installing the ms-vscode.wasm-wasi-core extension won't work.

Even though I mention that caveat promimently in the documentation, users are still likely to stumble over that problem since it differs from how VSCode extensions are usually installed.

IMHO, releasing a non-prerelease version of the WASI extension wouldn't be much of a problem if the major version number stays at 0, and the extension is marked as "Preview" in the marketplace?