Is gwasm-dispatcher meant to be covered by the GPLv3 LICENSE file in the root of the repo? If it's GPLv3 and if including gwasm-dispatcher is necessary to make a webassembly binary that runs on Golem, then I believe any webassembly binaries submitted to Golem using it would need to have their whole source be open and be GPLv3, which I'm assuming is not intended. Using a more permissive license like MIT or Apache 2 specifically for the part that people build into their wasm binaries would avoid this.

Once the program is started, a confirmation step should be shown for tasks which are significant (i.e. may have financial consequences for the user and/or affect a larger network).
As part of this change, a -y flag should also be added to the runner to allow for skipping the confirmation altogether.

• Write cargo docs.
• Write small examples & link tutorial
• API as 0.1 in crates.io

• buid task manifest
• dump lobs to files in execution workdir

gwasm-runner falls into an endless loop within split step in both Local and Golem Brass backends. It uses 100% of a single CPU core.

I'm using it under

• macOS 10.13.6 and
• Rust 1.37.0 (eae3437df 2019-08-13)
• and gwasm-runner from master (53f4627)

kinneret:~/git/gwasm-runner master (53f4627)$ ./target/release/gwasm-runner ../mandelbrot/target/wasm32-unknown-emscripten/release/mandelbrot.wasm --backend Brass -- 1000 1000 4
[2019-11-13T16:07:03Z INFO  gwasm_runner::brass_runner] Working directory: /Users/tworec/Library/Caches/g-wasm-runner/brass/84d3e0e9-1d3b-469f-bde9-41d6e1d66f31
[2019-11-13T16:07:03Z INFO  gwasm_runner::brass_runner] Using GolemConfig {
    address: "127.0.0.1:61011",
    bid: 1.0,
    data_dir: "/Users/tworec/Library/Application Support/golem/b0.22req",
    name: "gwasm-task",
    net: TestNet,
    task_timeout: Timeout(
        00:30:00,
    ),
    subtask_timeout: Timeout(
        00:10:00,
    ),
}
[2019-11-13T16:07:03Z INFO  sp_wasm_engine::sandbox::engine] Initializing SpiderMonkey engine
[2019-11-13T16:07:03Z INFO  sp_wasm_engine::sandbox] Setting exec args [ 'split', '/task_dir/', '1000', '1000', '4' ]
^C
kinneret:~/git/gwasm-runner master (53f4627)$ ./target/release/gwasm-runner ../mandelbrot/target/wasm32-unknown-emscripten/release/mandelbrot.wasm -- 1000 1000 1
[2019-11-13T16:09:09Z INFO  sp_wasm_engine::sandbox::engine] Initializing SpiderMonkey engine
work dir: /Users/tworec/Library/Caches/g-wasm-runner/local/40b3de66-f247-44fc-b838-d50af4a7b3c8/split
[2019-11-13T16:09:09Z INFO  sp_wasm_engine::sandbox] Setting exec args [ 'split', '/task_dir/', '1000', '1000', '1' ]
^C

• Parse manifest.
• Build output files.
• Serialize output.

gwasm-runner-0.3.1-win64.zip

I've noticed that if there is a dangling symlink present in the mapped/overlaid directory (as part of the HostFS set up in golemfactory/sp-wasm, gwasm-runner will fail with a rather cryptic error message.

After some prodding, I've managed to narrow the surface down to this bit of code in golemfactory/sp-wasm:

impl INode for Arc<DirFsInode> {
    type Stream = File;

    fn lookup(&self, name: &str) -> io::Result<Option<Self>> {
        let node = match name {
            "." => Some(self.clone()),
            ".." => Some(
                self.parent
                    .as_ref()
                    .map(Clone::clone)
                    .unwrap_or_else(|| self.clone()),
            ),
            _ => {
                let lookup_path = self.lookup_path.join(name);
                if lookup_path.exists() {
                    let m = lookup_path.metadata()?;

                    Some(Arc::new(DirFsInode {
                        lookup_path,
                        m,
                        parent: Some(self.clone()),
                    }))
                } else {
                    None
                }
            }
        };
        Ok(node)
    }
...
}

and in particular

            _ => {
                let lookup_path = self.lookup_path.join(name);
                if lookup_path.exists() {
                    let m = lookup_path.metadata()?;

                    Some(Arc::new(DirFsInode {
                        lookup_path,
                        m,
                        parent: Some(self.clone()),
                    }))
                } else {
                    None
                }
            }

in this bit, if we hit a dangling symlink, according to docs, lookup_path.exists returns false, and hence, this fn will return None which as a result causes errors later on down the line when, I believe, we try to mount the /.

Write wasm cmdline interfece sepc.

• Parses manifest for each subtask
• Parses the output of each subtask

test

gwasm-runner -b Golem target/wasm32-unknown-emscripten/release/mandelbrot.wasm 1000 1000 5

Result

On local

gwasm-runner -vv -b local target/wasm32-unknown-emscripten/release/mandelbrot.wasm 1000 1000 5

Description

Currently, Brass tasks are configured using a single JSON file. This file includes fields both related to the running Golem instance (e.g. address, Ethereum network) as well as specific to the task being created (task and subtask timeouts, task budget).

This setup proves to be impractical for running tasks with varying parameters. A more flexible approach would be to allow for passing task parameters in the call to gwasm_runner. The Golem Unlimited backend already uses an URL to define both node and task configuration.

Technical details

To run a Brass task, the following parameters are used (as defined in the config file along with default values):

{
    "data_dir": "/home/user/.local/share/golem/default",
    "address": "127.0.0.1:61000",
    "budget": 1.0,
    "name": "gwasm-task",
    "net": "testnet",
    "subtask_timeout": "00:10:00",
    "task_timeout": "00:30:00"
}

Having the data dir and network to run on we can obtain the node's address and port by reading its app_cfg.ini file. The task's name can default to the name of the WASM binary. In many cases the default path to the data dir will also be sufficient.

Proposed solution

Instead of a configuration file the Brass backend could use an URL passed directly to gwasm-runner. Example of a possible URL:
golem:///home/user/custom-datadir?net=mainnet&budget=5.0&task_timeout=00:15:00&subtask_timeout=00:05:00

Requirements:

• Sensible defaults for all required values with an ability to overwrite them if necessary (including data dir path).
• Using values from a set of predefined environment variables if they are present.

• Fix js engine: golemfactory/sp-wasm#33
• Create worker thread for each core.

1. The working runner should create lockfile for the current working dir.
2. On new workdir allocation, it should check if there any free working dirs to remove. (Cleanup should happen in background).

The runner-update script referenced here in the gWASM quick start guide incorrectly invokes mkdir -f here which is causing the following error output:

[jacobseth@localhost code]$ curl -sSf https://golemfactory.github.io/gwasm-runner/runner-update.sh | bash
-== GOLEM GWASM runner DEV Update ==-
which: no gwasm-runner in (/home/jacobseth/code/bin ... )
mkdir: invalid option -- 'f'
Try 'mkdir --help' for more information.
installing to /home/jacobseth/.cargo/bin/gwasm-runner
download gwasm-runner-linux-0.3.1.tar.gz   [ done ] 
installing to /home/jacobseth/.cargo/bin/gwasm-runner    [ done ] 

Oddly the file it's attempting to interact with $BASE_PATH isn't referenced beyond this call so I imagine its inclusion is erroneous. This was encountered on fedora 30.

Golem version 0.22 has introduced a new marketplace for WASM tasks. Payments to the providers are now based on their CPU usage during computations, as well as on the overall task's budget.
This means that we need to be able to specify the overall GNT budget for a tasks created by gwasm-runner.