chromium / axiom Goto Github PK

View Code? Open in Web Editor NEW

52.0 16.0 26.0 10.88 MB

Axiom Project

Home Page: https://chromium.github.io/axiom/web_shell/

License: Apache License 2.0

JavaScript 100.00%

axiom's Introduction

TODO

The axiom name is taken by a Yoeman-like workflow tool. We'll need a new name, stat.

Axiom: Universal File System, Streams, and Processes API for JS

Axiom is a cross browser library that provides primitives for File Systems, Streams, and Processes.

If you're not already familiar with Axiom, please read our explainer document for a brief introduction to the project.

Live demo

For a live demo of Axiom in action, check out the web_shell sample app on our github.io page. Read more about it in web_shell/README.md.

Axiom on node.js

Axiom can also be used from node.js. The same code behind our online demo can be used to start a shell in a native terminal. You'll need an xterm compatible terminal emulator for this, even on Windows. See issue #97 for the details.

Building Axiom

If you're already familiar with node.js, npm, and grunt, you can jump right in with...

$ cd path/to/axiom
$ npm install
$ grunt dist

For more detailed information see our build.md document.

The Axiom distribution

The Axiom distribution includes two libraries, axiom-base and axiom-wash.

The axiom-base library contains the file system library and drivers for a few stock file systems, including:

An in-memory file system called "jsfs".
A DOM File System based driver called "domfs". (Supported cross browser using a polyfill.
A Google Drive file system.
A node.js based file system which lets your access your local file system through the Axiom API in a node.js environment.

(The DOM, Google Drive, and node.js file system drivers may move out to separate packages at some point.)

The axiom-wash library contains a command line interface running on top of axiom-base. It includes the wash command shell and a few supporting executables (cd, cp, mv, etc.). If your application doesn't need to provide a command line interface you won't need to include this library.

These libraries are available as raw ES6 modules, individual AMD-compatible files, concatenated AMD-compatible bundle, and individual CommonJS modules. Choose whichever version suits your particular application.

Importing Axiom

NOTE: We've yet to finalize the name for "Axiom". Once that's done, we'll be publishing npm packages for the axiom-base and axiom-wash libraries. Until then, you need to build them yourself by following the instructions in our build.md document.

If your application is browser based you'll probably want to load the AMD bundle in a <script> tag.

If your application has an AMD loader already, you may use its require function to import axiom modules. Module access will look like var FileSystem = require('axiom/fs/base/file_system').FileSystem';.

If you don't have your own AMD loader you have two options. You can replace require with __axiomRequire__ as shown above, or you can export the modules to a global variable. To create a global variable, call __axiomExport__(window); before calling any Axiom code. This will create a window.axiom object containing the Axiom modules. You can access modules with code like var FileSystem = axiom.fs.base.file_system.FileSystem.

If you're using Axiom in a node.js environment, make sure to include the cjs/ directory from the Axiom distribution in your module path, and require modules with code like var FileSystem = require('axiom/fs/base/file_system').FileSystem;.

Axiom API

Documentation is a work-in progress. Stay tuned to our api.md document. Until that's done, start with the web_shell sample or the base file system classes.

axiom's People

Contributors

Stargazers

Watchers

axiom's Issues

add mount.chrome to nassh

https://chromium.googlesource.com/apps/libapps/+/axiom

https://chromium.googlesource.com/apps/libapps/+/axiom/nassh/doc/hack.txt

In a file called hterm_exe_mount_chrome.js

The append open mode doesn't work on JSFS (always truncates)

(I thought I had an issue filed for this, but can't find it).

`cat some_binary_file` prints the contents, but leaves terminal in broken state

E.g. cat some.pdf:

...
Q�9�␊W���°��Ԣ�漣����≠⎺┐��R�│�*���FL��\���·5���R��ݗ�⎻���≥-�£┘���EC
�]A⎺␍��Ȱ�R����␋┬+�␤\����,��M&├�?5�B�3�£��┌��M���ٷ"΃����T�≤���E�
�°�ڸ␌▒├: ±␍⎼␋┴␊:/▒/␉/⎻.⎻␍°: T≤⎻␊E⎼⎼⎺⎼: C▒┼┼⎺├ ⎼␊▒␍ ⎻⎼⎺⎻␊⎼├≤ '⎻␊
␊┐R␊└▒␋┼␋┼±B┤°' ⎺° ┤┼␍␊°␋┼␊␍
±␍⎼␋┴␊:/▒/␉>

The last line above is what the prompt becomes after this. This broken state is unrecoverable. Also, an attempt to type something in the prompt prints wrong ASCII chars (upper half).

At the same time, the console shows warnings:

...
Ignored CC1 code: "�"
hterm.amd.js:10832 Ignored CC1 code: "\u0013"
hterm.amd.js:10826 Unknown ESC code: "\u0004"
hterm.amd.js:10832 Ignored CC1 code: "\u0011"
hterm.amd.js:10832 Ignored CC1 code: "\u0005"

False positive of redirection when '>' is part of string on web_shell.

The >, <, | inside a string are all erroneously parsed as redirection operators, e.g.:

shell> echo '<html>Hello world!</html>' will fail as > is parsed as redirection.

Command line parsing is broken at least for `cd`

E.g.:

jsfs:/> cd exe
{
  "message": "Invalid sigil for \"_\", expected \"@\": \"$\"",
  "stack": "Error\n    at Error.AxiomError (https://localhost:8080/axiom3/tmp/samples
/web_shell/js/axiom_base.amd.concat.js:404:18)\n    at Error.AxiomError.init (https:/
/localhost:8080/axiom3/tmp/samples/web_shell/js/axiom_base.amd.concat.js:457:15)\n   
 at Error.AxiomError.Invalid [as ctor] (https://localhost:8080/axiom3/tmp/samples/web
_shell/js/axiom_base.amd.concat.js:546:52)\n    at new Arguments.Record (https://loca
lhost:8080/axiom3/tmp/samples/web_shell/js/axiom_base.amd.concat.js:887:15)\n    at n
ew Arguments (https://localhost:8080/axiom3/tmp/samples/web_shell/js/axiom_base.amd.c
oncat.js:801:18)\n    at Parse.parseArgs (https://localhost:8080/axiom3/tmp/samples/w
eb_shell/js/wash.amd.concat.js:2448:18)\n    at Wash.parseArgv (https://localhost:808
0/axiom3/tmp/samples/web_shell/js/wash.amd.concat.js:2218:20)\n    at null.<anonymous
> (https://localhost:8080/axiom3/tmp/samples/web_shell/js/wash.amd.concat.js:2174:26)
",
  "errorName": "Invalid",
  "errorValue": {
    "type": "Invalid sigil for \"_\", expected \"@\"",
    "value": "$"
  }
}

Make ExecuteContext.TTYRequest part of the signal stream

TTYRequest should not be an event, it should be part of the signal stream.
This is required to correctly support execute context over remote file systems.

Fix command line tests

As disabled in #214

Make relevant commands piping-ready

Depends on and gives practical meaning to #144.

Admittedly, not many of the existing commands could use piping, but some could (e.g. cat). That said, new commands will get added for which piping will definitely make sense (e.g. sort).

get web_shell working in firefox

See https://github.com/ebidel/idb.filesystem.js/ for a HTML5 file system polyfill. In theory this is all we'll need to get things working in firefox, but there may be more to it.

Ctrl-D doesn't work in Firefox

edit command doesn't work over https

If you load the web_shell sample over https: the edit command doesn't work. It's loading ace.js from http:, which fails due to mixed content.

`cd` prints cryptic message for bad path

html5:/> cd bad_dir
Error: NotFound {type: "path", value: "cd"}

Valid exec paths starting with './' are now reported as not found

... presumably after #179. The reason is that the only set of locations we're now trying to find an executable path at is @PATH. I have a half-finished fix for this.

Support for named streams

yhyyhy

Use npm packages in a sample app.

@umop

Change wash$ prompt to show pwd

Instead of "wash$ ", we should show the current directory. We don't have env variable substitution yet, so wash.js should just re-set this.promptString_ to the value of pwd after each command.

Add leading slash to paths

at the moment the canonical version our paths look like "filesystem:path/to/foo". We should add a slash after the color.

Also,

$ cd html5:/home
$ ls /home

...needs to be fixed. At the moment it fails because it tries to resolve html5:home/home.

Add FileSystem.prototype.copy

-- should take advantage of faster native copying available in the underlying FS, when copying files within the same FS. That will also preserve maximum number of file attributes. In addition, this is the only easy guaranteed way to implement copying of Google Docs formats on GDrive. See mv.js.

Support for muli-line paste

Send escapes characters of the prompt as escaped characters surrounded with \[ and \]
Update readline to skip these characters so that it knows the lenght of the prompt and where to put the cursor.
Fix terminal.js to split input string around "\n" characters
With these changes, the web_shell app should support pasting multiple commands in one paste operations.

Example:
[\033[1;36m][\u@\h \W]$[\033[m]

Implement stdout redirection

echo abc > a.txt should work.

Investigate: CORS-related error while downloading a Google Sheet

Steps:

mound gdrive
cat gdrive:/some_google_sheet

The resulting XHR (which corresponds to 'text/csv' conversion for the sheet) errors out, printing this to the console:

XMLHttpRequest cannot load https://docs.google.com/spreadsheets/export?id=1QDSdc5TByduLTlVc_8-uh2BlJnJIpDRxuChgcbHB1pQ&exportFormat=csv. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://localhost:8080' is therefore not allowed access.

At the same time, Google Docs can be downloaded just fine (using 'text/plain' conversion). One interesting data point is that URLs in entry.exportLinks look very different for Sheets and Docs.

Implement stdout piping

cat a.txt | sort should work.

Create editor sample

@umop

Create a sample script that defines an editor command which opens ace in a new window, and has the ability to write data back to the file system.

HTERM window is wrongly sized + resizing causes exception on Linux

This happens on Linux only for me.

The virtual screen is taller than the physical tab: once the screen gets filled, the text at the bottom together with the command line prompt scrolls out of view, and the only way to get it visible again is to blindly type clear.
Might be related to 1): an attempt to resize the HTERM window results in an exception in the line with var r = ... in the following code:

TerminalView.resizeIframes = function() {
  if (TerminalView.raf_)
    return;

  TerminalView.raf_ = requestAnimationFrame(function() {
    TerminalView.raf_ = null;

    var container = TerminalView.getIframeContainer();
    for (var i = 0; i < container.children.length; i++) {
      var htermElem = container.children[i];
      var r = htermElem.followObject.getBoundingClientRect();
      htermElem.style.top = r.top + 'px';
      htermElem.style.left = r.left + 'px';
      htermElem.style.width = r.width + 'px';
      htermElem.style.height = r.height + 'px';
    }
  });
};

The exception says:

Uncaught TypeError: Cannot read property 'getBoundingClientRect' of undefined

Implement stderr redirection

echo abc > out.txt 2>&1 should work as on *nix.

Compilation error related to `eval`

Error: missing module import from wash/exe/eval.js for path: wash/wash_util
Warning: Error compiling lib/wash/exe/eval.js Use --force to continue.

grunt wash on Windows

When running wash on node.js on Windows, there are a few issues:

An error message is displayed:

readline: Home cursor position unknown, won't redraw.

The prompt is not displayed
A fatal error occurs if hitting the cursor keys (apparently?) randomly a few times:

Welcome to wash version 1.0.0.
Type exit or Ctrl-D to exit.
readline: Home cursor position unknown, won't redraw.
readline: Home cursor position unknown, won't redraw.

Listing of "jsfs:/", 1 entry:
exe/ mimeType: "", mode: "D", mtime: "1969-12-03 16:00:00", size: 0
readline: Home cursor position unknown, won't redraw.
Fatal error: Cannot read property 'column' of null

Reconcile: 2 versions of findExecutable()

We have Wash.findExecutable() and washUtil.findExecutable(): the two look very similar (one was apparently copied from the other at some point). Converge on a single version.

add landing page for samples

We should have a landing page for samples so folks know where to go from chromium.github.io/axiom/.

add --save and --load options to the script command

script --save foo.js should load foo.js and record the url in "html5:/home/script.json"
script --load should load all scripts found in "html5:/home/script.json"

For bonus points, the "--file" argument could be used to override the location of script.json

`FileSystemManager` doesn't handle unmounted mountable FSes well

E.g.:

html5:/> cd gdrive:/
{
  "message": "dir: \"gdrive:/\"",
  "stack": "Error\n    at Error.AxiomError (https://localhost:8080/axiom2/tmp/samples/web_shell/js
/axiom_base.amd.concat.js:404:18)\n    at Error.AxiomError.init (https://localhost:8080/axiom2/tmp
/samples/web_shell/js/axiom_base.amd.concat.js:457:15)\n    at Error.AxiomError.TypeMismatch [as c
tor] (https://localhost:8080/axiom2/tmp/samples/web_shell/js/axiom_base.amd.concat.js:586:9)\n    
at https://localhost:8080/axiom2/tmp/samples/web_shell/js/wash.amd.concat.js:1813:19",
  "errorName": "TypeMismatch",
  "errorValue": {
    "expectedType": "dir",
    "gotValue": "gdrive:/"
  }
}

`cd some_missing_dir` hangs wash

Console:

Issues with wash on node.js

When listing entries of the node filesystem, we should skip entries we can't read instead of exiting wash with a fatal error.
We don't convert fs.Stats into StatResult properly: size and mode are not converted appropriately
We append too many / at the start of paths.
On Windows, ls nodefs:/ list all the file/directories of the root directory of the current drive. We should probably list all drive letters instead.

Anti-aliasing problem in ExecuteContext

Code in question:

export var ExecuteContext = function(fileSystem, stdio, path, args) {
...
  this.stdio = stdio;
  this.stdin = stdio.stdin;
  this.stdout = stdio.stdout;
...
};

Now a client-side usage that breaks this:

cx.setStdio(stdio1);  
cx.stdin = stdin2;
...
cx.stdio.onData.stdin.addListener(...); // attaches to stdio1.stdin
cx.stdin.onEnd.addListener(...); // attaches to stdin2, although should be an alias to stdio1.stdin

Add "install" method to axiom/fs/js/file_system.js

@umop

Add an install(...) method which is essentially:

FileSystem.prototype.install = function(obj) {
  var result = this.rootDirectory.resolve(new Path('exe'));
  // error checking
  result.entry.install(obj);
}

`cp` doesn't support existing dir as target (always wants new filename at the end)

cp expects the second argument to always specify the new filename at the end. Specifying just an existing directory makes it fail in various ways depending on the underlying file system. E.g.:

html5:/> mkdir a
html5:/> cp .wash_history a
Error: TypeMismatch {expectedType: "entry-type", gotValue: "html5:/a"}

cp should do one of the following:

try to stat the target path first, and it's a dir, take the target file name from the source;
let the target file system decide by adding a hintNewName arg to createOpenContext()

The latter makes the signature of createOpenContext() clumsier, but might save several XHRs for remote FSes, for example.

Numbers returned from executed processes are ignored

The block that handles process return values in Wash.prototype.dispatch looks suspicious:

        if (typeof value !== 'undefined' && typeof value !== 'number' &&
            value !== null) {
          stdio.stdout.write(JSON.stringify(value, null, '  ') + '\n');
        }

Specifically, typeof value !== 'number': this seems to silently ignore numbers as return values, which is a valid usage with our execution model, if I'm not mistaken.

JsOpenContext.prototype.write doesn't seek correctly before writing

...which always overwrites the file regardless of the whence and offset arguments.

mount command should use its own path to find sub-commands

See https://github.com/chromium/axiom/blob/master/lib/wash/exe/mount.js#L88

  var fsMountCmd = new Path('jsfs:/exe/mount.' + fsType);

The code currently a hard-coded path. It should use cx.path as the basis for creating the name of the sub-mount commands.

Inconsistent behavior of openContext for Jsfs, nodefs and domfs

There is a difference in behavior between jsfs and (domfs, nodefs) wrt to calling openContextwhen path points to a non existent file:

jsfs returns a rejected promise, where as (domfs, nodefs) return a fullfilled one
(domfs, nodefs) return a rejected promise when calling open

We probably should pick one, and I would vote for the (domfs, nodefs) one as it makes createOpenContext a lighter weight operation.

mv command broken on jsfs

Repro steps:

jsfs:/> echo "hello" >foo.txt
jsfs:/> mv foo.txt foo2.txt
Error: Duplicate {type: "directory-name", value: "foo.txt"}
jsfs:/>

The issue is somewhere inside the implementation of JsFileSystem.alias()

Sort output of `ls` with directories first

This would make reading the listing easier, although it's not the default with the regular ls.

normalize cd

@umop

Type "cd /" then "cd ..". You should get an error.

Also, cd "exe", then "cd ..", then "pwd", it should print "/" rather than "/exe/.."

DataType vs mime type vs encoding

We seem to be keep running into issue with the way we read and write files wrt to the "DataType" usage.

For example, jsfs always returns DataType.Value when reading files, even though they were saving with a different datatype. It begs the question of why we have a optional data type on the FileSystem.readFile function. What is the purpose?

Another issue is the GDrive file system. GDrive stores a mime type for every file, and allows specifying a "conversion" mime type when reading sheets or docs. Should we include this level abstraction into the Axiom API. How?

A proposal would be to generalize DataType to mimeType as follows:

writeFile(path, mimeType, data)

path is the file path (as usual)
mimeType is the mime type of the file
data is either a string (for text mime type) or an ArrayBuffer for all other types.
If mimeType is text, use the charset:encoding of mimeType to figure out the encoding.

readFile(path, mimeType)

path is the file path (as usual)
mimeType is optional and defines the mime type of the stored data. The file system uses mimeType encoding part as a for decoding stored text as a JavaScript string.
The return value is either a string (if mimeType is string) or an ArrayBuffer.

web_shell hangs when typing cd {"_": ""}

*_dist.json version numbers

@umop

Copy version numbers from main package.json into the axiom/wash specific files as a grunt task.

Typo in base/execute_context.js?

See https://github.com/chromium/axiom/blob/master/lib/axiom/fs/base/execute_context.js#L146

this.callee.onStdOut.removeListener(this.onStdOut.fire);
this.callee.onStdOut.removeListener(this.onStdErr.fire);
this.callee.onTTYRequest.removeListener(this.onTTYRequest.fire);

Looks like the 2nd line should say "this.callee.onStdErr..."

`chrome --exec-script`: intercept and communicate script errors to user

Right now, chrome --exec-script "some invalid script" ... returns null, rather than printing an informative message to the user about what what's happened. However, the console in the problematic window shows the error just fine. Try wrapping the user's script into a try-catch shell inside chrome_agent_client.js` before sending it for execution to the tab, and returning the caught exception.

Also, to return a result, the script currently must end in a statement that evaluates to that result, e.g. document.querySelector('...'): using return with it (return document.querySelector('...')) creates an error. Using return is rather intuitive, so we should find a way to allow it, e.g. wrap the user's script into a (function() { <script> })();.

Add a "umount" command

This would be useful for stream file systems

wash history not saved if html5:/home doesn't exist

We try to save the history here, but fail when the parent directory doesn't exist: https://github.com/chromium/axiom/blob/master/lib/wash/exe/wash.js#L553

chromium / axiom Goto Github PK

axiom's Introduction

TODO

Axiom: Universal File System, Streams, and Processes API for JS

Live demo

Axiom on node.js

Building Axiom

The Axiom distribution

Importing Axiom

Axiom API

axiom's People

Contributors

Stargazers

Watchers

Forkers

axiom's Issues

Recommend Projects

Recommend Topics

Recommend Org