finos / perspective Goto Github PK

"{"x":1,"y":2}" is not loadable while "[{"x":1,"y":2}]" is.

Particularly annoying at the end of a ~10min Travis build, the tooltip tests sometimes screencap before the tooltip has asynchronously loaded.

• Column drop target flickers when a column is dragged out of the computed column selector
• Selected columns are cleared when the computation type is changed
• Duplicate named columns silently fail
• Error messages are nice - but IMO we should replace these with just disabling the Save button if the inputs are invalid
• Dark theme has some missing colors

I tried to install perspective-jupyterlab

jupyter labextension install @jpmorganchase/perspective-jupyterlab

and got the following error:

[1/4] Resolving packages...
warning @jpmorganchase/perspective-jupyterlab > @jpmorganchase/perspective-viewer-highcharts > [email protected]: this package has been deprecated, just import modules from the highcharts folder
error Couldn't find the binary git
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[I 15:28:04.338 NotebookApp] > node C:\Users\FZBNBL\AppData\Local\Continuum\Anaconda3\lib\site-packages\jupyterlab\staging\yarn.js run build:prod
yarn run v1.3.2
$ webpack --config webpack.prod.config.js
module.js:540
    throw err;
    ^

Error: Cannot find module '@jpmorganchase\perspective-jupyterlab\package.json'
    at Function.Module._resolveFilename (module.js:538:15)
    at Function.resolve (internal/module.js:18:19)
    at C:\Users\FZBNBL\AppData\Local\Continuum\Anaconda3\share\jupyter\lab\staging\node_modules\@jupyterlab\buildutils\lib\build.js:22:43
    at Array.forEach (<anonymous>)
    at Object.ensureAssets (C:\Users\FZBNBL\AppData\Local\Continuum\Anaconda3\share\jupyter\lab\staging\node_modules\@jupyterlab\buildutils\lib\build.js:21:22)
    at Object.<anonymous> (C:\Users\FZBNBL\AppData\Local\Continuum\Anaconda3\share\jupyter\lab\staging\webpack.config.js:19:7)
    at Module._compile (module.js:643:30)
    at Object.Module._extensions..js (module.js:654:10)
    at Module.load (module.js:556:32)
    at tryModuleLoad (module.js:499:12)
    at Function.Module._load (module.js:491:3)
    at Module.require (module.js:587:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (C:\Users\FZBNBL\AppData\Local\Continuum\Anaconda3\share\jupyter\lab\staging\webpack.prod.config.js:5:14)
    at Module._compile (module.js:643:30)
    at Object.Module._extensions..js (module.js:654:10)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[W 15:32:22.989 NotebookApp] Build recommended
[W 15:32:22.989 NotebookApp] @jpmorganchase/perspective-jupyterlab needs to be included in build

Appreciate any words of wisdom anyone can spare.

for anyone who'd like to play with perspective.js live:

https://beta.observablehq.com/@randomfractals/perspective

🤗

npm ERR! path /src/packages/perspective/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/src/packages/perspective/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-06-26T20_21_36_854Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 254
npm ERR! @ _puppeteer: `docker run -it --rm --shm-size=2g -u root -e WRITE_TESTS=${WRITE_TESTS} -v $(pwd):/src -w /src/packages/${PACKAGE} perspective/puppeteer "npm" "run" "test"`
npm ERR! Exit status 254

versions:

> docker --version
Docker version 18.05.0-ce, build f150324

> zsh --version
zsh 5.3 (x86_64-apple-darwin17.0)

Maybe hbar/vbar?

Hi,

I received this error when running jupyter labextension install @jpmorganchase/perspective-jupyterlab on a valid jupyterlab installation.

Is that an issue you are aware of?

Thanks

If I have a column consisting of values [1.000, 2.5, 3.562, ...] then the column is interpreted as integer instead of float. A (bad) workaround is to increment the first value in the column by a negligible amount (e.g. 0.00000001).

I have a single dataset and I want to have multiple perspective-viewers showing different views of the data (e.g. one showing a chart and the other a grid). Right now, I'm loading the data into all the viewers individually. Is there a better way to do this? Something along the lines of binding a table to multiple perspective-viewers and simply loading data into that single table.

example: psp_viewer.load( [{"FirstName":"Martina", "LastName":"Šturdíková"}] )

I cannot get the Node examples to work, because I get the following error:

node_modules/@jpmorganchase/perspective/build/perspective.node.js:1850
module.exports = babel-runtime/helpers/typeof;
                                             ^

SyntaxError: Unexpected token ;
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:599:28)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)

The compiled output looks broken, I assume the following exports should be strings?

Any ideas?

In my streaming data feed I encode the 'insert/update' and 'delete' of the key rows separately.

I can use perspective to 'insert' (by calling .update) but I can't delete a row by (p)key.

Can this functionality be added?

When trying to use the node build of perspective the javascript errors out on initialization:

const perspective = = require("@jpmorganchase/perspective").default
Object.keys(perspective)
/* 
[ 'worker',
  'TYPE_AGGREGATES',
  'TYPE_FILTERS',
  'AGGREGATE_DEFAULTS',
  'FILTER_DEFAULTS',
  'SORT_ORDERS' ]
*/
const worker = perspective.worker()
/*
ReferenceError: window is not defined
    at Object.worker (/Users/f608687/dev/scratch/node_modules/@jpmorganchase/perspective/build/perspective.umd.js:1:8192)
*/

Would it be best to remove the webpack build for node and maybe spend time later on writing a proper node-gyp binding the perspective C++ runtime ala #212 ?

Sorting on a view with column pivots applied currently results int eh data set having a 'sort' order applied to each relevent column in the resulting data set left-to-right. This allows the reuse of the un-sorted output, but causes ordering to be dependent on this column order, which is typically unexpected and should not be default.

Would you be interested in a viewer for Chart.js ?
I'd be happy to give it a try.

As a users I want to be notified whenever the underlying view of the perspective viewer is updated so that I can I can perform additional actions when the displayed data changes. The callback in _view.on_update does not seem to expose such functionality and is limited to the scope of the viewer.

Hey guys, we're getting ready to release Arrow JS 0.3.0 soon and want to give you a heads up on breaking changes. We refactored things a bit to align better with the C++ implementation, and make it easier to implement the Arrow Writer APIs (coming soon).

I'm just looking at perspective.js#L281, but haven't searched the rest of the perspective codebase so there may be more places to update.

• Table now exposes its Schema

• Removed the eagerly-allocated columns list from the Table, in favor of lazily allocating columns via getColumnAt(i). You can map the schema's fields to get the columns:

  table.schema.fields.map((field, idx) => table.getColumnAt(idx))

• Removed the vector.name property, now accessible via the schema (Vectors can be combined in ways that they differ from the original schema, in which any tie to the original field metadata isn't necessarily valid anymore)

• Added DataType classes so now all the vector type information from the schema is available and strongly-typed at runtime. Now the vector.type field will refer to this instance. We also export the DataType classes and enums on the Arrow.type namespace, so you can do enum or instanceof comparisons.

• Added TypeVisitor and VectorVisitor classes, to make it easier to walk the schema and vector trees:

  import { Vector, visitor, type } from 'apache-arrow';
  // Visitor to convert Vector<Date | Timestamp> to Vector<Int> of epoch ms
  class DateTimeVisitor extends visitor.VectorVisitor {
      visitDateVector(vec: Vector<type.Date_>) {
          return vec.asEpochMilliseconds();
      }
      visitTimestampVector(vec: Vector<type.Timestamp>) {
          return vec.asEpochMilliseconds();
      }
      visitNullVector(vec: Vector<type.Null>) { return vec; }
      visitBoolVector(vec: Vector<type.Bool>) { return vec; }
      visitIntVector(vec: Vector<type.Int>) { return vec; }
      visitFloatVector(vec: Vector<type.Float>) { return vec; }
      visitUtf8Vector(vec: Vector<type.Utf8>) { return vec; }
      visitBinaryVector(vec: Vector<type.Binary>) { return vec; }
      visitFixedSizeBinaryVector(vec: Vector<type.FixedSizeBinary>) { return vec; }
      visitTimeVector(vec: Vector<type.Time>) { return vec; }
      visitDecimalVector(vec: Vector<type.Decimal>) { return vec; }
      visitListVector(vec: Vector<type.List>) { return vec; }
      visitStructVector(vec: Vector<type.Struct>) { return vec; }
      visitUnionVector(vec: Vector<type.Uniona{ return vector; }>): vecny;
      visitDictionaryVector(vec: Vector<type.Dictionary>) { return vec; }
      visitIntervalVector(vec: Vector<type.Interval>) { return vec; }
      visitFixedSizeListVector(vec: Vector<type.FixedSizeList>) { return vec; }
      visitMapVector(vec: Vector<type.Map_>) { return vec; }
  }
  const table = Table.from(buf);
  const visitor = new DateTimeVisitor();
  const cols = table.schema.fields.map((field, idx) => {
      return visitor.visit(table.getColumnAt(idx));
  });

That's all I can think of for now. We have a repo where we push new releases for testing and use ahead of the apache release vote, so feel free to link to this in your package.json if you want to test it out: https://github.com/graphistry/arrow

Best,
Paul

…not everybody has the emscripten compiler installed to build it from the source.

using https://jpmorganchase.github.io/perspective/examples/superstore.html in IE 11,

scenario:

1. switch to grid mode
2. unselect country and city
3. move country to columns, after sub-category

results:

1. it's veeery slow - is it because of perspective or because of hypergrid?
2. console shows:
   SCRIPT5022: Assertion failed
   webcomponents.min.js (11,1555)
   View cancelled
   View cancelled
   Setting up initial schema and data load into HyperGrid

that's probably indicating an internal fault?

As a user i'd like to filter out all empty values in a specific column, or I would like to view rows with empty values in a specific column. Currently the _get_view_filters fails to recognize filter values set to empty string.

If we set filters through the filters attribute on perspective viewer to [["column_name", "==", "" ]], the view is updated correctly, but the input is in error state.

Compare the "index" column to the pivot

see this and this

"Basically, if you import a file that is prefixed by JUPYTERLAB_(RAW|URL|FILE)LOADER, it will use that particular webpack loader. So if in your build step, you copy the relevant files to your build directory and prefix them with JUPYTERLAB_FILE_LOADER_, they can be bundled up with webpack using its file loader"

When I built on MacOS (having installed Boost using brew) everything is fine.

When I followed the directions on Ubuntu, the lerna build script fails whenever a C++ module needs Boost. I have Boost installed with apt install libboost-dev (so it's in /usr/include).

$ em++ --version
emcc (Emscripten gcc/clang-like replacement) 1.37.33 ()
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


Hi,

I've deployed a small demo featuring Perspective and a couple of other libraries and several people report that loading csv data did not work for them. Initially I thought it was on my side, but I cannot reproduce and they also pointed out that https://unpkg.com/@jpmorganchase/[email protected]/build/csv.html is also failing. Just wanted to let you know and will report if I find anything Perspective-related.

Here is a bit of discussion if that helps - https://news.ycombinator.com/item?id=17489658

For e.g. the Highcharts plugin, because the plugin renders the entire contents of the view, certain configurations can cause the browser to become unresponsive as the HTML/SVG/Canvas parts are rendered, even though Perspective itself may not similarly struggle. We should have a plugin-defined threshold that tells the viewer to display a user-overridable warning when the view is too large, rather than naively render.

steps to reproduce:

1. open coincap example
2. using settings, remove pivots to view a 0 sided data set
3. add filter: long == bla and hit enter

Result: data is not filtered out, error in console: hypergrid.plugin.js:1 Uncaught (in promise) TypeError: Cannot read property 'length' of undefine

Can be reproduced using highcharts with 0sided context.

jupyterlab/jupyterlab#3691

When aggregates producing float values are applied to integer values, the output numbers are floats while the schema still treats them as integers.

Issue is visible when hovering over distinct float values, as the tooltip uses the schema to correctly parse data. Because the schema treats Score 1 as an integer instead of a float average, the tooltip parser cannot treat the value correctly.

Aggregations that produce float values:

• avg
• mean
• pct sum parent
• pct sum grand total

Schema shows integer when the data is a float.

Hover tooltip shows integer when data is a float.

See #207 - these changes need to ported to JSDoc

The orange line's earliest points are 20:42, while the points to its left are 20:44

There're many scenarios that user wants to drill down to a detailed view for one or more selected pivot cells.

The pivottable.js has provided a preliminary implementation (https://pivottable.js.org/examples/mps_prepop.html) which only supports click event. It would be very useful for perspective to provide more flexible API such as

1. Cell selection change
2. Right click / double click events
3. Context menu if possible

As a user I want to easily find a column of interest in the perspective viewer column chooser in order to move it around or hide it from the view.

Currently the visible columns define the order the columns are displayed in the grid. It there is a large number of columns it is hard to find the column of interest and then move it around or hide it.

something like a "copy to clipboard" from the context menu when clicking a grid.

Is there any ability to configure the chart so the header columns are vertical? I.e. the normal horizontal layout looks like

And i'm looking for something like:a

fin-hypergrid/core#723

Anyway to detect longs as timestamps? Notice the x axis

On mobile (iOS 11.4 + Safari & Chrome), perspective-viewer allows for touch zoom and scroll. These should be disabled.

• On Chrome, pinch-to-zoom is disabled but drag-and-drop causes scroll.
  

• On Safari, pinch-to-zoom and scroll are both enabled.
  

Hi, and thanks for the great library! I am trying to delete a table to free up the memory but what I see printed in the console is the following:

Uncaught TypeError: Cannot convert "[object Object]" to unsigned int
    at Object.toWireType (psp.js:2)
    at Object.unregister_gnode (psp.js:2)
    at _table.delete (perspective.js:678)
    at perspective.js:1092

Digging in the toWireType method, brings me to:

registerType(primitiveType, {
            name: name,
            "fromWireType": fromWireType,
            "toWireType": (function(destructors, value) {
                if (typeof value !== "number" && typeof value !== "boolean") {
                    throw new TypeError('Cannot convert "' + _embind_repr(value) + '" to ' + this.name)
                }

In the CSV there are primitive values such as strings, numbers, booleans and Perspective itself works fine. Any suggestions?

Perspective should support a date type. This should work otherwise identically to current datetime, but should render just the date portion correctly in all views.

Would be nice if it could auto-promote this types in the type inference algorithm as well.

When performing many row removes by primary key, I get a number of different memory issues (fails in a non-deterministic way).

This test case demonstrates the problem:

        it("multiple single element removes", async function () {
            let table = perspective.table(meta, {index: "x"});
            for (let i = 0; i < 100; i++) {
                table.update([{x: i, y: "test", z: false}]);
            }
            for (let i = 1; i < 100; i++) {
                table.remove([i]);
            }
            let view = table.view();
            let result = await view.to_json();
            expect(result).toEqual([{x: 0, y: "test", z: false}]);
            expect(result.length).toEqual(1);
        });

``
This test usually produces WASM memory out of bounds access warnings, cannot enlarge array warnings, or the following:

I have also seen times where the amount of memory allocated in the JS ArrayBuffer used by the compiled C++ code rapidly increases to sizes >1GB.

When trying to debug this, I also built perspective using SAFE_HEAP=1 and ASSERTIONS=1 flags which gave the following additional information:

Any advice would be much appreciated