vega / datalib Goto Github PK

View Code? Open in Web Editor NEW

732.0 732.0 133.0 3.36 MB

JavaScript data utility library.

Home Page: http://vega.github.io/datalib/

License: BSD 3-Clause "New" or "Revised" License

JavaScript 99.50% Shell 0.50%

datalib's Introduction

Vega: A Visualization Grammar

Vega is a visualization grammar, a declarative format for creating, saving, and sharing interactive visualization designs. With Vega you can describe data visualizations in a JSON format, and generate interactive views using either HTML5 Canvas or SVG.

For documentation, tutorials, and examples, see the Vega website. For a description of changes between Vega 2 and later versions, please refer to the Vega Porting Guide.

Build Instructions

For a basic setup allowing you to build Vega and run examples:

Clone https://github.com/vega/vega.
Run yarn to install dependencies for all packages. If you don't have yarn installed, see https://yarnpkg.com/en/docs/install. We use Yarn workspaces to manage multiple packages within this monorepo.
Once installation is complete, run yarn test to run test cases, or run yarn build to build output files for all packages.
After running either yarn test or yarn build, run yarn serve to launch a local web server — your default browser will open and you can browse to the "test" folder to view test specifications.

This repository includes the Vega website and documentation in the docs folder. To launch the website locally, first run bundle install in the docs folder to install the necessary Jekyll libraries. Afterwards, use yarn docs to build the documentation and launch a local webserver. After launching, you can open http://127.0.0.1:4000/vega/ to see the website.

Internet Explorer Support

For backwards compatibility, Vega includes a babel-ified IE-compatible version of the code in the packages/vega/build-es5 directory. Older browser would also require several polyfill libraries:

<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/7.4.4/polyfill.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/runtime.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/fetch.umd.min.js"></script>

Contributions, Development, and Support

Interested in contributing to Vega? Please see our contribution and development guidelines, subject to our code of conduct.

Looking for support, or interested in sharing examples and tips? Post to the Vega discussion forum or join the Vega slack organization! We also have examples available as Observable notebooks.

If you're curious about system performance, see some in-browser benchmarks. Read about future plans in our roadmap.

datalib's People

Contributors

Stargazers

Watchers

Forkers

marcprux iros he0x cscheid jfroeschke shotishu ramnathv gijs datalytica neusoft-psd kuroky360 jonasagx fortranlee adityavs anselmo sandbox ekovac vizshine freeman-lab mathisonian mcanthony nyurik fastfedora achchg damianofusco brinnpaul vitage agile-innovations sksundaram-learning jeffbaumes fstfwd sureddy sandy4321 lathropd dbirchak fengweijp kevinwkc ironybemyshield yeekzhang yngve-sk crispamares jisraeltoledo heoa rgbkrk curran awesomehumphrey simaq amitkaps fdnklg junijland acdaniel catataw jalamao shashankkarnati salmanbao lgeiger roaldl zvone848 orf53975 visiky vpallavi25 lidad gajmp wegiangb willium abrahamlewis4867 manjhawar96 forkkit viethien acikgozmehmet observedobserver leonchenucr thibaultdhondt arenaswan grian100 fagan2888 renzhongpiao patrickfung arholla simeonbeal nayeer julienmol devopstoday11 rchrd2199 scrawfor lyuji282 maiadaelgawad manthan2501 yrammos pal jhoselin826 admariner minghin-cheung jssellars adeleh erisonliang stevenjhiggs90 peterrcook cuulee drsagare

datalib's Issues

agg._mod doesn't update min for values

var dl = require('./src/index');

var add = [28, 55, 43, 91, 81, 53, 19, 87, 52, 48, 24, 49, 87, 66, 17, 27, 68, 16, 49, 15];

var mod = [{x: 56, prev: 28}, {x: 86, prev: 43}, {x: 162, prev: 81}, {x: 38, prev: 19}, {x: 104, prev: 52}, {x: 48, prev: 24}, {x: 174, prev: 87}, {x: 34, prev: 17}, {x: 136, prev: 68}, {x: 98, prev: 49}, {x: 110, prev: 55}, {x: 182, prev: 91}, {x: 106, prev: 53}, {x: 174, prev: 87}, {x: 96, prev: 48}, {x: 98, prev: 49}, {x: 132, prev: 66}, {x: 54, prev: 27}, {x: 32, prev: 16}, {x: 30, prev: 15}]

var agg = dl.groupby()
  .stream(true)
  .summarize([{
    name: "value",
    get: dl.identity,
    ops: ["min", "max"],
    as:  ["min", "max"]
  }]);

add.forEach(function(t) { agg._add(t) });

console.log(agg.result()); // Correct expected output: [ { min: 15, max: 91 } ]

mod.forEach(function(t) { agg._mod(t.x, t.prev); });

console.log(agg.result());  // Incorrect output: [ { min: 15, max: 182 } ]. Expected min to be 30.

Automatic number format for large ranges uses the same precision prefix

I don;t know what the intended behavior of auto formatting is but I found that datalib formats large ranges with the same precision prefix (e.g. M for million)

> a = [0,1,10,100,1000,10000, 100000, 1000000]; f = dl.format.auto.number(a, 1, 's'); a.map(f)
["0M", "0M", "0M", "0M", "0M", "0M", "0M", "1M"]

I would expect to get something similar to what this code produces (minus the 0m, which should just be 0).

a = [0,1,10,100,1000,10000, 100000, 1000000]; a.map(function(d) { return dl.format.auto.number([d], 1, 's')(d); })
> ["0m", "1", "10", "100", "1k", "10k", "100k", "1M"]

I'm not sure whether I am misunderstanding the intended behavior of the function.

dl.histogram returning repeated bins

Hey all,

This library has been extremely useful, but I'm hitting an issue with the histogram method. A simple reproduction is available here: https://observablehq.com/d/cfefd9c0c388f478

The issue is that the values that are returned seem to repeat when a high number of bins is used. For example, I'd expect to get bins with values of ..., .27, .28, .29, ... but instead get two bins at .28 and no bin at .29.

Its possible that I'm doing something wrong with the options (or just generally unreasonable) in which case please let me know.

Access column in data loaded from csv

Hello,

I'm having difficulties in understanding how to use your library.
I would like to load a csv file and then get the unique value in a column.

but this code:

var data = dl.csv("Data/FabSDO.csv")
dl.$('Distretto')(data)

returns undefined.

Thanks

Selecting all rows of a data-frame where a given column is equal to a given value

I want to select all rows where a column is equal to a given value, eg where sex = 'male', but I have failed to find the datalib function to help me do it.

Fix failing build

The CI build is currently failing:

`s` formatting collapses numbers

$ dl.range(20).map(dl.format.auto.number([0, 20], 20, 's'))
["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "10", "10", "10", "10", "20", "20", "20", "20", "20"]

$ dl.range(20).map(dl.format.auto.number([0, 20], 20, ''))
["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19"]

It looks like we don't get enough significant digits in https://github.com/vega/datalib/blob/master/src/format.js#L99

@types

Is there anyone working on a typescript definition file?

Support for non-US dates.

Hello, is there any way to infer a non US style date?

> dl.type.infer(['25/01/2016'])
"string"
> dl.type.infer(['01/25/2016'])
"date"

Perhaps I didn't check enough but I didn't see anything related to locale or similar inside the documentation.

Thanks,
M.

Format "date:'%H:%M'" results in Error: Format pattern must be quoted: '%H

Dear devs,

Got a small issue with time value parsing in Vega.
This works:

dl.read([{"a": "13h15"}], {parse: {a: "date:'%Hh%M'"}});

Gives:

[{a: new Date(...)}]

But this does not work:

dl.read([{"a": "13:15"}], {parse: {a: "date:'%H:%M'"}});

Gives:

Error: Format pattern must be quoted: '%H

I bet it's got something to do with the colon 😉

Kind regards,
Jerry

EDIT: Corrected API calls as per comment of @domoritz below.

Should stats.profile build histogram table?

Right now it only counts cardinality (# of distinct values).

We need to do some math in templates to support quarters in Vega-Lite

@bobocandys would like to show quarters in Vega-Lite.

To format dates, we use datalib templates. We automatically generate template strings that look like:

Q{{datum.data | time:'%m' | quarter}}{{datum.data | time:'-%B'}}

The problem with datum.data | time:'%m' | quarter is that %m is in the range [1,12] but quarter expects [0,11]. We could either

a) Change the behavior of the quarter template filter
b) Support expressions in templates such as expr:"x/12+3". This would actually allow us to remove the quarter filter because it's a simple expression. However, this means we would have to add vega-expression as a dependency.
c) Add a decrement filter to datalib :-(
d) Add a way to call the getMonth method on the date (which returns a number for the month in the range of [0,11]).

Max is null after streaming remove

var table = [
      {a: 1, b: 1},
      {a: 2, b: 2},
      {a: 3, b: 2},
      {a: 3, b: 1}
    ];

var gb = dl.groupby()
  .stream(true)
  .summarize({ "a": ["count", "sum", "min", "max"] });

gb.insert(table).result();  // max_a = 3
gb.remove([{a: 3, b: 1}]).result(); // max_a = null

Groupby Nested Array

I`m trying to find a data mining library in JavaScript that would make it easy to manipulate JSON files. I have been recommended on stackoverflow to use this one. I have to say i like the Pandas like syntax to manipulate and summarize datasheets. However there is one use case I fail to solve with this library.

You have the initial JSON given a book you know the name of the author. In the Final JSON given an author you wanna know which book he wrote.

Intuitively I was looking to groupby author. But as a result the datas are grouped by group of authors.

Does this kind of transformation supported by Datalib? Is there a work around?

 [
     {
        "title": "",
        "author": [
          {
            "given": "",
            "family": "",
            "affiliation": []
          },
          {
            "given": "",
            "family": "",
            "affiliation": []
          }
         ]
      },{
        "title": "",
        "author": [
          {
            "given": "",
            "family": "",
            "affiliation": []
          },
          {
            "given": "",
            "family": "",
            "affiliation": []
          }
         ]
      }
    ]

Full example with input and desired output

Add note to docs that Datalib is not maintained

I came here searching for some details on Vega's CSV parsing, and only found out from the below issue that Vega doesn't use that functionality from this library any more:

We don't actively maintain datalib anymore (functionality is now provided by Vega dataflow and transforms). Having said that, we would be delighted to review pull requests.

Originally posted by @domoritz in #100 (comment)

It would be great to add a note to the README to indicate that the library is not maintained, and add a link to the relevant bits of Vega/Vega-Lite code that replace it.

Thanks!

aggregate: geometric mean?

Could you support a geometric-mean? I see "avg", which I assume is an arithmetic mean, and you have "median". I know I'll need geomean in the not too distant future.

vega/vega-lite#575

agg._mod doesn't correctly update min

I'm not sure if this is a bug or a question. Perhaps there's something I'm not calling? But the following test case doesn't currently correctly:

var dl = require('./src/index')
var agg = dl.groupby()
  .stream(true)
  .key("_id")
  .summarize({'value': ['min', 'max'] });

var add = [
  { _id: 1, value: 28 },
  { _id: 2, value: 55 },
  { _id: 3, value: 43 },
  { _id: 4, value: 91 },
  { _id: 5, value: 81 },
  { _id: 6, value: 53 },
  { _id: 7, value: 19 },
  { _id: 8, value: 87 },
  { _id: 9, value: 52 },
  { _id: 10, value: 48 },
  { _id: 11, value: 24 },
  { _id: 12, value: 49 },
  { _id: 13, value: 87 },
  { _id: 14, value: 66 },
  { _id: 15, value: 17 },
  { _id: 16, value: 27 },
  { _id: 17, value: 68 },
  { _id: 18, value: 16 },
  { _id: 19, value: 49 },
  { _id: 20, value: 15 }
];

var mod = [
  { _id: 1, value: 56 },
  { _id: 2, value: 110 },
  { _id: 3, value: 86 },
  { _id: 4, value: 182 },
  { _id: 5, value: 162 },
  { _id: 6, value: 106 },
  { _id: 7, value: 38 },
  { _id: 8, value: 174 },
  { _id: 9, value: 104 },
  { _id: 10, value: 96 },
  { _id: 12, value: 98 },
  { _id: 11, value: 48 },
  { _id: 13, value: 174 },
  { _id: 14, value: 132 },
  { _id: 15, value: 34 },
  { _id: 16, value: 54 },
  { _id: 17, value: 136 },
  { _id: 18, value: 32 },
  { _id: 19, value: 98 },
  { _id: 20, value: 30 },
];

for(var i=0, len=add.length; i<len; ++i) {
  agg._add(add[i]);
}

console.log(agg.result());  // Correct: [ { min_value: 15, max_value: 91 } ]

for(i=0, len=mod.length; i<len; ++i) {
  agg._mod(mod[i], add[i]);
}

console.log(agg.result());  // Incorrect: [ { min_value: 15, max_value: 182 } ] Expected min_value == 30

Feature request: can left- rigth, etc., "joins" be implemented ?

Date parsing differs if we add time

I'd expect the date parser to assume midnight if no time is given.

> dl.date('2012-01-01 00:00:00') == dl.date('2012-01-01')
false

However, it looks like dl is using UTC if no time is given.

dl.date('2012-01-01')
1325376000000
Sun, 01 Jan 2012 00:00:00 GMT

Which is December 31st in my time zone (12/31/2011, 4:00:00 PM GMT-8:00).

And local time when the time is given. This seems right because I didn't parse the date using UTC.

dl.date('2012-01-01 00:00')
1325404800000
Sun, 01 Jan 2012 08:00:00 GMT

I'm happy to change this but want to confirm the intended behavior first.

dl.duplicate does not preserve prototype properties

var a = {a: 1}, b = Object.create(a);
var c = dl.duplicate(b);
// b.a = 1; c = {}

This is due to dl.duplicate using JSON.stringify which does not resolve prototype properties. One possible fix is to walk them manually: http://stackoverflow.com/questions/8779249/how-to-stringify-inherited-objects-to-json

Separate standalone utils methods in util.js from methods that depend on other libraries

Some methods in util.js depends on d3-time and buffer.
Since we are requiring util.jsin Vega-LIte but we do not use d3-time or buffer. This leads to larger compiled file size in Vega-Lite. From a conversation with @jheer, we agree that methods that depend on other libraries should be splitted.

agg._mod with accessors incorrectly calculates min/max

var dl = require('./src/index')
var agg = dl.groupby().stream(true).key("_id").summarize({'vg_dataref': ['min', 'max'] });

var add1 = [
  { x: 1, y: 28, _id: 1 },
  { x: 2, y: 55, _id: 2 },
  { x: 3, y: 43, _id: 3 },
  { x: 4, y: 91, _id: 4 },
  { x: 5, y: 81, _id: 5 },
  { x: 6, y: 53, _id: 6 },
  { x: 7, y: 19, _id: 7 },
  { x: 8, y: 87, _id: 8 },
  { x: 9, y: 52, _id: 9 },
  { x: 10, y: 48, _id: 10 }
];

var add2 = [
  { a: 11, b: 24, _id: 11 },
  { a: 12, b: 49, _id: 12 },
  { a: 13, b: 87, _id: 13 },
  { a: 14, b: 66, _id: 14 },
  { a: 15, b: 17, _id: 15 },
  { a: 16, b: 27, _id: 16 },
  { a: 17, b: 68, _id: 17 },
  { a: 18, b: 16, _id: 18 },
  { a: 19, b: 49, _id: 19 },
  { a: 20, b: 15, _id: 20 }
];

var mod1 = [
  { x: 1, y: 56, _id: 1 },
  { x: 2, y: 110, _id: 2 },
  { x: 3, y: 86, _id: 3 },
  { x: 4, y: 182, _id: 4 },
  { x: 5, y: 162, _id: 5 },
  { x: 6, y: 106, _id: 6 },
  { x: 7, y: 38, _id: 7 },
  { x: 8, y: 174, _id: 8 },
  { x: 9, y: 104, _id: 9 },
  { x: 10, y: 96, _id: 10 }
];

var mod2 = [
  { a: 11, b: 48, _id: 11 },
  { a: 12, b: 98, _id: 12 },
  { a: 13, b: 174, _id: 13 },
  { a: 14, b: 132, _id: 14 },
  { a: 15, b: 34, _id: 15 },
  { a: 16, b: 54, _id: 16 },
  { a: 17, b: 136, _id: 17 },
  { a: 18, b: 32, _id: 18 },
  { a: 19, b: 98, _id: 19 },
  { a: 20, b: 30, _id: 20 }
];

function mod(acc, mod, add) {
  agg.accessors({ 'vg_dataref': acc });

  for(var i=0; i<mod.length; ++i) {
    var t = mod[i],
        a = add[i],
        p;

    if(acc == 'y') {
      p = {x: a.x, y: a.y, _id: a.id};
      a.x = t.x; a.y = t.y;
    } else {
      p = {a: a.a, b: a.b, _id: a.id};
      a.a = t.a; a.b = t.b;
    }

    agg._mod(a, p);
  }
}

This produces the correct result:

agg.accessors({ 'vg_dataref': 'y' });
add1.forEach(function(x) { agg._add(x); });

console.log(agg.result()); // [ { min_vg_dataref: 19, max_vg_dataref: 91 } ]

mod('y', mod1, add1);

console.log(agg.result()); // [ { min_vg_dataref: 38, max_vg_dataref: 182 } ]

As does this:

agg.accessors({ 'vg_dataref': 'b' });
add2.forEach(function(x) { agg._add(x); });

console.log(agg.result()); // [ { min_vg_dataref: 15, max_vg_dataref: 87 } ]

mod('b', mod2, add2)

console.log(agg.result()); // [ { min_vg_dataref: 30, max_vg_dataref: 174 } ]

But this produces an incorrect result:

agg.accessors({ 'vg_dataref': 'y' });
add1.forEach(function(x) { agg._add(x); });

agg.accessors({ 'vg_dataref': 'b' });
add2.forEach(function(x) { agg._add(x); });

console.log(agg.result()); // Correct: [ { min_vg_dataref: 15, max_vg_dataref: 91 } ]

mod('y', mod1, add1);
mod('b', mod2, add2)

console.log(agg.result()); // Incorrect: [ { min_vg_dataref: 30, max_vg_dataref: 174 } ]. Expected max to be 182.

Package as module for easy import in Observable

To import datalib in Observable, requires it to be manually bundled first. This is what is suggested by the Module Require Debugger

require('https://bundle.run/[email protected]')

Can we bundle it in a way that either of the following works?

require('[email protected]/src/index.js')
import('https://unpkg.com/[email protected]?module')

renaming in field definition

Hi, do you plan to support renaming in field definitions? I'd like to do something like:

dl.groupby(['YEAR', {name: 'CTRY_CDE', as: 'COUNTRY_CODE'}, 'ORG']).count().execute(data);

Thank you!

The wrong skew measure?

datalib currently calculates what is supposed to be Pearson's mode skewness, but does so as:
(avg - med) / std

This is non-parametric skewness, rather than:

Pearson's mode skewness which is
(avg - mode) / std

Or Pearson's median skewness, which is
3 * (avg - med) / std

So this is only giving correct values when med == mode.

Should I:

Rename the function to reflect that it's nonparametric skew.
Add a mode function, and then fix the modeskew function by calling mode.
Add a factor of 3 and rename the function to reflect that it's median skew.
Some combination of those things.
None of those things.

feature request: writing data

it's of course possible to write files with fs, but maybe it would be nice to have writing files pare of datalib as well?

agg._mod doesn't account for modified groupby field

var table = [{"country":"US","type":"gold","count":12,"_id":0},{"country":"US","type":"silver","count":13,"_id":1},{"country":"US","type":"bronze","count":15,"_id":2},{"country":"Canada","type":"gold","count":5,"_id":3},{"country":"Canada","type":"silver","count":4,"_id":4},{"country":"Canada","type":"bronze","count":3,"_id":5}];

var agg = dl.groupby("country").key("_id").stream(true).summarize({ count: ['min', 'max'] });
agg.execute(table);

var old = dl.duplicate(table[0]);
table[0].country = "India";
agg._mod(table[0], old);

agg.result();

Tests fail on travis

Problem seems to be an additional columns property.

See https://travis-ci.org/vega/datalib/jobs/93466916

AssertionError: expected [ { a: '1', b: 'aaa', c: 'true', d: '1/1/2001', e: '1.2' },
  { a: '2', b: 'bbb', c: 'false', d: '1/2/2001', e: '3.4' },
  { a: '3', b: 'ccc', c: 'false', d: '1/3/2001', e: '5.6' },
  { a: '4', b: 'ddd', c: 'true', d: '1/4/2001', e: '7.8' },
  columns: [ 'a', 'b', 'c', 'd', 'e' ] ] to deeply equal [ { a: '1', b: 'aaa', c: 'true', d: '1/1/2001', e: '1.2' },
  { a: '2', b: 'bbb', c: 'false', d: '1/2/2001', e: '3.4' },
  { a: '3', b: 'ccc', c: 'false', d: '1/3/2001', e: '5.6' },
  { a: '4', b: 'ddd', c: 'true', d: '1/4/2001', e: '7.8' } ]
      + expected - actual

Datalib is inferring my string as a number

Given the following TSV file, datalib is inferring the name column to be a number.

Example TSV:

owner_slug	slug	aggregate	name	square_footage
company_demo	1bf8caed-89d0-4547-b1f9-feac7d72e91b	TRUE	Restaurant 11057	3000

Datalib call:

datalib.tsv(
  {
    url: 'example.tsv'
  },
  function (error, data) {
    if (error) {
      console.log(error)
    } else {
      console.log(data)
    }
  }
)

Unique measure to groupby aggregation

We have dl.unique. If we were able to add a unique measure to the groupby aggregation, we could get rid of the top-level "Unique" transform in Vega.

dl.summary fails when field names have a period in them.

Shortest working example:

var test = [{"5.a": 0, "a":1},{"5.a": 2, "a":3}];
dl.summary(test);

Support standard error

@ayush29feb You might wanna look at this folder: https://github.com/vega/datalib/tree/master/src/aggregate to add standard error.

No activity in last 3 months?

Hi there,
Vega is something that introduces a very interesting way to define diagrams and graphs, we would like to use it, however I am unsure how many maintainers you currently have and if you are still in active development.
Could you give us a hint on the plans and momentum you see on the project?

Thanks again for the great library

min/max not updating correctly after mod

There's a chance I've been looking at this too long, but the following does not result in correct output for me. I've tried this with a number of permutations of this data (from the index_chart example) and the min/max always only reflect the min/max of the last series (AAPL in this case), which is odd given there's no groupby definition.

var dl = require('./src/index'),
    assert = require('assert');

var add = [{"_id":13203,"symbol":"AMZN","indexed_price":0.4937528921795466},{"_id":13204,"symbol":"AMZN","indexed_price":0.5934752429430821},{"_id":13205,"symbol":"AMZN","indexed_price":0.5502082369273484},{"_id":13206,"symbol":"AMZN","indexed_price":0.2769551133734382},{"_id":13207,"symbol":"AMZN","indexed_price":0.11776955113373447},{"_id":13208,"symbol":"AMZN","indexed_price":-0.15987968533086527},{"_id":13209,"symbol":"AMZN","indexed_price":-0.30310041647385466},{"_id":13210,"symbol":"AMZN","indexed_price":-0.03979639055992593},{"_id":13211,"symbol":"AMZN","indexed_price":-0.11059694585839892},{"_id":13212,"symbol":"AMZN","indexed_price":-0.15270708005552988},{"_id":13213,"symbol":"AMZN","indexed_price":-0.42873669597408604},{"_id":13214,"symbol":"AMZN","indexed_price":-0.6399814900509023},{"_id":13215,"symbol":"AMZN","indexed_price":-0.599490976399815},{"_id":13216,"symbol":"AMZN","indexed_price":-0.7642295233688108},{"_id":13217,"symbol":"AMZN","indexed_price":-0.7633040259139287},{"_id":13218,"symbol":"AMZN","indexed_price":-0.6348912540490513},{"_id":13219,"symbol":"AMZN","indexed_price":-0.6138361869504858},{"_id":13220,"symbol":"AMZN","indexed_price":-0.6726052753354929},{"_id":13221,"symbol":"AMZN","indexed_price":-0.7110134197130957},{"_id":13222,"symbol":"AMZN","indexed_price":-0.7931513188338732},{"_id":13223,"symbol":"AMZN","indexed_price":-0.8618695048588617},{"_id":13224,"symbol":"AMZN","indexed_price":-0.838500694123091},{"_id":13225,"symbol":"AMZN","indexed_price":-0.7380842202683943},{"_id":13226,"symbol":"AMZN","indexed_price":-0.7496529384544193},{"_id":13227,"symbol":"AMZN","indexed_price":-0.6716797778806108},{"_id":13228,"symbol":"AMZN","indexed_price":-0.6737621471540953},{"_id":13229,"symbol":"AMZN","indexed_price":-0.6691346598796853},{"_id":13230,"symbol":"AMZN","indexed_price":-0.6138361869504858},{"_id":13231,"symbol":"AMZN","indexed_price":-0.5782045349375289},{"_id":13232,"symbol":"AMZN","indexed_price":-0.6240166589541879},{"_id":13233,"symbol":"AMZN","indexed_price":-0.6656640444238778},{"_id":13234,"symbol":"AMZN","indexed_price":-0.6543267006015734},{"_id":13235,"symbol":"AMZN","indexed_price":-0.6314206385932438},{"_id":13236,"symbol":"AMZN","indexed_price":-0.5520592318371125},{"_id":13237,"symbol":"AMZN","indexed_price":-0.459740860712633},{"_id":13238,"symbol":"AMZN","indexed_price":-0.5629338269319759},{"_id":13239,"symbol":"AMZN","indexed_price":-0.49444701527070795},{"_id":13240,"symbol":"AMZN","indexed_price":-0.49074502545118},{"_id":13241,"symbol":"AMZN","indexed_price":-0.39773253123553903},{"_id":13242,"symbol":"AMZN","indexed_price":-0.3361869504858861},{"_id":13243,"symbol":"AMZN","indexed_price":-0.1695974086071263},{"_id":13244,"symbol":"AMZN","indexed_price":-0.15964831096714482},{"_id":13245,"symbol":"AMZN","indexed_price":-0.036557149467838924},{"_id":13246,"symbol":"AMZN","indexed_price":0.07172605275335496},{"_id":13247,"symbol":"AMZN","indexed_price":0.1205460434983804},{"_id":13248,"symbol":"AMZN","indexed_price":0.25937066173068024},{"_id":13249,"symbol":"AMZN","indexed_price":0.24872744099953725},{"_id":13250,"symbol":"AMZN","indexed_price":0.21749190189726975},{"_id":13251,"symbol":"AMZN","indexed_price":0.16612679315131884},{"_id":13252,"symbol":"AMZN","indexed_price":-0.004858861638130515},{"_id":13253,"symbol":"AMZN","indexed_price":0.0013882461823230513},{"_id":13254,"symbol":"AMZN","indexed_price":0.00879222582137905},{"_id":13255,"symbol":"AMZN","indexed_price":0.1221656640444239},{"_id":13256,"symbol":"AMZN","indexed_price":0.2586765386395187},{"_id":13257,"symbol":"AMZN","indexed_price":-0.09949097639981484},{"_id":13258,"symbol":"AMZN","indexed_price":-0.11753817677001385},{"_id":13259,"symbol":"AMZN","indexed_price":-0.054604349838037934},{"_id":13260,"symbol":"AMZN","indexed_price":-0.21031929662193422},{"_id":13261,"symbol":"AMZN","indexed_price":-0.0819065247570569},{"_id":13262,"symbol":"AMZN","indexed_price":0.024757056918093483},{"_id":13263,"symbol":"AMZN","indexed_price":0},{"_id":13264,"symbol":"AMZN","indexed_price":-0.1860249884312818},{"_id":13265,"symbol":"AMZN","indexed_price":-0.2070800555298472},{"_id":13266,"symbol":"AMZN","indexed_price":-0.25127255900046275},{"_id":13267,"symbol":"AMZN","indexed_price":-0.17838963442850533},{"_id":13268,"symbol":"AMZN","indexed_price":-0.23438223044886616},{"_id":13269,"symbol":"AMZN","indexed_price":0.04465525219805645},{"_id":13270,"symbol":"AMZN","indexed_price":-0.012031466913465896},{"_id":13271,"symbol":"AMZN","indexed_price":0.04812586765386392},{"_id":13272,"symbol":"AMZN","indexed_price":-0.07774178621008791},{"_id":13273,"symbol":"AMZN","indexed_price":0.12124016658954193},{"_id":13274,"symbol":"AMZN","indexed_price":0.09093012494215641},{"_id":13275,"symbol":"AMZN","indexed_price":0.037019898195279996},{"_id":13276,"symbol":"AMZN","indexed_price":-0.1337343822304489},{"_id":13277,"symbol":"AMZN","indexed_price":-0.15478944932901428},{"_id":13278,"symbol":"AMZN","indexed_price":-0.18533086534012028},{"_id":13279,"symbol":"AMZN","indexed_price":-0.1992133271633503},{"_id":13280,"symbol":"AMZN","indexed_price":-0.10504396112910688},{"_id":13281,"symbol":"AMZN","indexed_price":-0.3778343359555761},{"_id":13282,"symbol":"AMZN","indexed_price":-0.2866728366496992},{"_id":13283,"symbol":"AMZN","indexed_price":-0.2568255437297548},{"_id":13284,"symbol":"AMZN","indexed_price":-0.11869504858861628},{"_id":13285,"symbol":"AMZN","indexed_price":-0.06663581675150383},{"_id":13286,"symbol":"AMZN","indexed_price":-0.08699676075890787},{"_id":13287,"symbol":"AMZN","indexed_price":-0.12841277186487732},{"_id":13288,"symbol":"AMZN","indexed_price":-0.09440074039796387},{"_id":13289,"symbol":"AMZN","indexed_price":-0.07936140675613142},{"_id":13290,"symbol":"AMZN","indexed_price":0.41901897269782507},{"_id":13291,"symbol":"AMZN","indexed_price":0.5997223507635354},{"_id":13292,"symbol":"AMZN","indexed_price":0.5828320222119389},{"_id":13293,"symbol":"AMZN","indexed_price":0.8172142526608054},{"_id":13294,"symbol":"AMZN","indexed_price":0.8489125404905136},{"_id":13295,"symbol":"AMZN","indexed_price":1.1552521980564556},{"_id":13296,"symbol":"AMZN","indexed_price":1.0627024525682556},{"_id":13297,"symbol":"AMZN","indexed_price":1.095326237852846},{"_id":13298,"symbol":"AMZN","indexed_price":1.14345210550671},{"_id":13299,"symbol":"AMZN","indexed_price":0.7977788061082833},{"_id":13300,"symbol":"AMZN","indexed_price":0.491670522906062},{"_id":13301,"symbol":"AMZN","indexed_price":0.6496992133271633},{"_id":13302,"symbol":"AMZN","indexed_price":0.8192966219342896},{"_id":13303,"symbol":"AMZN","indexed_price":0.8884775566867192},{"_id":13304,"symbol":"AMZN","indexed_price":0.6966682091624248},{"_id":13305,"symbol":"AMZN","indexed_price":0.7663118926422954},{"_id":13306,"symbol":"AMZN","indexed_price":0.8697362332253588},{"_id":13307,"symbol":"AMZN","indexed_price":0.6834798704303565},{"_id":13308,"symbol":"AMZN","indexed_price":0.32438685793614075},{"_id":13309,"symbol":"AMZN","indexed_price":-0.012031466913465896},{"_id":13310,"symbol":"AMZN","indexed_price":0.18648773715872288},{"_id":13311,"symbol":"AMZN","indexed_price":0.3609440074039797},{"_id":13312,"symbol":"AMZN","indexed_price":0.4990745025451182},{"_id":13313,"symbol":"AMZN","indexed_price":0.6992133271633503},{"_id":13314,"symbol":"AMZN","indexed_price":0.8630263766774641},{"_id":13315,"symbol":"AMZN","indexed_price":0.8044886626561776},{"_id":13316,"symbol":"AMZN","indexed_price":0.9356779268857011},{"_id":13317,"symbol":"AMZN","indexed_price":0.9842665432670061},{"_id":13318,"symbol":"AMZN","indexed_price":0.8785284590467376},{"_id":13319,"symbol":"AMZN","indexed_price":1.1601110596945858},{"_id":13320,"symbol":"AMZN","indexed_price":1.7489588153632578},{"_id":13321,"symbol":"AMZN","indexed_price":2.1446089773253125},{"_id":13322,"symbol":"AMZN","indexed_price":2.1124479407681633},{"_id":13323,"symbol":"AMZN","indexed_price":1.9016658954187875},{"_id":13324,"symbol":"AMZN","indexed_price":1.7394724664507175},{"_id":13325,"symbol":"AMZN","indexed_price":1.9805645534474778},{"_id":13326,"symbol":"AAPL","indexed_price":-0.32535760728218466},{"_id":13327,"symbol":"AAPL","indexed_price":-0.25461638491547467},{"_id":13328,"symbol":"AAPL","indexed_price":-0.11703511053315994},{"_id":13329,"symbol":"AAPL","indexed_price":-0.19349804941482446},{"_id":13330,"symbol":"AAPL","indexed_price":-0.45383615084525364},{"_id":13331,"symbol":"AAPL","indexed_price":-0.3188556566970091},{"_id":13332,"symbol":"AAPL","indexed_price":-0.33914174252275686},{"_id":13333,"symbol":"AAPL","indexed_price":-0.20754226267880374},{"_id":13334,"symbol":"AAPL","indexed_price":-0.6650195058517555},{"_id":13335,"symbol":"AAPL","indexed_price":-0.7456436931079323},{"_id":13336,"symbol":"AAPL","indexed_price":-0.7854356306892067},{"_id":13337,"symbol":"AAPL","indexed_price":-0.8065019505851755},{"_id":13338,"symbol":"AAPL","indexed_price":-0.718855656697009},{"_id":13339,"symbol":"AAPL","indexed_price":-0.7628088426527959},{"_id":13340,"symbol":"AAPL","indexed_price":-0.7131339401820546},{"_id":13341,"symbol":"AAPL","indexed_price":-0.6686605981794538},{"_id":13342,"symbol":"AAPL","indexed_price":-0.740442132639792},{"_id":13343,"symbol":"AAPL","indexed_price":-0.6977893368010404},{"_id":13344,"symbol":"AAPL","indexed_price":-0.7555266579973993},{"_id":13345,"symbol":"AAPL","indexed_price":-0.7589076723016905},{"_id":13346,"symbol":"AAPL","indexed_price":-0.798179453836151},{"_id":13347,"symbol":"AAPL","indexed_price":-0.7716514954486345},{"_id":13348,"symbol":"AAPL","indexed_price":-0.7230169050715215},{"_id":13349,"symbol":"AAPL","indexed_price":-0.7152145643693109},{"_id":13350,"symbol":"AAPL","indexed_price":-0.6785435630689207},{"_id":13351,"symbol":"AAPL","indexed_price":-0.717815344603381},{"_id":13352,"symbol":"AAPL","indexed_price":-0.6920676202860858},{"_id":13353,"symbol":"AAPL","indexed_price":-0.6842652795838752},{"_id":13354,"symbol":"AAPL","indexed_price":-0.6970091027308193},{"_id":13355,"symbol":"AAPL","indexed_price":-0.7695708712613785},{"_id":13356,"symbol":"AAPL","indexed_price":-0.8015604681404421},{"_id":13357,"symbol":"AAPL","indexed_price":-0.8080624187256177},{"_id":13358,"symbol":"AAPL","indexed_price":-0.811443433029909},{"_id":13359,"symbol":"AAPL","indexed_price":-0.7911573472041612},{"_id":13360,"symbol":"AAPL","indexed_price":-0.7984395318595578},{"_id":13361,"symbol":"AAPL","indexed_price":-0.8137841352405721},{"_id":13362,"symbol":"AAPL","indexed_price":-0.8132639791937581},{"_id":13363,"symbol":"AAPL","indexed_price":-0.8046814044213265},{"_id":13364,"symbol":"AAPL","indexed_price":-0.8161248374512354},{"_id":13365,"symbol":"AAPL","indexed_price":-0.8150845253576073},{"_id":13366,"symbol":"AAPL","indexed_price":-0.7664499349804942},{"_id":13367,"symbol":"AAPL","indexed_price":-0.7521456436931079},{"_id":13368,"symbol":"AAPL","indexed_price":-0.7258777633289988},{"_id":13369,"symbol":"AAPL","indexed_price":-0.7058517555266579},{"_id":13370,"symbol":"AAPL","indexed_price":-0.7305591677503251},{"_id":13371,"symbol":"AAPL","indexed_price":-0.7024707412223667},{"_id":13372,"symbol":"AAPL","indexed_price":-0.728218465539662},{"_id":13373,"symbol":"AAPL","indexed_price":-0.7219765929778934},{"_id":13374,"symbol":"AAPL","indexed_price":-0.706631989596879},{"_id":13375,"symbol":"AAPL","indexed_price":-0.6889466840052015},{"_id":13376,"symbol":"AAPL","indexed_price":-0.6483745123537061},{"_id":13377,"symbol":"AAPL","indexed_price":-0.6647594278283485},{"_id":13378,"symbol":"AAPL","indexed_price":-0.635110533159948},{"_id":13379,"symbol":"AAPL","indexed_price":-0.576853055916775},{"_id":13380,"symbol":"AAPL","indexed_price":-0.5794538361508452},{"_id":13381,"symbol":"AAPL","indexed_price":-0.5513654096228869},{"_id":13382,"symbol":"AAPL","indexed_price":-0.49596879063719124},{"_id":13383,"symbol":"AAPL","indexed_price":-0.31859557867360216},{"_id":13384,"symbol":"AAPL","indexed_price":-0.1279583875162549},{"_id":13385,"symbol":"AAPL","indexed_price":-0.1625487646293888},{"_id":13386,"symbol":"AAPL","indexed_price":0},{"_id":13387,"symbol":"AAPL","indexed_price":0.16671001300390106},{"_id":13388,"symbol":"AAPL","indexed_price":0.08374512353706108},{"_id":13389,"symbol":"AAPL","indexed_price":-0.06215864759427829},{"_id":13390,"symbol":"AAPL","indexed_price":0.03407022106631977},{"_id":13391,"symbol":"AAPL","indexed_price":-0.04265279583875164},{"_id":13392,"symbol":"AAPL","indexed_price":0.10923276983094916},{"_id":13393,"symbol":"AAPL","indexed_price":0.2195058517555266},{"_id":13394,"symbol":"AAPL","indexed_price":0.3942782834850454},{"_id":13395,"symbol":"AAPL","indexed_price":0.4977893368010403},{"_id":13396,"symbol":"AAPL","indexed_price":0.7638491547464236},{"_id":13397,"symbol":"AAPL","indexed_price":0.8697009102730818},{"_id":13398,"symbol":"AAPL","indexed_price":0.963849154746424},{"_id":13399,"symbol":"AAPL","indexed_price":0.7812743823146941},{"_id":13400,"symbol":"AAPL","indexed_price":0.6312093628088425},{"_id":13401,"symbol":"AAPL","indexed_price":0.8306892067620285},{"_id":13402,"symbol":"AAPL","indexed_price":0.5544863459037711},{"_id":13403,"symbol":"AAPL","indexed_price":0.4894668400520156},{"_id":13404,"symbol":"AAPL","indexed_price":0.767490247074122},{"_id":13405,"symbol":"AAPL","indexed_price":0.7646293888166447},{"_id":13406,"symbol":"AAPL","indexed_price":1.0020806241872562},{"_id":13407,"symbol":"AAPL","indexed_price":1.108712613784135},{"_id":13408,"symbol":"AAPL","indexed_price":1.3838751625487644},{"_id":13409,"symbol":"AAPL","indexed_price":1.2065019505851755},{"_id":13410,"symbol":"AAPL","indexed_price":1.2296488946684005},{"_id":13411,"symbol":"AAPL","indexed_price":1.200520156046814},{"_id":13412,"symbol":"AAPL","indexed_price":1.4163849154746422},{"_id":13413,"symbol":"AAPL","indexed_price":1.5955786736020803},{"_id":13414,"symbol":"AAPL","indexed_price":2.1518855656697005},{"_id":13415,"symbol":"AAPL","indexed_price":2.1739921976592975},{"_id":13416,"symbol":"AAPL","indexed_price":2.426788036410923},{"_id":13417,"symbol":"AAPL","indexed_price":2.6015604681404416},{"_id":13418,"symbol":"AAPL","indexed_price":2.991417425227568},{"_id":13419,"symbol":"AAPL","indexed_price":3.940182054616385},{"_id":13420,"symbol":"AAPL","indexed_price":3.739141742522756},{"_id":13421,"symbol":"AAPL","indexed_price":4.151625487646293},{"_id":13422,"symbol":"AAPL","indexed_price":2.5204161248374515},{"_id":13423,"symbol":"AAPL","indexed_price":2.25149544863459},{"_id":13424,"symbol":"AAPL","indexed_price":2.732119635890767},{"_id":13425,"symbol":"AAPL","indexed_price":3.5240572171651494},{"_id":13426,"symbol":"AAPL","indexed_price":3.9089726918075423},{"_id":13427,"symbol":"AAPL","indexed_price":3.3547464239271783},{"_id":13428,"symbol":"AAPL","indexed_price":3.1339401820546158},{"_id":13429,"symbol":"AAPL","indexed_price":3.409102730819245},{"_id":13430,"symbol":"AAPL","indexed_price":1.9560468140442129},{"_id":13431,"symbol":"AAPL","indexed_price":1.7981794538361506},{"_id":13432,"symbol":"AAPL","indexed_price":1.4101430429128736},{"_id":13433,"symbol":"AAPL","indexed_price":1.2197659297789334},{"_id":13434,"symbol":"AAPL","indexed_price":1.3440832249674899},{"_id":13435,"symbol":"AAPL","indexed_price":1.3227568270481143},{"_id":13436,"symbol":"AAPL","indexed_price":1.7339401820546163},{"_id":13437,"symbol":"AAPL","indexed_price":2.272561768530559},{"_id":13438,"symbol":"AAPL","indexed_price":2.532119635890767},{"_id":13439,"symbol":"AAPL","indexed_price":2.7042912873862157},{"_id":13440,"symbol":"AAPL","indexed_price":3.2494148244473333},{"_id":13441,"symbol":"AAPL","indexed_price":3.3747724317295185},{"_id":13442,"symbol":"AAPL","indexed_price":3.820546163849154},{"_id":13443,"symbol":"AAPL","indexed_price":3.9024707412223667},{"_id":13444,"symbol":"AAPL","indexed_price":4.1992197659297785},{"_id":13445,"symbol":"AAPL","indexed_price":4.480624187256176},{"_id":13446,"symbol":"AAPL","indexed_price":3.9950585175552664},{"_id":13447,"symbol":"AAPL","indexed_price":4.3217165149544865},{"_id":13448,"symbol":"AAPL","indexed_price":4.800260078023406}]
var mod = [{"_id":13203,"symbol":"AMZN","indexed_price":0.7138306344571277},{"_id":13204,"symbol":"AMZN","indexed_price":0.8282452880276082},{"_id":13205,"symbol":"AMZN","indexed_price":0.77860366339262},{"_id":13206,"symbol":"AMZN","indexed_price":0.46509158481550295},{"_id":13207,"symbol":"AMZN","indexed_price":0.2824528802760818},{"_id":13208,"symbol":"AMZN","indexed_price":-0.03610299973453675},{"_id":13209,"symbol":"AMZN","indexed_price":-0.2004247411733475},{"_id":13210,"symbol":"AMZN","indexed_price":0.1016724183700557},{"_id":13211,"symbol":"AMZN","indexed_price":0.020440668967347916},{"_id":13212,"symbol":"AMZN","indexed_price":-0.027873639500929234},{"_id":13213,"symbol":"AMZN","indexed_price":-0.34457127687815237},{"_id":13214,"symbol":"AMZN","indexed_price":-0.5869392089195646},{"_id":13215,"symbol":"AMZN","indexed_price":-0.5404831430846828},{"_id":13216,"symbol":"AMZN","indexed_price":-0.7294929652243165},{"_id":13217,"symbol":"AMZN","indexed_price":-0.7284311122909477},{"_id":13218,"symbol":"AMZN","indexed_price":-0.5810990177860367},{"_id":13219,"symbol":"AMZN","indexed_price":-0.556941863551898},{"_id":13220,"symbol":"AMZN","indexed_price":-0.6243695248208124},{"_id":13221,"symbol":"AMZN","indexed_price":-0.6684364215556146},{"_id":13222,"symbol":"AMZN","indexed_price":-0.7626758693920893},{"_id":13223,"symbol":"AMZN","indexed_price":-0.8415184496947173},{"_id":13224,"symbol":"AMZN","indexed_price":-0.8147066631271569},{"_id":13225,"symbol":"AMZN","indexed_price":-0.6994956198566499},{"_id":13226,"symbol":"AMZN","indexed_price":-0.712768781523759},{"_id":13227,"symbol":"AMZN","indexed_price":-0.6233076718874436},{"_id":13228,"symbol":"AMZN","indexed_price":-0.6256968409875232},{"_id":13229,"symbol":"AMZN","indexed_price":-0.6203875763206795},{"_id":13230,"symbol":"AMZN","indexed_price":-0.556941863551898},{"_id":13231,"symbol":"AMZN","indexed_price":-0.5160605256172021},{"_id":13232,"symbol":"AMZN","indexed_price":-0.568622245818954},{"_id":13233,"symbol":"AMZN","indexed_price":-0.6164056278205469},{"_id":13234,"symbol":"AMZN","indexed_price":-0.60339792938678},{"_id":13235,"symbol":"AMZN","indexed_price":-0.5771170692859039},{"_id":13236,"symbol":"AMZN","indexed_price":-0.4860631802495355},{"_id":13237,"symbol":"AMZN","indexed_price":-0.38014335014600475},{"_id":13238,"symbol":"AMZN","indexed_price":-0.498539952216618},{"_id":13239,"symbol":"AMZN","indexed_price":-0.41996283514733207},{"_id":13240,"symbol":"AMZN","indexed_price":-0.41571542341385714},{"_id":13241,"symbol":"AMZN","indexed_price":-0.3089992036103},{"_id":13242,"symbol":"AMZN","indexed_price":-0.23838598354127954},{"_id":13243,"symbol":"AMZN","indexed_price":-0.04725245553490844},{"_id":13244,"symbol":"AMZN","indexed_price":-0.035837536501194624},{"_id":13245,"symbol":"AMZN","indexed_price":0.10538890363684626},{"_id":13246,"symbol":"AMZN","indexed_price":0.22962569684098746},{"_id":13247,"symbol":"AMZN","indexed_price":0.28563843907618786},{"_id":13248,"symbol":"AMZN","indexed_price":0.4449163790814971},{"_id":13249,"symbol":"AMZN","indexed_price":0.43270507034775674},{"_id":13250,"symbol":"AMZN","indexed_price":0.3968675338465621},{"_id":13251,"symbol":"AMZN","indexed_price":0.33793469604459775},{"_id":13252,"symbol":"AMZN","indexed_price":0.14175736660472515},{"_id":13253,"symbol":"AMZN","indexed_price":0.14892487390496414},{"_id":13254,"symbol":"AMZN","indexed_price":0.15741969737191397},{"_id":13255,"symbol":"AMZN","indexed_price":0.28749668170958315},{"_id":13256,"symbol":"AMZN","indexed_price":0.44411998938147057},{"_id":13257,"symbol":"AMZN","indexed_price":0.03318290416777276},{"_id":13258,"symbol":"AMZN","indexed_price":0.012476771967082528},{"_id":13259,"symbol":"AMZN","indexed_price":0.08468277143615603},{"_id":13260,"symbol":"AMZN","indexed_price":-0.09397398460313244},{"_id":13261,"symbol":"AMZN","indexed_price":0.05335810990177855},{"_id":13262,"symbol":"AMZN","indexed_price":0.17573666047252448},{"_id":13263,"symbol":"AMZN","indexed_price":0.147332094504911},{"_id":13264,"symbol":"AMZN","indexed_price":-0.06610034510220339},{"_id":13265,"symbol":"AMZN","indexed_price":-0.09025749933634188},{"_id":13266,"symbol":"AMZN","indexed_price":-0.14096097690469875},{"_id":13267,"symbol":"AMZN","indexed_price":-0.057340058401911434},{"_id":13268,"symbol":"AMZN","indexed_price":-0.12158216087071935},{"_id":13269,"symbol":"AMZN","indexed_price":0.19856649853995212},{"_id":13270,"symbol":"AMZN","indexed_price":0.1335280063711176},{"_id":13271,"symbol":"AMZN","indexed_price":0.20254844704008482},{"_id":13272,"symbol":"AMZN","indexed_price":0.05813644810193782},{"_id":13273,"symbol":"AMZN","indexed_price":0.28643482877621446},{"_id":13274,"symbol":"AMZN","indexed_price":0.25165914520838856},{"_id":13275,"symbol":"AMZN","indexed_price":0.18980621183966015},{"_id":13276,"symbol":"AMZN","indexed_price":-0.006105654366870294},{"_id":13277,"symbol":"AMZN","indexed_price":-0.030262808601008773},{"_id":13278,"symbol":"AMZN","indexed_price":-0.06530395540217682},{"_id":13279,"symbol":"AMZN","indexed_price":-0.08123174940270778},{"_id":13280,"symbol":"AMZN","indexed_price":0.02681178656756034},{"_id":13281,"symbol":"AMZN","indexed_price":-0.2861693655428723},{"_id":13282,"symbol":"AMZN","indexed_price":-0.18157685160605264},{"_id":13283,"symbol":"AMZN","indexed_price":-0.14733209450491117},{"_id":13284,"symbol":"AMZN","indexed_price":0.011149455800371693},{"_id":13285,"symbol":"AMZN","indexed_price":0.07087868330236266},{"_id":13286,"symbol":"AMZN","indexed_price":0.047517918768250574},{"_id":13287,"symbol":"AMZN","indexed_price":0},{"_id":13288,"symbol":"AMZN","indexed_price":0.039023095301300735},{"_id":13289,"symbol":"AMZN","indexed_price":0.056278205468542536},{"_id":13290,"symbol":"AMZN","indexed_price":0.6280860100876028},{"_id":13291,"symbol":"AMZN","indexed_price":0.8354127953278471},{"_id":13292,"symbol":"AMZN","indexed_price":0.8160339792938677},{"_id":13293,"symbol":"AMZN","indexed_price":1.0849482346694983},{"_id":13294,"symbol":"AMZN","indexed_price":1.121316697637377},{"_id":13295,"symbol":"AMZN","indexed_price":1.4727900185824263},{"_id":13296,"symbol":"AMZN","indexed_price":1.3666047252455535},{"_id":13297,"symbol":"AMZN","indexed_price":1.4040350411468012},{"_id":13298,"symbol":"AMZN","indexed_price":1.4592513936819749},{"_id":13299,"symbol":"AMZN","indexed_price":1.062649323068755},{"_id":13300,"symbol":"AMZN","indexed_price":0.7114414653570479},{"_id":13301,"symbol":"AMZN","indexed_price":0.8927528537297583},{"_id":13302,"symbol":"AMZN","indexed_price":1.0873374037695778},{"_id":13303,"symbol":"AMZN","indexed_price":1.1667109105388904},{"_id":13304,"symbol":"AMZN","indexed_price":0.9466418900982213},{"_id":13305,"symbol":"AMZN","indexed_price":1.0265463233342182},{"_id":13306,"symbol":"AMZN","indexed_price":1.1452083886381736},{"_id":13307,"symbol":"AMZN","indexed_price":0.9315104857977171},{"_id":13308,"symbol":"AMZN","indexed_price":0.5195115476506503},{"_id":13309,"symbol":"AMZN","indexed_price":0.1335280063711176},{"_id":13310,"symbol":"AMZN","indexed_price":0.3612954605787098},{"_id":13311,"symbol":"AMZN","indexed_price":0.5614547385187151},{"_id":13312,"symbol":"AMZN","indexed_price":0.719936288823998},{"_id":13313,"symbol":"AMZN","indexed_price":0.9495619856649853},{"_id":13314,"symbol":"AMZN","indexed_price":1.1375099548712502},{"_id":13315,"symbol":"AMZN","indexed_price":1.070347756835678},{"_id":13316,"symbol":"AMZN","indexed_price":1.2208654101406953},{"_id":13317,"symbol":"AMZN","indexed_price":1.2766126891425538},{"_id":13318,"symbol":"AMZN","indexed_price":1.1552959915051764},{"_id":13319,"symbol":"AMZN","indexed_price":1.478364746482612},{"_id":13320,"symbol":"AMZN","indexed_price":2.1539686753384655},{"_id":13321,"symbol":"AMZN","indexed_price":2.6079108043535966},{"_id":13322,"symbol":"AMZN","indexed_price":2.5710114149190337},{"_id":13323,"symbol":"AMZN","indexed_price":2.3291744093443056},{"_id":13324,"symbol":"AMZN","indexed_price":2.1430846827714363},{"_id":13325,"symbol":"AMZN","indexed_price":2.4196973719139896},{"_id":13326,"symbol":"AAPL","indexed_price":-0.6974221392744664},{"_id":13327,"symbol":"AAPL","indexed_price":-0.6656946226525138},{"_id":13328,"symbol":"AAPL","indexed_price":-0.6039892686340838},{"_id":13329,"symbol":"AAPL","indexed_price":-0.6382829814534001},{"_id":13330,"symbol":"AAPL","indexed_price":-0.7550449084334538},{"_id":13331,"symbol":"AAPL","indexed_price":-0.6945060072320075},{"_id":13332,"symbol":"AAPL","indexed_price":-0.7036043392044792},{"_id":13333,"symbol":"AAPL","indexed_price":-0.6445818266651114},{"_id":13334,"symbol":"AAPL","indexed_price":-0.8497608771725185},{"_id":13335,"symbol":"AAPL","indexed_price":-0.8859209144990086},{"_id":13336,"symbol":"AAPL","indexed_price":-0.9037676425988569},{"_id":13337,"symbol":"AAPL","indexed_price":-0.9132159104164237},{"_id":13338,"symbol":"AAPL","indexed_price":-0.8739064504840779},{"_id":13339,"symbol":"AAPL","indexed_price":-0.8936195030910999},{"_id":13340,"symbol":"AAPL","indexed_price":-0.8713402542867141},{"_id":13341,"symbol":"AAPL","indexed_price":-0.8513939111162954},{"_id":13342,"symbol":"AAPL","indexed_price":-0.8835880088650414},{"_id":13343,"symbol":"AAPL","indexed_price":-0.8644581826665111},{"_id":13344,"symbol":"AAPL","indexed_price":-0.8903534352035459},{"_id":13345,"symbol":"AAPL","indexed_price":-0.8918698238656246},{"_id":13346,"symbol":"AAPL","indexed_price":-0.9094832614020762},{"_id":13347,"symbol":"AAPL","indexed_price":-0.897585442668844},{"_id":13348,"symbol":"AAPL","indexed_price":-0.8757727749912515},{"_id":13349,"symbol":"AAPL","indexed_price":-0.8722734165403009},{"_id":13350,"symbol":"AAPL","indexed_price":-0.8558264318208328},{"_id":13351,"symbol":"AAPL","indexed_price":-0.8734398693572846},{"_id":13352,"symbol":"AAPL","indexed_price":-0.8618919864691473},{"_id":13353,"symbol":"AAPL","indexed_price":-0.8583926280181967},{"_id":13354,"symbol":"AAPL","indexed_price":-0.864108246821416},{"_id":13355,"symbol":"AAPL","indexed_price":-0.8966522804152572},{"_id":13356,"symbol":"AAPL","indexed_price":-0.910999650064155},{"_id":13357,"symbol":"AAPL","indexed_price":-0.9139157821066138},{"_id":13358,"symbol":"AAPL","indexed_price":-0.9154321707686924},{"_id":13359,"symbol":"AAPL","indexed_price":-0.9063338387962206},{"_id":13360,"symbol":"AAPL","indexed_price":-0.9095999066837747},{"_id":13361,"symbol":"AAPL","indexed_price":-0.9164819783039776},{"_id":13362,"symbol":"AAPL","indexed_price":-0.916248687740581},{"_id":13363,"symbol":"AAPL","indexed_price":-0.9123993934445351},{"_id":13364,"symbol":"AAPL","indexed_price":-0.9175317858392628},{"_id":13365,"symbol":"AAPL","indexed_price":-0.9170652047124694},{"_id":13366,"symbol":"AAPL","indexed_price":-0.895252537034877},{"_id":13367,"symbol":"AAPL","indexed_price":-0.8888370465414673},{"_id":13368,"symbol":"AAPL","indexed_price":-0.8770558730899335},{"_id":13369,"symbol":"AAPL","indexed_price":-0.8680741863991601},{"_id":13370,"symbol":"AAPL","indexed_price":-0.8791554881605039},{"_id":13371,"symbol":"AAPL","indexed_price":-0.8665577977370815},{"_id":13372,"symbol":"AAPL","indexed_price":-0.8781056806252187},{"_id":13373,"symbol":"AAPL","indexed_price":-0.8753061938644582},{"_id":13374,"symbol":"AAPL","indexed_price":-0.8684241222442552},{"_id":13375,"symbol":"AAPL","indexed_price":-0.8604922430887672},{"_id":13376,"symbol":"AAPL","indexed_price":-0.8422955791438237},{"_id":13377,"symbol":"AAPL","indexed_price":-0.8496442318908201},{"_id":13378,"symbol":"AAPL","indexed_price":-0.8363466697772075},{"_id":13379,"symbol":"AAPL","indexed_price":-0.810218126676776},{"_id":13380,"symbol":"AAPL","indexed_price":-0.8113845794937594},{"_id":13381,"symbol":"AAPL","indexed_price":-0.7987868890703371},{"_id":13382,"symbol":"AAPL","indexed_price":-0.7739414440685874},{"_id":13383,"symbol":"AAPL","indexed_price":-0.6943893619503091},{"_id":13384,"symbol":"AAPL","indexed_price":-0.6088883704654147},{"_id":13385,"symbol":"AAPL","indexed_price":-0.624402192931296},{"_id":13386,"symbol":"AAPL","indexed_price":-0.5514988918698238},{"_id":13387,"symbol":"AAPL","indexed_price":-0.4767292663011781},{"_id":13388,"symbol":"AAPL","indexed_price":-0.5139391111629534},{"_id":13389,"symbol":"AAPL","indexed_price":-0.5793771141957308},{"_id":13390,"symbol":"AAPL","indexed_price":-0.5362183599673394},{"_id":13391,"symbol":"AAPL","indexed_price":-0.5706287180683541},{"_id":13392,"symbol":"AAPL","indexed_price":-0.5025078735565147},{"_id":13393,"symbol":"AAPL","indexed_price":-0.453050274116412},{"_id":13394,"symbol":"AAPL","indexed_price":-0.37466464481511724},{"_id":13395,"symbol":"AAPL","indexed_price":-0.3282398226991718},{"_id":13396,"symbol":"AAPL","indexed_price":-0.20891169952175445},{"_id":13397,"symbol":"AAPL","indexed_price":-0.16143706987052378},{"_id":13398,"symbol":"AAPL","indexed_price":-0.1192114778957191},{"_id":13399,"symbol":"AAPL","indexed_price":-0.20109646564796463},{"_id":13400,"symbol":"AAPL","indexed_price":-0.2684007931879156},{"_id":13401,"symbol":"AAPL","indexed_price":-0.17893386212527707},{"_id":13402,"symbol":"AAPL","indexed_price":-0.3028111512889304},{"_id":13403,"symbol":"AAPL","indexed_price":-0.3319724717135192},{"_id":13404,"symbol":"AAPL","indexed_price":-0.20727866557797747},{"_id":13405,"symbol":"AAPL","indexed_price":-0.20856176367665938},{"_id":13406,"symbol":"AAPL","indexed_price":-0.10206462148606088},{"_id":13407,"symbol":"AAPL","indexed_price":-0.05424005598973528},{"_id":13408,"symbol":"AAPL","indexed_price":0.06917065204712461},{"_id":13409,"symbol":"AAPL","indexed_price":-0.010381430071153629},{"_id":13410,"symbol":"AAPL","indexed_price":0},{"_id":13411,"symbol":"AAPL","indexed_price":-0.013064271550215846},{"_id":13412,"symbol":"AAPL","indexed_price":0.08375131225941902},{"_id":13413,"symbol":"AAPL","indexed_price":0.1641199113495858},{"_id":13414,"symbol":"AAPL","indexed_price":0.4136241689023678},{"_id":13415,"symbol":"AAPL","indexed_price":0.4235390178467281},{"_id":13416,"symbol":"AAPL","indexed_price":0.5369182316575293},{"_id":13417,"symbol":"AAPL","indexed_price":0.6153038609588241},{"_id":13418,"symbol":"AAPL","indexed_price":0.7901551382246588},{"_id":13419,"symbol":"AAPL","indexed_price":1.2156771258602588},{"_id":13420,"symbol":"AAPL","indexed_price":1.1255103231074302},{"_id":13421,"symbol":"AAPL","indexed_price":1.310509739881022},{"_id":13422,"symbol":"AAPL","indexed_price":0.5789105330689375},{"_id":13423,"symbol":"AAPL","indexed_price":0.45829931179283784},{"_id":13424,"symbol":"AAPL","indexed_price":0.6738597923713985},{"_id":13425,"symbol":"AAPL","indexed_price":1.0290446751428903},{"_id":13426,"symbol":"AAPL","indexed_price":1.2016796920564563},{"_id":13427,"symbol":"AAPL","indexed_price":0.953108596757261},{"_id":13428,"symbol":"AAPL","indexed_price":0.8540767525953573},{"_id":13429,"symbol":"AAPL","indexed_price":0.9774874606322174},{"_id":13430,"symbol":"AAPL","indexed_price":0.3257902717835063},{"_id":13431,"symbol":"AAPL","indexed_price":0.2549865857926047},{"_id":13432,"symbol":"AAPL","indexed_price":0.08095182549865855},{"_id":13433,"symbol":"AAPL","indexed_price":-0.004432520704537614},{"_id":13434,"symbol":"AAPL","indexed_price":0.05132392394727623},{"_id":13435,"symbol":"AAPL","indexed_price":0.041759010848011174},{"_id":13436,"symbol":"AAPL","indexed_price":0.22617520121311094},{"_id":13437,"symbol":"AAPL","indexed_price":0.4677475796104047},{"_id":13438,"symbol":"AAPL","indexed_price":0.5841595707453633},{"_id":13439,"symbol":"AAPL","indexed_price":0.6613787472296746},{"_id":13440,"symbol":"AAPL","indexed_price":0.905867257669427},{"_id":13441,"symbol":"AAPL","indexed_price":0.9620902834480345},{"_id":13442,"symbol":"AAPL","indexed_price":1.1620202962790154},{"_id":13443,"symbol":"AAPL","indexed_price":1.1987635600139974},{"_id":13444,"symbol":"AAPL","indexed_price":1.3318558264318208},{"_id":13445,"symbol":"AAPL","indexed_price":1.458066021229441},{"_id":13446,"symbol":"AAPL","indexed_price":1.2402892802986119},{"_id":13447,"symbol":"AAPL","indexed_price":1.3867957541117462},{"_id":13448,"symbol":"AAPL","indexed_price":1.60142307243672}];

// Checking data
assert.equal(add.length, mod.length);
for(var i=0, len=add.length; i<len; ++i) {
  assert.equal(add[i]._id, mod[i]._id);
  assert.notEqual(add[i].indexed_price, mod[i].indexed_price);
}

var agg = dl.groupby()
  .stream(true)
  .key("_id")
  .summarize([{
    name: "value",
    get: dl.$('indexed_price'),
    ops: ["min", "max"],
    as:  ["min", "max"]
  }]);

for(i=0, len=add.length; i<len; ++i) { agg._add(add[i]); }

console.log(agg.result()[0]); // { min: -0.8618695048588617, max: 4.800260078023406 }
console.log(dl.min(add, dl.$('indexed_price')), dl.max(add, dl.$('indexed_price'))); // Correct

for(i=0, len=mod.length; i<len; ++i) agg._mod(mod[i], add[i]);

console.log(agg.result()[0]); // Incorrect: { min: -0.8618695048588617, max: 4.800260078023406 }
console.log(dl.min(mod, dl.$('indexed_price')), dl.max(mod, dl.$('indexed_price'))); // Correct: -0.9175317858392628 2.6079108043535966

Consider loading data from ndjson

Any chance you'd consider adding another data format for ndjson?

How to get equivalent of pandas.DataFrame.pivot_table?

How to get equivalent of pandas.DataFrame.pivot_table like here?

table = pivot_table(df, values='D', index=['A', 'B'], columns=['C'], aggfunc=[np.sum])

Input:

>>> df
     A    B      C  D
0  foo  one  small  1
1  foo  one  large  2
2  foo  one  large  2
3  foo  two  small  3
4  foo  two  small  3
5  bar  one  large  4
6  bar  one  small  5
7  bar  two  small  6
8  bar  two  large  7

Output:

C        large  small
A   B
bar one    4.0    5.0
    two    7.0    6.0
foo one    4.0    1.0
    two    NaN    6.0

Comma operator in the "switch()" statemnt

This looks very suspicious:

https://github.com/vega/datalib/blob/master/src/format.js#L136

P.S. assignments inside statements are evil :)

How to support signals in dl.template

Right now, the function produced by dl.template.source take a single argument for the data object to parse the handlebars against. In Vega, signals are not namespaced, and I'm wondering how best we should support them with template. Some ideas:

Use dl.extend to pass dl.template an object containing both item.datum properties and signals.
Switch the dl.template function to take vargs, and iterate through them until the property is found.

I prefer the second option, as running dl.extend on every tuple every time an encoder is overly expensive.

In either case, I also need to know which signals are being used within a template expression, in order to compile the signal values and send them into the encoder. i.e., I need from dl.template a list of the handlebar strings used.

Bivariate Stats

mutual information
correlation.

Why does `dl.date` return a unix timestamp?

I was wondering why the datalib method dl.date returns a unix timestamp rather than a Date() object.

> dl.date('2015')
1420099200000
> new Date(dl.date('2015'))
Thu Jan 01 2015 00:00:00 GMT-0800 (PST)

Undefined error w/streaming keyed groupby

This is emulating streaming modifications to tuples. First, add tuples are added to the aggregation, and we get the result. Then, modified tuples are first split into removing the previously seem values (rem) and then adding the new values (mod). Getting the result gives TypeError: Cannot read property 'y' of undefined

var add = [{"x":1,"y":28,"_id":1},{"x":2,"y":55,"_id":2},{"x":3,"y":43,"_id":3},{"x":4,"y":91,"_id":4},{"x":5,"y":81,"_id":5},{"x":6,"y":53,"_id":6},{"x":7,"y":19,"_id":7},{"x":8,"y":87,"_id":8},{"x":9,"y":52,"_id":9},{"x":10,"y":48,"_id":10},{"x":11,"y":24,"_id":11},{"x":12,"y":49,"_id":12},{"x":13,"y":87,"_id":13},{"x":14,"y":66,"_id":14},{"x":15,"y":17,"_id":15},{"x":16,"y":27,"_id":16},{"x":17,"y":68,"_id":17},{"x":18,"y":16,"_id":18},{"x":19,"y":49,"_id":19},{"x":20,"y":15,"_id":20}]

var rem = [{"x":1,"y":28,"_id":1},{"x":3,"y":43,"_id":3},{"x":5,"y":81,"_id":5},{"x":7,"y":19,"_id":7},{"x":9,"y":52,"_id":9},{"x":11,"y":24,"_id":11},{"x":13,"y":87,"_id":13},{"x":15,"y":17,"_id":15},{"x":17,"y":68,"_id":17},{"x":19,"y":49,"_id":19},{"x":2,"y":55,"_id":2},{"x":4,"y":91,"_id":4},{"x":6,"y":53,"_id":6},{"x":8,"y":87,"_id":8},{"x":10,"y":48,"_id":10},{"x":12,"y":49,"_id":12},{"x":14,"y":66,"_id":14},{"x":16,"y":27,"_id":16},{"x":18,"y":16,"_id":18},{"x":20,"y":15,"_id":20}]

var mod = [{"x":2,"y":56,"_id":1},{"x":6,"y":86,"_id":3},{"x":10,"y":162,"_id":5},{"x":14,"y":38,"_id":7},{"x":18,"y":104,"_id":9},{"x":22,"y":48,"_id":11},{"x":26,"y":174,"_id":13},{"x":30,"y":34,"_id":15},{"x":34,"y":136,"_id":17},{"x":38,"y":98,"_id":19},{"x":2,"y":110,"_id":2},{"x":4,"y":182,"_id":4},{"x":6,"y":106,"_id":6},{"x":8,"y":174,"_id":8},{"x":10,"y":96,"_id":10},{"x":12,"y":98,"_id":12},{"x":14,"y":132,"_id":14},{"x":16,"y":54,"_id":16},{"x":18,"y":32,"_id":18},{"x":20,"y":30,"_id":20}]

var agg = dl.groupby().stream(true).key("_id").summarize({ y: ["min", "max"] })
agg.execute(add)
agg.remove(rem).insert(mod).result()

Expose dl.template.source

For use within Vega encoders, which don't have access to item.datum when being compiled to call dl.template directly.

Why is `weekdays` as a time bin unit pluralized?

From https://github.com/vega/datalib/wiki/Statistics#dl_bins_date,
all singular time unit refers to single-part time units
while all plural time units refer to multi-part time units except "weekdays".

Not sure if there is a reason for this?

Map to data set with Country coloumn

Hi, I have i data set which includes a coloumn called Country and naturally consists of a list of countries. I would like to join (assuming via 'lookup' and 'key' properties in vega lite with this data set so that I can plot my data onto the world-110m according to the country.

I could see from other examples (different maps) that this can be done via 'Id' or'Name_1' field however I cannot seem to get it anything to work for world-110m.

Could you kindly confirm what field I should use to join this 'world-110m' with the 'Country' field in my data set?
Thanks

Uncaught ReferenceError: dl is not defined in datalib homepage.

I tried typing dl in the console in http://vega.github.io/datalib/.

Standalone browser build

For those of us already using d3 it would be sweet to have a build without d3 bundled, to save on JS payload sizes.

Parse numbers with , or .

Datalib can parse some numbers such as

$ dl.number('100.00')
100
$ dl.number('100,00')   // common in europe
NaN

But infer does something different

$ dl.type.infer('100')
integer
$ dl.type.infer('100.00')
string
$ dl.type.infer('100,00')
string

So datalib can parse something as a number although infer does not infer it to be a number (this is understandable).

I'm aware that it is a bit problematic to parse . and , since 100,000 could be either 100000 or 100 but maybe inferAll can make a smarter prediction.

Messytables does those predictions very aggressively and I'm not sure that we want to go that far. However, we should be at least able to parse csv with correctly.

a
"100.00"
"42.10"
"7.62"
"41.71"

dl.csv({url: 'https://dl.dropboxusercontent.com/u/12770094/simple.csv'}, {parse: 'auto'})

parses as strings unless you force it

dl.csv({url: 'https://dl.dropboxusercontent.com/u/12770094/simple.csv'}, {parse: {a: 'number'}})

I'm mostly asking for advice whether datalib should handle this case at all or whether I have to change my code to replace , or . by hand or even write a smart type prediction.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.