r4j4h / crossfilter-ma Goto Github PK

So right now when using .top(3) for example, only the top 3 records are fetched from the source group and their % change or moving averages calculated from there on. This could lead to misleading numbers.

An argument could be made that all the records should be fetched from the source group and their % change or moving averages calculated and only the top 3 of those post-calculation results returned.

In the meantime, using .top(Infinity) or .all() and then grabbing/slicing from the returned complete array to maintain prior sorting should always provide accurate numbers. If sorted % change or moving averages is desired, then either of those in combination with the appropriate sortBy should always provide accurate numbers.

IOW, .top(K) with fewer K than there are data points currently works off those data points as if only they exist.

The area I want to explore is intelligently padding the K argument to include needed data in calculations while keeping it out of the results. At least for % change and short-term moving averages this should make up for the incorrect values whilst avoiding requiring iterating the entire data set.

Inner iteration processing on percentage change works well and is very useful and should also exist in moving average.

Currently there is a leaky abstraction towards the dependency of sorting the keys.

Strings like "2012-01-9" and "2012-01-10" will be handled differently than "2012-01-09" and "2012-01-10".

The user can prepare their data accordingly, but having crossfilter$ma strengthened against all possible cases, or at least configurable to handle different data schemas would be ideal.

Right now, unit tests run great in browsers but on some systems sort the NaNs differently when the tests are run headlessly.

While we want NaN tests, this implementation specific detail only muddies the tests. This issue is to rewrite the unit tests to test NaN specifically, and to not fail when the only problem is NaN sorting behavior.

Right now, if we give in data for days 13 and 15, then the moving average and percent change account these as neighboring nodes.

Really though, day 14 is missing. Behavior in this edge case should be configurable.

Potential paths:

• Ignore the missing data point and assume it stays the same
• Bucket the missing data points with 0
• Throw an error

Bucketing the missing data points with 0 can be done before handing the group in, but would nice to have built-in as it is easy to run in to.