Fresh ML library from the master branch about ml HOT 8 CLOSED

Hi Pindash,

Thanks for the message, the function is actually doings as it's designed to from what I can see, the take function in the form x#y where x is a list of symbols and y is an unkeyed table will return an un-keyed table containing the subset of columns x from y as in the below example.

q)5#table:([]100?0b;100?1f;100?0b;100?1f)
x         x1 x2         x3
--------------------------
0.7433285 0  0.8481567  0 
0.2371288 1  0.389056   1 
0.567081  1  0.391543   1 
0.4269177 1  0.08123546 0 
0.7704774 0  0.9367503  0 
q)binaryfunc:{2=count distinct x}       /find where columns contain only 'binary' data
q)show binarycols:where binaryfunc each flip table
`x1`x3
q)5#binarytab:binarycols#table
x1 x3
-----
0  0 
1  1 
1  1 
1  0 
0  0 

This is outlined at https://code.kx.com/q/ref/lists/#take . If you can provide a specific edge case where you have experienced an issue with the function in a working example I'll be more than happy to take a look at it and add in a fix.

Kind regards,

Conor

from ml.

from ml.

Yeah if you could rerun in the morning that would be great, I'm just after cloning from the git and rerunning the example that's in on code.kx (first running the feature creation section followed by the feature significance section below it) with out any issues. I'm also running 3.6 on Mac but it's also been tested on windows and linux machines.

Let me know what you find tomorrow.

All the best

from ml.

Hey so I tested again today and it still fails:

Here is sample code to reproduce:
\l ml/ml.q
.ml.loadfile:fresh/init.q t:flip absenergy_orderSizeabssumchange_orderSizecount_orderSize!(4 625 2116 1 4 1 1 10000 10000 6290064f;0 0 0 0 0 0 0 0 0 0f;1 1 1 1 1 1 1 1 1 1)
tgt:enlist each 108.3051 108.3051 108.3051 108.3051 108.3051 108.3051 108.3051 108.3051 108.3051 108.3051
.ml.fresh.significantfeatures[t;tgt]

from ml.

So the error is based on the fact that there are no 'binary' columns in the dataset you've provided for the feature significance tests, if it's passed the outputs from the feature creation procedure that's unlikely to be an issue, I'll modify the function tomorrow to handle cases that don't have binary or real components but I think this is likely only a problem in situations where the feature significance procedure is used without considering the feature creation part of the FRESH algorithm which it's built for

I'll have a look at this in the morning though

from ml.

from ml.

I've now modified the code to take in your changes. Thanks for highlighting the issue!

from ml.

from ml.