First of all, thanks for the paper and code, it's very inspiring.

Specifically, first of all, I tried to generate 100 classes of vectors (each class contains 300 samples). These vectors follow multivariate student t distribution of 100 class-dependent means and variances. Then I split these 100 classes into 80 and 20 for training and inferencing respectively. Finally, I try to use rbig following:

#rBIG
n_layers = 1000
rotation_type = 'pca'
random_state = 123
zero_tolerance = 10 # I also tried 60

#Initialize RBIG class
rbig_model = RBIG(n_layers=n_layers, rotation_type=rotation_type, random_state=random_state, zero_tolerance=zero_tolerance)

train_dataset_rbig = rbig_model.fit_transform(train_dataset)
test_dataset_rbig = rbig_model.transform(test_dataset)

then train_dataset_rbig is Okay, but NaN appears in test_dataset_rbig.

Am I wrongly using rbig or something else?

Thanks!

Right now the inverse is not perfect. It means that you dont get the same input, i.e. x' = f^-1(f(x)) x'=!x.

For instance you can see it in this notebook

This is probably related to this #6

We can save a lot of memory by not saving the parameters when estimating information theory measures.

• Total Correlation
• Entropy

Hey,
Is the code for estimating KL-Divergence working?
Right now I see that the code is in comment mode.

rbig._src.kld \ RBIGKLD

Need a way to handle boundary issues during the uniform distribution.

Some Ideas

The repo from the DDL library has a way to do this. Some sort of clipping based on the support of the input data. Maybe save this into a named tuple for the parameters.

• Make Interior Probability
• Make Finite

Util Functions

• Extend Domain
• Boundary Warning

Hi Team,
Thanks for addressing the issue of density estimation for multidimensional data.
I have a few questions as I am trying to implement information theory metrics:

• Q1.Is this method apt for high dimensional tabular data?
• Q2.I have been trying to run RBIG mutual info() over a tabular data and the results are exact same for all of them, I did check the results using SK learn MI score and got variables results (results not normalized in both cases- SK learn and RBIG). I don't understand the error, can you in anyway help me with this?

below is the piece of code I used:

X: features (attributes not in Y)
Y: set of y attributes (attributes not in X) (let's say y1,y2,y3,y4)
def calculate_miscore_xa(data,X,Y):
mis_xy = []
y_attributes = []
for y in Y:
rbig_model = MutualInfoRBIG(max_layers = 10000)
rbig_model.fit(data[X], data[[y]]);
mi_rbig = rbig_model.mutual_info() * np.log(2)
mis_xy.append(mi_rbig)
y_attributes.append(a)
mis_xy = pd.DataFrame({'Y':y_attributes, 'I(Xi,Y)': mis_xy})
return mis_xy

basically the results I am getting is
I(X,y1) = I(X,y2) = I(X,y3) = I(X,y4) = exact same
It's unusual hence I checked the results using https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mutual_info_score.html and the results for I(X,y1), I(X,y2), I(X,y3),I(X,y4) differ.
Can you help me understand if there is anythings I am doing wrong ?

Also the original calculation using entropy implemented in information theory notebook can be used used as base for tabular data by substituting respective X and Y in 2d format?

Thanks and Regards
Surbhi

There is something with the probability estimation that makes it take too long.

I train a model of 3k features with 50k samples and try to compute the probability for a batch of 100 and it can, and even for 1 sample takes more time than training.

There are other projects that needed to be added into the Documentation:

• RBIG 4 IT
• RBIG 4 EO
  • Paper
  • Repo

Need some demo notebooks for the documentation.

Demo Notebooks

• Univariate Transformations
• Full Gaussianization
• Stopping Criteria
• Sampling
• Information Theory Measures
  • Less Memory Version (less memory)

In the notebooks, there is no consistency with which params are default and which are not. Need to ensure that they all are the same.

Params so far:

• zero_tolerance=60 - the amount of iterations before convergence, example.
• pdf_extension=10 - the extension for the support, example

Need to incorporate some simple plotting functionality for the 2D examples. Seaborn is good for this.

• 1D Histograms
• 2D Joint Plots
  • KDE
  • Scatter
• GIFs for RBIG Layers

I find that the MATLAB comments are much more helpful in understanding what RBIG is doing and the python comments are much better at describing the inputs. (example). It would be nice to sync these up.

Need to test univariate transformations.

Tests

• Input Output Shapes
• Invertibility (small residuals)
• Compound transforms (MG)

Transforms

• Univariate Histogram
• Inverse CDF
• KDE Transforms
  • Exact
  • FFT
  • KNN