m parameter about kitnet-py HOT 5 CLOSED

from kitnet-py.

Yes but if the Input layer is allready of dimension 1. We cannot reduce more than 1. So the encoding layer will be size one.

from kitnet-py.

Ah yes, I misunderstood you. When m=1 it means that the maximum input for any autoencoder in the ensemble layer should be at most 1. So yes, these autoencoders are one dimensional. However, the output layer will have 1 input for every autoencoder in the ensemble layer. So, in essence, the entire architecture reduces to one wide autoencoder (the output layer). Of course, the output autoencoder will have compression in its hidden layer.

from kitnet-py.

Ok, but the autoencoder in the output layer, take as entry the RMSE of the n(number of statistics) one dimensional auto encoder(Ensemble layer) ? And if these previous autoencoder(ensemble layer) are one dimensional they will learn the identity and then their RMSE will be allways 0. So the output layer autoencoder will have as entry always n zeros ?

from kitnet-py.

It is more likely that there will be large mistakes since one neuron will have trouble fitting the distribution of the input feature. As a result, the RMSE distribution will likely reflect the original distribution.

Regardless, I highly recommend mot setting m to small values, especially m=1. This is because the purpose of the ensemble layer is to learn meaningful representations from sets of features, to thus make the collective computation more efficient. If you are setting m=1, you are better off not using KitNET and rather using a regular autoencoder.

Hope this helps

from kitnet-py.