weihua916 / imsat Goto Github PK

Hi @weihua916,
Thanks for the code release.
To run calculate_distance.py, I had to replace mnist = fetch_mldata('MNIST original', data_home=PATH) by mnist = fetch_openml('mnist_784') since fetch_mldata is deprecated. Note that this should not affect the data at all.

I cannot reproduce 10th_neighbors.txt
The first values of the file I generated are :
1.260706806182861328e+01 9.380294799804687500e+00 1.302325439453125000e+01 4.264658927917480469e+00 9.928882598876953125e+00 1.266661167144775391e+01 4.756112575531005859e+00 1.012734127044677734e+01 4.143903255462646484e+00 9.276198387145996094e+00

The problem is that your method is less stable with these values. Here are 10 runs :
0.98, 0.98, 0.91, 0.88, 0.87, 0.98, 0.98, 0.91, 0.96, 0.86.

Can you make sure that calculate_distance.py generates your own 10th_neighbors.txt or update the depo?
Thanks for you time and help,
Thibault

Hey,

Are you sure the computation of pairwise mutual information in the hashing example is correct? I've been working through the maths (and reverse-engineering what you compute) and I'm not entirely convinced that the value you compute is correct.

You seem to be evaluating p(Y_a, Y_b | X) * lg (p(Y_a, Y_b | X) / p(Y_a, Y_b))

Whereas I was expecting the correct term to be I(Y_a ; Y_b) = H(Y_a) - H(Y_a | Y_b) =p(Y_a) * lg(p(Y_a)) - p(Y_a, Y_b) * lg (p(Y_a, Y_b) / p(Y_b))

I'm not certain yet though. Do you have any notes on how you derived the current form?