Custom losses, coherent embeddings about umap HOT 6 OPEN

It is a little hard to do that and still maintain efficiency; potentially it could be added as an extra code-path on the side that is slower but more flexible. That would be a more significant project however.

from umap.

Thank you for the prompt reply!
I'm actually looking forward seeing your write-up about UMAP algorithm, hoping to reimplement it in a flexible way. For example, the most naive implementation of tsne loss with TensorFlow boils down to something like this:

def tsne_kl_loss(points, P):
    n = tf.shape(points)[0]
    Q = 1.0 / (1.0 + pdist2(points))
    sQ = tf.reduce_sum(Q) - tf.cast(n, tf.float32)
    return tf.reduce_sum(P*tf.log(P/Q)) + tf.log(sQ)

Then one can combine this loss with others and use one of standard optimizers.
Do you think if this kind of approach can be adopted to UMAP?

from umap.

At that level, yes almost certainly; if you are willing to do N^2 work then you can certainly have custom loss -- I was generally seeking to avoid that. On that front you might be interested by smallvis which implements t-SNE, LargeVis and UMAP in a common framework which I suspect would easily be adaptable to custom loss functions. The cathc is that it only supports small datasets for exactly the reason cited above. As a way to experiment, however, it is quite powerful.

from umap.

Thank you for pointing me to smallvis! I think that's exactly what I needed.
N^2 can actually go surprisingly far with efficient GPU implementation. For example, here is a random youtube video showing n^2 nbody with 60k particles at 30fps.

Still, I like to think of algorithmic optimizations, like employing BH or something else as yet another, partially orthogonal component.

from umap.

How to obtain the matrix P from UMAP? Is it self.graph_ from fuzzy_simplicial_set function?

Thanks!

from umap.

That is the equivalent of it, yes.

from umap.