bug when using tp.compute_interpolated_distance about moscot HOT 7 CLOSED

Hi @mbensouda ,

If you want to assess the interpolation between 0 and 4, you need to have the OT problem computed between 0 and 4 (as well as for 0,2 and 2,4). Therefore, you can just add the problem [0,4] and solve it, see e.g. https://moscot.readthedocs.io/en/latest/notebooks/examples/problems/300_adding_and_removing_problems.html

We often observed the interpolation to not work particularly well, curious to see whether it works for you. I would rather validate the OT mapping based on annotations, e.g. via cell_transition

from moscot.

thank you so much for you quick answer Dominik!

1. I am going to try this and also compare the result with what is obtained in the Schiebinger dataset in this notebook :https://nbviewer.org/github/broadinstitute/wot/blob/master/notebooks/Notebook-7-ot-validation.ipynb, have you generated a comparison between moscot and what what was generated in this notebook? I can share with you the results on this dataset if you need for a tutorial for Moscot interpolation.
2. Unfortunately for the datasets I have I have a little information about annotations but will give it a try for sure also!

How would you recommend to fine tune the OT problem? The choice of hyperparameters is not so easy to interpret.

Thank you so much for your help!

from moscot.

Hi just to let you know that it worked! but all my Wasserstein distances (tp.compute_interpolated_distance, tp.compute_random_distance, tp.compute_time_point_distances) are generating the following warning message:
WARNING Solver did not converge in the x/y term.

1. Would you know how to solve this warning? Should I fine tune with changing hyperparameters of the OT solving?
2. How to interpret these plots to say if the OT problem has worked? I am going to reproduce the ones in this notebook from the Schiebinger paper if you are interested! I can share it with you: https://nbviewer.org/github/broadinstitute/wot/blob/master/notebooks/Notebook-7-ot-validation.ipynb, I dont know how they generate error-bars also, they certainly bootstrap the problem.

Thank you for your answer!! that is very helpful!

from moscot.

Hi @mbensouda ,

happy it works.
There are multiple reasons why I think this is of limited feasibility. First, the quality of the interpolation depends a lot on the entropic regularisation. For eps -> inf, everyhting will be projected onto one point. Second, the barycentric projection assumes linearity, even if the cost is non-linear. Depending on the dataset, this could be a huge issue (e.g. the pancreas dataset in the moscot paper).

We compute the Sinkhorn divergence to compare the distributions. Do you know which distr. distance WOT uses? This is also where the error message coems from (as you have to compute three ent. OT plans to compute the Sink. div.). Try increasing epsilon via the kwargs to sinkhorn_divergence, see the docs here: https://moscot.readthedocs.io/en/latest/genapi/moscot.problems.time.TemporalProblem.compute_random_distance.html#moscot.problems.time.TemporalProblem.compute_random_distance

Yes, they also use random samples from the transport map, I'm not sure why they do this (hence we also do it, and you can get different results based on the seed). In expectation that's obv. the same as the McCann interpolant.

Lmk if that helps.

from moscot.

thank you very much for your answers! I will look more deeply into this and come back to you.
also, I have a comment regarding my first question, in the WOT script they use the OT problem computed for all time points to generate the interpolated distribution:

• for example for t_1, t_2, t_3, you generate the transport map from t_1 to t_3 by computing the coupling matrix between t_1 and t_3 and not solving the OT problent between t_1 and t_3

from moscot.

yes, you can also do that with moscot. If you don't solve t_1 to t_3, then doing e.g. pull(t_1, t_3) should be done using the concatenation of the transport maps.

from moscot.

Closing this issue for now. Please feel free to re-open if you have any more questions.

from moscot.