Comments (7)
Hi @mbensouda ,
If you want to assess the interpolation between 0 and 4, you need to have the OT problem computed between 0 and 4 (as well as for 0,2 and 2,4). Therefore, you can just add the problem [0,4] and solve it, see e.g. https://moscot.readthedocs.io/en/latest/notebooks/examples/problems/300_adding_and_removing_problems.html
We often observed the interpolation to not work particularly well, curious to see whether it works for you. I would rather validate the OT mapping based on annotations, e.g. via cell_transition
from moscot.
thank you so much for you quick answer Dominik!
- I am going to try this and also compare the result with what is obtained in the Schiebinger dataset in this notebook :https://nbviewer.org/github/broadinstitute/wot/blob/master/notebooks/Notebook-7-ot-validation.ipynb, have you generated a comparison between moscot and what what was generated in this notebook? I can share with you the results on this dataset if you need for a tutorial for Moscot interpolation.
- Unfortunately for the datasets I have I have a little information about annotations but will give it a try for sure also!
How would you recommend to fine tune the OT problem? The choice of hyperparameters is not so easy to interpret.
Thank you so much for your help!
from moscot.
Hi just to let you know that it worked! but all my Wasserstein distances (tp.compute_interpolated_distance, tp.compute_random_distance, tp.compute_time_point_distances
) are generating the following warning message:
WARNING Solver did not converge in the
x/y term.
- Would you know how to solve this warning? Should I fine tune with changing hyperparameters of the OT solving?
- How to interpret these plots to say if the OT problem has worked? I am going to reproduce the ones in this notebook from the Schiebinger paper if you are interested! I can share it with you: https://nbviewer.org/github/broadinstitute/wot/blob/master/notebooks/Notebook-7-ot-validation.ipynb, I dont know how they generate error-bars also, they certainly bootstrap the problem.
Thank you for your answer!! that is very helpful!
from moscot.
Hi @mbensouda ,
happy it works.
There are multiple reasons why I think this is of limited feasibility. First, the quality of the interpolation depends a lot on the entropic regularisation. For eps -> inf, everyhting will be projected onto one point. Second, the barycentric projection assumes linearity, even if the cost is non-linear. Depending on the dataset, this could be a huge issue (e.g. the pancreas dataset in the moscot paper).
We compute the Sinkhorn divergence to compare the distributions. Do you know which distr. distance WOT uses? This is also where the error message coems from (as you have to compute three ent. OT plans to compute the Sink. div.). Try increasing epsilon
via the kwargs to sinkhorn_divergence
, see the docs here: https://moscot.readthedocs.io/en/latest/genapi/moscot.problems.time.TemporalProblem.compute_random_distance.html#moscot.problems.time.TemporalProblem.compute_random_distance
Yes, they also use random samples from the transport map, I'm not sure why they do this (hence we also do it, and you can get different results based on the seed). In expectation that's obv. the same as the McCann interpolant.
Lmk if that helps.
from moscot.
thank you very much for your answers! I will look more deeply into this and come back to you.
also, I have a comment regarding my first question, in the WOT script they use the OT problem computed for all time points to generate the interpolated distribution:
- for example for t_1, t_2, t_3, you generate the transport map from t_1 to t_3 by computing the coupling matrix between t_1 and t_3 and not solving the OT problent between t_1 and t_3
from moscot.
yes, you can also do that with moscot. If you don't solve t_1 to t_3, then doing e.g. pull(t_1, t_3)
should be done using the concatenation of the transport maps.
from moscot.
Closing this issue for now. Please feel free to re-open if you have any more questions.
from moscot.
Related Issues (20)
- Buggy Test: test_gw_problem gives joint_attr HOT 1
- Buggy Test: `scale_cost` given to prepare in tests/problems/time/test_mixins.py HOT 1
- check the convergence flag
- change links to papers
- adapt low rank gamma default for linear problems HOT 1
- describe birth death process better HOT 1
- Extract the cells in the trace HOT 4
- adapt user guide
- Refactor: Check if both x and x_callback is given
- Refactor: Maybe GW should inherit from FGW instead of vice versa HOT 2
- check for failing tests HOT 1
- check for unrecognized in `solve` methods
- check for warnings thrown by us
- Add Elastic Costs again or asses if they are necessary or not HOT 2
- Plotting Tests Fail HOT 1
- `.impute` consumes too much memory HOT 2
- output the raw cell transition data without aggregation HOT 6
- Full rank initializers
- Questions about patient manifold.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moscot.