Learned SDE is quite smooth about torchsde HOT 7 CLOSED

This is a known problem with latent SDEs -- the diffusion is always a bit too small. I think this could be mitigated by tweaking the training objective (probably the reconstruction loss encourages small diffusions), or by additionally training wrt a discriminator (see Neural SDEs as Infinite-Dimensional GANs). To my knowledge no-one has really tried this, or tried working out the details though.

from torchsde.

Have you tried increasing the penalty on the path-space KL?

The prior diffusion is unlikely to collapse, since it's forced to match multiple posterior diffusions approximately, simultaneously.

from torchsde.

This is a known problem with latent SDEs -- the diffusion is always a bit too small. I think this could be mitigated by tweaking the training objective (probably the reconstruction loss encourages small diffusions), or by additionally training wrt a discriminator (see Neural SDEs as Infinite-Dimensional GANs). To my knowledge no-one has really tried this, or tried working out the details though.

Thanks for the reply. I am trying to adjust some loss functions.

Have you tried increasing the penalty on the path-space KL?

The prior diffusion is unlikely to collapse, since it's forced to match multiple posterior diffusions approximately, simultaneously.

Thank you, so which loss should I increase log_pxs or logqp0 + logqp_path?

from torchsde.

I'm referring to the logqp_path part.

from torchsde.

I'm referring to the logqp_path part.

Yes, it helps! Thank you guys for the timely reply. I am closing this issue.

from torchsde.

I'm referring to the logqp_path part.

Yes, it helps! Thank you guys for the timely reply. I am closing this issue.

Awesome! Feel free to reopen if/when you have follow-up questions.

from torchsde.

This is really cool to see -- thank you for sharing!
Honestly, "better neural SDEs" is still an open research topic. I would love to see more work like yours in this space. :)

from torchsde.