Comparison to alternative ODE models about torchcde HOT 4 CLOSED

Using a CDE to compute the integral of its input path is straightforward. This is Example 3.2 in On Neural Differential Equations.

from torchcde.

Hey Deniz, thanks for your interest. The answer is that the control should include time as a channel. That is, assume we're given some data x_i observed at times t_i and let x be such that x(t_i) = x_i. Then you should drive the CDE by the path (t, x(t)), not just x(t).

If you check the technical statement of the theoretical result it should be including time as a channel. And it is for similar reasons that the paper also emphasises that time should be included as a channel when training models in practice. (As this really does affect expressivity of the model.)

Your example is a good one, though! This is actually the canonical example for demonstrating why time should be included as a channel: it's Example 3.8 in On Neural Differential Equations. And conversely there exist results on universal approximation, comparison-to-alternative-ODEs, etc. stating that you can approximate anything as long as you do include time as a channel.

from torchcde.

I appreciate the prompt reply 😊 I think now I understand why we must include also the time into the control path. However, I'm still wondering what exactly would be the form of such a neural CDE to calculate the integral of its "input path" 🤔 Do you think it is something trivial or would it be some complicated set of functions, one for the input network, one for the neural vector field, and one for the output network (borrowing terminology from this paper)?

from torchcde.

That example is the perfect answer, brilliant! Thank you.

from torchcde.