Computational time for Brownian Interval about torchsde HOT 2 OPEN

When I change from th.tensor([0.0, 1.0]), dt=0.01 to th.tensor([0.0, 5.0]), dt=0.05.

from torchsde.

So this is a bit weird! In particular because one tends to imagine a BrownianInterval being scale-invariant. I've been able to figure out why this happens, but I don't have a fix in mind yet.

The reason has to do with the binary tree heuristic built in to the BrownianInterval. Once >100 queries have been made, the BrownianInterval averages the step sizes it has observed over those queries, and uses those as an estimate of the average step size for the rest of the SDE solve. This is used to build up a binary tree, as per Section E.2, "Backward pass" of the paper. (Which I refer to as it sounds like you've read it.)

The dt=0.01 case makes 101 steps, which triggers the calculation of this heuristic. Evaluating that heuristic (building up the binary tree) is what takes up so much time. The other cases make only 100 steps, and the solve actually completes before the heuristic even triggers.

Why different number of steps between these apparently-scale-invariant cases? Floating point inaccuracies:

>>> import torch
>>> x = torch.tensor(0.01)
>>> sum = 0
>>> for _ in range(100):
...     sum = sum + x
...
>>> sum
tensor(1.0000)
>>> sum < 1
tensor(True)

The real bug here is simply that the heuristic takes so much time to compute. I'll need to have a deeper look, later, to figure out what might be done to resolve this.

from torchsde.