Comparison with `torch.compile` instead of Eager about lightning-thunder HOT 3 CLOSED

Hi! thanks for your contribution!, great first issue!

from lightning-thunder.

We love torch.compile and use it a lot, too, and when enabling it in thunder (and perhaps disabling other backends) we expect numbers to be in the same ballpark. Do try it out!
That said, PyTorch eager is a more stable anchor point for us (maybe like the Prototype Metre in olden times), so we think it is most useful to our measurement.

from lightning-thunder.

Thank you @bonlime, as @t-vi wrote our baseline is eager.
We are all about compounding the effect of multiple optimizations and have torch.compile in the mix.

In the case of the plot it was roughly equivalent to the first bar in that particular experiment, but again, we are not interested in a head to head comparison as it would be misleading with respect to the goals of the project.

from lightning-thunder.