Different results using skorch and my training function about braindecode HOT 5 CLOSED

Great that you put up that example. This is exactly what we need at the moment to verify real use cases. A quick scan through the code I didn't notice any reason unfortunately (you took care of shuffling, batch size etc.).
The accuracy_score at the very end of the script what does it output?

from braindecode.

One thing I didn't fix is random_seed (and I think we should freeze it) but for each run you can draw similar conclusions. I've run it again and the accuracy_score is 0.32 (validation: 0.29 from 1st epoch) in comparison to 0.45 from my function (validation 0.58 for epoch 21).

from braindecode.

Today I will submit an issue to skorch, to ask what is the difference between skorch training loop and the on defined by me. I saved numpy arrays to remove not needed data processing and include everything except model defininton in one file https://github.com/sliwy/braindecode/blob/strange_example/examples/skorch_ext_example.py
Let you know if I know anything new.

from braindecode.

I'm closing it as I've found the reason, in my train function there was a typo and I was setting model to train mode only at the beginning of training. After first validation step the model was in eval mode till the end of training.

from braindecode.

I see, great catch. Then it means that in your case it is actually better to train in eval mode?

from braindecode.