Flash Attention Support? about longlm HOT 4 CLOSED

Flashattention support requires heavy work. Now, we plan to finish our paper first, after that, we will implement it. Thanks!

from longlm.

Thanks, will be looking forward to it

from longlm.

I am looking forward to it, too

from longlm.

I am looking forward to it, too

We've already had the implementation. Actually,the new results we released on X (previous Twitter) with Googles' Gemma are based on this implementation (otherwise we cannot do it on sequences > 30k). However, with the current implementation, we cannot reach the same results（on Longbench）as what we reported in the paper (based on the no flash attention version). There is a minor performance gap between the two versions. We are still trying to figure out the reason.

from longlm.