[Local] Context lost on regenerate about maid HOT 9 CLOSED

So it took me a while to notice the correlation but I did some tests and am now sure about this: Pressing the reload button (which let's the model replace it's answer by a new one) does delete the whole context. This means that it forgets everything that was spoken before the last prompt (which obviously leads to hallucinations).

Ahh OK that makes sense, I was starting to suspect something like that is going on. I'll look into it.

from maid.

Yeah this is turning out to be a bigger issue than i expected. Might take a while

from maid.

In a quick test, deleting this line, it successfully works during regeneration.

Context wasn't lost, and the regenerate response was aligned.

Yes but the point of regeneration is the previous context needs to be deleted because its creating a new branch from the previous message, then the context needs to be regenerated and added in. i know how to do this part the problem is its causing a memory error.

from maid.

I'm unable to test at the moment; can someone try removing this line?

https://github.com/Mobile-Artificial-Intelligence/maid/blob/dc44cfd1b7d2db947d21caa38ed9cd9decdbbbdb/lib/providers/session.dart#L146C7-L146C8

from maid.

I'm unable to test at the moment; can someone try removing this line?

https://github.com/Mobile-Artificial-Intelligence/maid/blob/dc44cfd1b7d2db947d21caa38ed9cd9decdbbbdb/lib/providers/session.dart#L146C7-L146C8
no thats not it, its because the messages arnt being passed into it properly. I can fix that but it ends up causing another issue with llama.cpp

from maid.

So you've tried it ?

from maid.

So you've tried it ?

No i havnt tried it but i know what it does and i know its not what is causing the error

from maid.

In a quick test, deleting this line, it successfully works during regeneration.

Context wasn't lost, and the regenerate response was aligned.

from maid.

I did my test and can confirm that this is fixed.

from maid.