Performance and stability issues in post 1.1.8 builds about maid HOT 10 CLOSED

yeah im doing alot of work on core.cpp at the moment its likely unstable for the time being.

this build seemed to be more stable on my machine though so maybe give it a try

https://github.com/Mobile-Artificial-Intelligence/maid/actions/runs/7519009453

from maid.

weird, it works fine on linux.

back to the drawing board

from maid.

I think ill move the logic that excludes the user input and system prompt to dart. For some reason I think that's a source of issues and moving it to dart will make it easier to debug

from maid.

Haha yeah the only reason its stable now is because i reverted everything back to 1.1.8 and started again lol.

And yeah i can add that pretty easily but all the chat controls are already pretty cluttered so i will probably change them at some point.

from maid.

yeah im doing alot of work on core.cpp at the moment its likely unstable for the time being.

this build seemed to be more stable on my machine though so maybe give it a try

https://github.com/Mobile-Artificial-Intelligence/maid/actions/runs/7519009453

now it seems that with chatml prompt formating is crashing(I'm on Android btw) running stablelm3b based models, with sheared llama3b it generates a little bit and crashes on alpaca prompt format, on chatml it runs a little bit more but at the end of <|im_start|>user it crashes.

from maid.

I'm able to get it to prompt with calypso 3B (Alpaca) on my phone but it seems to crash after the second prompt.

from maid.

minima-3b-layla-v2.q3_k_m.gguf runs weirdly doesn't stop talking but it hasn't crashed. On llama.cpp is working normal.

from maid.

I noticed something similar on desktop with xwin-mlewd-7b, its almost like the format strings arnt being added. Another issue I noticed was sometimes the response would continue the users message.

So I'll definitely have to look into the format logic

from maid.

minima-3b-layla-v2.q3_k_m.gguf runs weirdly doesn't stop talking but it hasn't crashed. On llama.cpp is working normal.

with chatml doesn't crash too but same strange output, and with raw prompt format it kinda leaks the system prompt.

from maid.

can you add the possibility to edit the AI's response? btw now its seems more stable, notice that it's a recommended thing uninstalling the old version and installing the new version, sometimes I simply installed one on top of each other and it inherits some bugs

from maid.