Comments (9)
@ibehnam Yes, I thought the problem is gone, but I gave it so much flexibility because otherwise weird problems arise, like the llm adding endless zeroes to a number, always writing float number even when give a integer number etc. I will try to write a grammar generator that defines only necessary whitespaces, and gives the llm the option to use a space or linebreak. Right now it is free to generate as much whitespaces it wants.
from llama-cpp-agent.
But that takes some time, I will try to do this. I think I will do it this month.
from llama-cpp-agent.
@Maximilian-Winter Thanks a lot! Sure, I'll check it out as soon as I can and will update here.
from llama-cpp-agent.
@ibehnam Thank you for the model, I could reproduce the issue and fixed it in the repo.
The problem was an line break followed by whitespace
from llama-cpp-agent.
Thanks, I know it's a lot of work, and I'd appreciate it. Didn't know about the other weird problems that could arise without giving more flexibility to the LLM. The json.gbnf
example on llama.cpp also uses ws
, but in that example, two ws
s can't appear next to each other. It looks like the source of the problem is that llama-cpp-agent allows for that to happen.
from llama-cpp-agent.
@ibehnam Hi, can you take a look at your issue with the latest commit? I think I found the problem and fixed it.
from llama-cpp-agent.
@ibehnam Did you manage to check the new version?
from llama-cpp-agent.
@Maximilian-Winter Hi, yes I actually just tried it again. It's definitely gotten better than before, but sometimes even simple Pydantic classes like the following lead to errors with the latest llama.cpp server:
class Bio(BaseModel):
first_name: str = Field(default=..., description="The person's first name")
last_name: str = Field(default=..., description="The person's last name")
age: int = Field(default=..., description="The person's age")
I get bad request errors (10% of the time). I haven't been streaming the responses so I'm not sure if it's due to infinite generations of \n
/<space>
.
from llama-cpp-agent.
Thank you! I will also test it on a bit more advanced classes and update here if there's any issues.
from llama-cpp-agent.
Related Issues (20)
- Multiple models context management like Ollama.
- Dependency Dashboard
- Stop LLM output on user request? HOT 11
- Some about documentation page of project. HOT 3
- Is it necessary add additional_fields to AgentChainElement ? HOT 3
- Chat format ignored HOT 5
- Slow processing of follow-up prompt HOT 3
- Return control after function executed HOT 9
- Using 01_Basics example, the model is not loading in GPU HOT 8
- llama-cpp-agent v0.2.1 does not stop generating text HOT 2
- Crash, when setting top_k, top_p, or repeat_penalty HOT 15
- Is it possible to add mermaid syntax grammar support? HOT 1
- llama_model.reset() Does Not Clear Context History (Phi-3 4k)
- Avoid generating newlines and spaces rules in the GBNF grammar HOT 3
- Stuck at output HOT 6
- Enable streaming for chains and structured output HOT 1
- Request for image input support HOT 1
- Add support for openAI LLM HOT 3
- Add support for Exllamav2 HOT 1
- Very poor performance writing code, possibly due to structure?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llama-cpp-agent.