lightonai / paradigm-client Goto Github PK

This snippet:

from paradigm_client.remote_model import RemoteModel
from paradigm_client.request import CreateParameters, CreateRequest


host = "https://llm.lighton.ai"
model = RemoteModel(host, model_name="llm-mini")

requests = CreateRequest(text="Hello I am")
response = model.create_from_objects(requests)

generates an error response:
1 validation error for CreateRequest\nuse_session\n value could not be parsed to a boolean (type=type_error.bool)
while changing the last line to

response = model.create_from_objects(requests, use_session=True)

works fine.

For some reason, the default value of use_session is interpreted as an extra param in the request?

In request.py there is a constant

MAX_SEQ_LEN = 2048

which is used in the @validator of the parameters.

If using a model with longer context length, this becomes a problem to use.

I've tried overriding the validator with a children class, but it doesn't seem to be possible with pydantic v1 (it's kinda possible with the v2, when using root_validator instead of validator)

Perhaps the best option would be to have the the MAX_SEQ_LEN load from an env variable (and default to 2048).

Happy to make a PR

If I don't have an aws region specified, the api throws an error. We should allow passing args to it.