I'm running Starcoder2 locally with llama.cpp. I've set http://localhost:8080/completion
for the URL based on a little trial and error but that could be incorrect.
task 763 - error: Either "prompt" or "messages" must be provided
{"tid":"0x1f3a73ac0","timestamp":1711458389,"level":"ERR","function":"process_single_task","line":1517,"msg":"error while launching slot","inputs":" scope :sorted, -> { order(:cached_person) }\n","parameters":{"temperature":0.2,"max_new_tokens":256,"top_p":0.9,"repetition_penalty":1.2}}
{"tid":"0x1f3a73ac0","timestamp":1711458389,"level":"INFO","function":"update_slots","line":1672,"msg":"all slots are idle"}
{"tid":"0x16c0e3000","timestamp":1711458389,"level":"INFO","function":"log_server_request","line":2740,"msg":"request","remote_addr":"127.0.0.1","remote_port":54807,"status":400,"method":"POST","path":"/completion","params":{}}