[FEAT]: Support sotopia repo's code with our self-defined model checkpoint. about sotopia-pi HOT 3 CLOSED

This is what people use for deploying LLMs on babel (prev tir). https://github.com/ProKil/lti-llm-deployment

Known problems:

1. Not batching requests. So the speed is very, very slow.
2. The default client is not using aiohttp (non-blocking), but using requests (blocking)

from sotopia-pi.

I am trying to use vllm as an alternative backend. It supports an OpenAI-like API.

from sotopia-pi.

Done at the sotopia side.

from sotopia-pi.