• RWKV model management and one-click startup
• Fully compatible with the OpenAI API, making every ChatGPT client an RWKV client. After starting the model, open http://127.0.0.1:8000/docs to view more details.
• Automatic dependency installation, requiring only a lightweight executable program
• Configs with 2G to 32G VRAM are included, works well on almost all computers
• User-friendly chat and completion interaction interface included
• Easy-to-understand and operate parameter configuration
• Built-in model conversion tool
• Built-in download management and remote model inspection
• Built-in one-click LoRA Finetune
• Can also be used as an OpenAI ChatGPT and GPT-Playground client
• Multilingual localization
• Theme switching
• Automatic updates

ab -p body.json -T application/json -c 20 -n 100 -l http://127.0.0.1:8000/chat/completions

body.json:

{
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ]
}

If you are using langchain, just use OpenAIEmbeddings(openai_api_base="http://127.0.0.1:8000", openai_api_key="sk-")

import numpy as np
import requests


def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))


values = [
    "I am a girl",
    "我是个女孩",
    "私は女の子です",
    "广东人爱吃福建人",
    "我是个人类",
    "I am a human",
    "that dog is so cute",
    "私はねこむすめです、にゃん♪",
    "宇宙级特大事件！号外号外！"
]

embeddings = []
for v in values:
    r = requests.post("http://127.0.0.1:8000/embeddings", json={"input": v})
    embedding = r.json()["data"][0]["embedding"]
    embeddings.append(embedding)

compared_embedding = embeddings[0]

embeddings_cos_sim = [cosine_similarity(compared_embedding, e) for e in embeddings]

for i in np.argsort(embeddings_cos_sim)[::-1]:
    print(f"{embeddings_cos_sim[i]:.10f} - {values[i]}")

• RWKV-4-World: https://huggingface.co/BlinkDL/rwkv-4-world/tree/main
• RWKV-4-Raven: https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main
• ChatRWKV: https://github.com/BlinkDL/ChatRWKV
• RWKV-LM: https://github.com/BlinkDL/RWKV-LM
• RWKV-LM-LoRA: https://github.com/Blealtan/RWKV-LM-LoRA