A toy experiment in using an LLM in Rust.
Uses LLM model llama-2-7b-chat.ggmlv3.q2_K.bin
found on huggingface.com by HuggingFace user, TheBloke.
Uses the latest main branch(commit id: 9376078c12ea1990bd42e63432656819a056d379
) of Rust's llm
GitHub repo as of 2023-12-19.
To build this, I needed to use an example provided in the repo, modifying the example provided by the llm
crate's official documentation on crates.io.