https://huggingface.co/Salesforce/xgen-7b-8k-base

The link OpenLLaMA-7b-preview-300bt is now a 404 page.

I wasn't sure which of family of models released by TogetherCompute you'd like to add to the list, at least one of the RedPajama 3B/7B base models I guess. The 7B model is still training, but the 3B one is released. More info in their official release document.

I'll let you decide which one goes in the list. Thanks for making this repo.

• https://www.together.xyz/blog/redpajama-models-v1

Do feel free to close this ! 🙌🏽

Isn't OpenLLaMa supposed to be in this list ?

Thank you for your valuable contributions to the language model community. I'm curious if you would be interested in transforming this repository into an Awesome repository subsequently.

https://lmsys.org/blog/2023-03-30-vicuna/

We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.

I'm not an expert (or I would've sent a PR), but Vicuna is quite well-known?

https://github.com/InternLM/InternLM

Can OpenLlama model apply lora trained by llama?

Please add DLite to this list.

Name: DLite

Release Date: 2023/05

Blog post: https://medium.com/ai-squared/announcing-dlite-v2-lightweight-open-llms-that-can-run-anywhere-a852e5978c6e

Params (B): 0.124 - 1.5

Context Length: 1024

License: Apache 2.0

From the repository of StableLM (https://github.com/stability-AI/stableLM/):

Due to the original non-commercial license of LlaMA, we can only release the weights of our model as deltas over the original model's weights. StableVicuna's delta weights are released under (CC BY-NC-SA-4.0).

Which means that the effective model cannot be used with the here specified CC BY-SA-4.0 license, but effectively the GPLv3 license of the LlaMA model applies for the merged one.

As this repository's intention is to provide an overview about open and commercially usable models, I would either propose to remove the StableLM model (unfortunately) or to indicate the above described issue with a note in the table.

What do you think?

DeciCoder 1B just got released with Apache 2.0 license
Can you add it to your list?
https://huggingface.co/Deci/DeciCoder-1b

Firstly, thank you so much for putting this list together!

Is there a plan to add Elo ratings from https://lmsys.org/blog/2023-05-03-arena/ for each model in the table?

Hello,

Thanks for this beautiful resource.

Please add OpenHermes too.

https://huggingface.co/teknium/OpenHermes-7B
https://huggingface.co/teknium/OpenHermes-13B

Best,
Orkut

Please add OpenAlpaca (based on OpenLlama) and OpenChatKit (based on Pythia).

https://github.com/THUDM/ChatGLM-6B

Apache-2.0 license

how about the baichuan-7B?

MPT-7B-Instruct is CC-By-SA-3.0 not Apache as implied by the table.

At the moment, the first column in the Dataset for finetuning section is Name and includes the name of the corresponding model.

I find this very confusing as I would expect the first column to be the name of the dataset.

Therefore I would propose to either:

• Remove the current first column and content Name completely + instead move the Dataset column there.
• Rename the current first column to Trained Model and switch the positions with the Dataset column.

What do you think?

Now that Llama2 has been released, looks like an update is in order.

It might also be helpful to add the dates when the benchmark/test was performed.

• https://huggingface.co/mosaicml/mpt-30b
• https://huggingface.co/mosaicml/mpt-30b-instruct
• https://huggingface.co/mosaicml/mpt-30b-chat

This is a note to self, though if someone makes a PR I'll gladly merge it ;)

1:Redpajama incite now have fully trained 7B weight released.
2:StarCoder plus:https://huggingface.co/bigcode/starcoderplus
It was trained Futher on 600B additional data,Futher improving its performance.Also,I notice that it can also used as a chat bot now(thank to general English data from refined web),performing similar to llama 13b based models,and with 8192 ctx.
3:Tiger bot release it's giant 180B model.https://huggingface.co/TigerResearch/tigerbot-180b-research
Seems under Apache 2.0 license(no any other license or restriction mentioned on their repo,and every repo is shown Apache 2.0 license,but did not mention explicitly about their model license.) I hope it is truelly under Apache 2.0 license.
7B base and sft model available also.

this is the current link:

it should be https://arxiv.org/abs/2204.06745, which is:

Hi,

are there possibility to add a performance benchmark of the open-sourced LLMs?

baichuan-7B is under a coustom license,refer to https://huggingface.co/baichuan-inc/baichuan-7B/blob/main/baichuan-7B%20%E6%A8%A1%E5%9E%8B%E8%AE%B8%E5%8F%AF%E5%8D%8F%E8%AE%AE.pdf
This should be distinguished from most opensource license.

It would be very valuable, if the table included information about language of the model/dataset.
While obviously many people are interested in English, some of the models support/were trained on languages different than English.

I'm an advocate for minimalism and self-documenting code. I understand that the links are "just enough info" that there shouldn't be need for summaries.

However, I would appreciate if each model had a 1-line summary (or, if possible, multiple tiny lines explaining "the what and the why").

Not everyone has time to click every link and read the pages pointed by them.

I just came here and I see all of the models as "the same thing". I genuinely have no idea which ones are appropriate for my use-cases.

I'm willing to visit all URLs, to open a PR for this, but I don't have time this month.

BTW, thanks to all contributors for their time! And I'm sorry for my ignorance (if there's already a summary that I missed)

Summary

Currently there is the idea to add the following to the table:

• length of context window
• number of tokens trained

This issue is to discuss if we want to do so and what the implications are. I believe this is an important decision to make moving forward, so I would like to bring this to our attention here.

Implications

If we want to add these, we could have one separate row per published model version. Model version here indicates the standalone model variant published by the authors. This could either be due to different model sizes (see LlaMA-7B, 13B, 33B, 65B) or due to different training procedures (MPT-7B-base, vs -instruct, -chat, -storywriter). This will have an effect on the assigned properties in our table (model size, number of tokens trained, context window, ...)

In short, including more information inside the table would lead to:

• more columns, for more properties
• more rows, as we need to differentiate between each model version (alternatively, one could indicate the span for all models in one single row, e.g 1T - 10T for number of tokens or 1024 - 4096 for context width)

with the following consequences to the audience:

• more complete information, greater level of detail
• more difficult to get a quick overview, might damage the table clarity

What are your thoughts on this?