chrisociepa / allamo Goto Github PK

I've trained a small model for hungarian language for 5 days, text generation is working well. Is it possible to use this model for text classification too? (after finetraining on 100k text+class pairs dataset somehow)
Where/how can be the classification hidden layer added/connected to this model?

line 137:
import_tokenizer(args.input_tokenizer_path, args.output_data_dir)
should be:
import_tokenizer(args.input_tokenizer_path, args.output_data_dir, args.max_block_size)

hello, thanks for the great work

i know that sample.py, sample_api.py exist but i just want to use the model in a standalone python script

because of the way AllamoConfiguration.__post_init__() is defined, i cannot create a new instance AllamoConfiguration to use it

is there any way to do it properly without touching the source code?

many thanks

Like example datasets and configs.

Could you please give more details on how to finetune llama on my own QA data?

Hey @chrisociepa, awesome repo! Could you pls shed some light on how the train.txt file should look like?

Thank you!

It would be valuable to provide at least one sample config file...

Much larger universe of people who can use truly open Llama variants such as the two models above. Limiting the code to Facebook-approved academics severely limits its value and reach.