Giter VIP home page Giter VIP logo

lora-instruct's Issues

Error message during training

Hi, I truncated "alpaca_data.json" to reduce training time and saved it in another file for training. Unfortunately, using this file gives me the error "pyarrow.lib.ArrowInvalid: JSON parse error: Column() changed from object to array in row 0".

MPTForCausalLM.forward() got an unexpected keyword argument 'inputs_embeds'

i 'm trying to finetune MPT-7b instruct, but getting this error.

File "/home/sulabh/new_gen_data/lora-instruct/finetune.py", line 383, in train
    trainer.train(resume_from_checkpoint=resume_from_checkpoint)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 1940, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 2735, in training_step
    loss = self.compute_loss(model, inputs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 2767, in compute_loss
    outputs = model(**inputs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/peft/peft_model.py", line 678, in forward
    return self.base_model(
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
TypeError: MPTForCausalLM.forward() got an unexpected keyword argument 'inputs_embeds'

Error message when training MPT-7B

Hi I got a message when I try to use Lora to train MPT-B, do you have any ideas to solve it?

ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (instruction in this case) have excessive nesting (inputs type list where type int is expected).

Falcon -7B training loss not reducing

Thanks for the wonderful code. While training Falcon-7B on Alpaca dataset, the training loss is not reducing. It used to work fine. Were there any recent changes?

Support for QLORA

Hello. Is there support for quantized lora finetuning of those llms?
Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.