Fine-tune about lisa HOT 5 CLOSED

How about opening some layers or enabling the finetuning some parameters of the original LISA, do you think it would be doable? @X-Lai

from lisa.

There is only a single finetuning state. The reasoning segmentation dataset is directly involved with other datasets like the semantic / referring segmentation datasets.

from lisa.

There is only a single finetuning state. The reasoning segmentation dataset is directly involved with other datasets like the semantic / referring segmentation datasets.

emm,I want to employ the LISA to conduct the image-caption task, whose generated caption is more detailed than other model. But there is a domain difference between the train dataset and industrial dataset, so I wanna finetune LISA to apply to industrial domain.

from lisa.

That sounds great! Currently, we do not plan to release the code to further fine-tuning LISA. I think you could try to adopt the current hybrid-training paradigm and use your customized dataset together with other datasets.

You can add your customized data into the current training pipeline by simply configuring the dataset argument. As you want to use VQA data to conduct the image-caption task, I think you can directly replace the current llava-instruct-150k dataset with yours by simply configuring the vqa_data argument in train_ds.py.

from lisa.

That sounds great! Currently, we do not plan to release the code to further fine-tuning LISA. I think you could try to adopt the current hybrid-training paradigm and use your customized dataset together with other datasets.

You can add your customized data into the current training pipeline by simply configuring the dataset argument. As you want to use VQA data to conduct the image-caption task, I think you can directly replace the current llava-instruct-150k dataset with yours by simply configuring the vqa_data argument in train_ds.py.

Your model is so excellent! I will try your suggestion, thank you!

from lisa.