Comments (1)
Use PeftModel from pretrained seems to work
import torch
from transformers import AutoModel, AutoTokenizer, AutoConfig
from peft import PeftModel
# Load the base model and tokenizer
model_path = "/home/yerong2/models/internlm-xcomposer2d5-7b"
model = AutoModel.from_pretrained(model_path, trust_remote_code=True).half().cuda()
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
# Load the adapter/finetuned weights
checkpoint_path = "lorra_finetune/math/best"
# adapter_weights = torch.load(f"{checkpoint_path}/adapter_model.bin")
model = PeftModel.from_pretrained(model, checkpoint_path)
model = model.merge_and_unload()
# Verify that the model has loaded the weights
print("Model successfully loaded with finetuned weights.")
model.tokenizer = tokenizer
query = 'Image1 <ImageHere>; Image2 <ImageHere>; Image3 <ImageHere>; I want to buy a car from the three given cars, analyze their advantages and weaknesses one by one'
image = ['./examples/cars1.jpg',
'./examples/cars2.jpg',
'./examples/cars3.jpg',]
with torch.autocast(device_type='cuda', dtype=torch.float16):
response, his = model.chat(tokenizer, query, image, do_sample=False, num_beams=3, use_meta=True)
print(response)
from internlm-xcomposer.
Related Issues (20)
- web 模板咨询 HOT 2
- Inference time HOT 4
- out of memory when fine-tune the ICX2.5 HOT 4
- 图文写作生成结果相关问题 HOT 4
- emm 为啥你们的模型不能判断这个13.11和13.8谁大的问题,参数量不够吗 HOT 2
- Fine tuning of quanitized internlm/internlm-xcomposer2-4khd-7b model? HOT 3
- The feedforward dimension: BATCH_SIZE x k HOT 3
- nested BATCH_SIZE --per_device_train_batch_size should be always 1 HOT 2
- inference multi-image input using the 4-bit model HOT 3
- demo网页都打不开了 HOT 1
- Batches for text only branches is wrong too HOT 2
- [Bug] for internlm/internlm-xcomposer2d5-7b-4bit fine tune (ValueError: The model you want to train is loaded in 8-bit precision. if you want to fine-tune an 8-bit model, please make sure that you have installed `bitsandbytes>=0.37.0`) HOT 3
- 图文写作的实现逻辑 HOT 1
- 2d5 finetune type error HOT 11
- TypeError: 'NoneType' object is not callable HOT 2
- 多模态应用问题请教
- 图合成相关问题请教 HOT 1
- Generation: RuntimeError: FlashAttention only support fp16 and bf16 data type HOT 8
- Fine tuning saved checkpoint is too large HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from internlm-xcomposer.