Comments (7)
Great, thank you so much! I'll keep you posted depending on how far I get.
from imagebind.
Can I know how you run the model to generate images?
from imagebind.
^^^ upvoting WilTay1's question. also wondering if you know how to train ImageBind without using LoRA?
from imagebind.
^^^ upvoting WilTay1's question. also wondering if you know how to train ImageBind without using LoRA?
@ChloeL19 you can train the model without LoRA using ImageBind-LoRA. Simply remove the --lora argument when calling train.py and set --full_model_checkpointing. I don't have the resources to fine-tune it but it should work in theory. Try it out on the toy dataset provided in the repo, and later you can try to implement dataloaders for the original datasets
from imagebind.
Can I know how you run the model to generate images?
late, but there are many repositories that just replace CLIP for ImageBind
A good example is https://github.com/Birch-san/imagebind-guided-diffusion
from imagebind.
Nice work. Thanks
Could you a little explain why you used LoRA. Did the paper use it?
from imagebind.
There is a repo that uses LoRA to bind modalities to language.
from imagebind.
Related Issues (20)
- 多模态数据对
- `load_and_transform_text` method exec failed HOT 1
- Something wrong with EncodedVideo in load_and_transform_video_data HOT 2
- 预训练模型的输出问题
- Custom sensor as one of the multimodality? HOT 1
- Question regarding SelectElement(index=0) in the modality heads HOT 1
- Using Depth Embeddings in NyuV2 Zero-Shot Classification HOT 4
- Directly using images from S3 bucket using URL.
- Can Inference Time Be Improved by Using ONNX Model? HOT 1
- IMU inference
- Running into lots of dependency issues HOT 3
- Get the multimodal embeddings HOT 2
- Imagebind for commercial purposes
- Simply replacing Detic's CLIP-based ‘class’ enbedding with imagebind audio embedding HOT 1
- How to use ImageBind to locate sound sources in video?
- issue building wheel for cartopy (Windows 11) HOT 3
- 3 and more modalities in one model HOT 1
- What is your perspective on LanguageBind surpassing ImageBind? HOT 1
- Questions for demo sites audio and image data usage.
- Initialization of Thermal backbone
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from imagebind.