Comments (11)
Sorry to bother you, there are some other problems when trying to use the geochat_demo.py file:
even I try to use the example you give, it will only prompt "Error"
![image](https://private-user-images.githubusercontent.com/95007242/316397765-95d0fd44-a93f-441f-985f-8d4d6da8b1bf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjMyNjM3NzEsIm5iZiI6MTcyMzI2MzQ3MSwicGF0aCI6Ii85NTAwNzI0Mi8zMTYzOTc3NjUtOTVkMGZkNDQtYTkzZi00NDFmLTk4NWYtOGQ0ZDZkYThiMWJmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODEwVDA0MTc1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU5NzBmOWQ4MDQwMTg2ZDIxN2VjMDk0OWIzNmE2YjYyODA5ZWUyN2EyMjUzYWU3YzU0Zjg1NTE2Mzg3ZGIzNDgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.JIg5tmWtqFzZtTsgFCZ_OWcd7LPos579ScZrUIQlzLw)
In the terminal, the outputs are follows:
1.
Traceback (most recent call last):
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/routes.py", line 437, in run_predict
output = await app.get_blocks().process_api(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/blocks.py", line 1352, in process_api
result = await self.call_function(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/blocks.py", line 1093, in call_function
prediction = await utils.async_iteration(iterator)
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/utils.py", line 341, in async_iteration
return await iterator.anext()
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/utils.py", line 334, in anext
return await anyio.to_thread.run_sync(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/utils.py", line 317, in run_sync_iterator_async
return next(iterator)
File "geochat_demo.py", line 532, in gradio_stream_answer
streamer = chat.stream_answer(conv=chat_state,
File "/root/GeoChat/geochat/conversation.py", line 458, in stream_answer
generation_kwargs = self.answer_prepare(conv, img_list, **kargs)
File "/root/GeoChat/geochat/conversation.py", line 432, in answer_prepare
images=img_list[0],
IndexError: list index out of range
Traceback (most recent call last):
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/routes.py", line 437, in run_predict
output = await app.get_blocks().process_api(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/blocks.py", line 1352, in process_api
result = await self.call_function(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/blocks.py", line 1093, in call_function
prediction = await utils.async_iteration(iterator)
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/utils.py", line 341, in async_iteration
return await iterator.anext()
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/utils.py", line 334, in anext
return await anyio.to_thread.run_sync(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, args)
File "/root/miniconda3/envs/geochat/lib/python3.8/site-packages/gradio/utils.py", line 317, in run_sync_iterator_async
return next(iterator)
File "geochat_demo.py", line 531, in gradio_stream_answer
chat.encode_img(img_list)
File "/root/GeoChat/geochat/conversation.py", line 498, in encode_img
image = process_images_demo([raw_image], self.vis_processor )
File "/root/GeoChat/geochat/mm_utils.py", line 48, in process_images_demo
image = expand2square(image, tuple(int(x255) for x in image_processor.image_mean))
AttributeError: 'NoneType' object has no attribute 'image_mean'
from geochat.
Hi @KjAeRsTuIsK ,
I got the same error and the code is already is the latest. I have followed the instructions on ReadMe to set up environment and update the code.
Lokking forward to hearing from you soon!
from geochat.
Hi @xianjiedai
I got the same error as below. Could you let me know how you fixed?
'AttributeError: 'NoneType' object has no attribute 'image_mean'
By properly setting the model path. If you check builder.py, you will notice that
if 'geochat' in model_name.lower():
requires geochat in the model path and you may need to change the path or the name of the folder that contains model files cloned from hugging face.
If you can fix it and encounter the same problem as I do:
Hi @KjAeRsTuIsK ,
I have fixed the above problem but I got a new one with the example provided. Could you help me with this error? I used the example image and sentence as inputs.
please let me know
from geochat.
Hi @zhtstar, can you please update the code from the repo once and rerun it.
from geochat.
![image](https://private-user-images.githubusercontent.com/92470082/319249028-09afa2a8-feb9-4ff8-bc93-3f63e9f617ba.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjMyNjM3NzEsIm5iZiI6MTcyMzI2MzQ3MSwicGF0aCI6Ii85MjQ3MDA4Mi8zMTkyNDkwMjgtMDlhZmEyYTgtZmViOS00ZmY4LWJjOTMtM2Y2M2U5ZjYxN2JhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODEwVDA0MTc1MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTA3M2U5N2UzNjc2M2Q3ODVkM2I5NjI4Mzc4MjY4MTBmZTU0MjZiNjQ3OWYxOTBiYTU0YzlkYjcyMDQ1MjkxNTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.AEmC_74mT4OPduTNuSoYxKvT2enBFQJ_08vL3GjV_xs)
Hi @KjAeRsTuIsK ,
I have fixed the above problem but I got a new one with the example provided. Could you help me with this error? I used the example image and sentence as inputs.
from geochat.
Hi @xianjiedai
I got the same error as below. Could you let me know how you fixed?
'AttributeError: 'NoneType' object has no attribute 'image_mean'
from geochat.
@xianjiedai thank you!
from geochat.
Thank you for your excellent work. I encountered some problems when trying to use the geochat_demo.py file. Although I tried changing the conda environment many times and changing computers and servers, my problem still did not solve. I'm looking for your help here. My problem is:
- In the "Install" section you wrote "Clone this repository and navigate to LLaVA folder". What is "LLaVA folder"? Do I need to create a new folder myself? If so, what is the directory structure of this folder compared to other files?
- When I configure the environment according to your instructions and execute the code "python geochat_demo.py --model-path ./models", I will encounter the following error:
(geochat) root@Y9000K:/mnt/d/GeoChat# python geochat_demo.py --model-path ./models
/root/anaconda3/envs/geochat/lib/python3.8/site-packages/gradio_client/documentation.py:103: UserWarning: Could not get documentation group for <class 'gradio.mix.Parallel'>: No known documentation group for module 'gradio.mix'
warnings.warn(f"Could not get documentation group for {cls}: {exc}")
/root/anaconda3/envs/geochat/lib/python3.8/site-packages/gradio_client/documentation.py:103: UserWarning: Could not get documentation group for <class 'gradio.mix.Series'>: No known documentation group for module 'gradio.mix'
warnings.warn(f"Could not get documentation group for {cls}: {exc}")
Initializing Chat
Loading checkpoint shards: 50%|████████████████████████████████████████████████████ | 1/2 [02:06<02:06, 126.92s/it]
Traceback (most recent call last):
File "geochat_demo.py", line 53, in
tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, args.load_8bit, args.load_4bit, device=args.device)
File "/mnt/d/GeoChat/geochat/model/builder.py", line 125, in load_pretrained_model
model = AutoModelForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs)
File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained
return model_class.from_pretrained(
File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2903, in from_pretrained
) = cls._load_pretrained_model(
File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/modeling_utils.py", line 3260, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/modeling_utils.py", line 717, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/accelerate/utils/modeling.py", line 348, in set_module_tensor_to_device
raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([577, 1024]) in "weight" (which has shape torch.Size([1297, 1024])), this look incorrect.- If I do not use the model from GeoChat-7B, but directly use "facebook/opt-350m", the interface can be loaded, but nothing happens when clicking the "send" button
Sorry to bother you, I haven't been able to solve this problem even though I tried many times. I hope I can get your help.
请问你解决了吗?
from geochat.
Thank you for your excellent work. I encountered some problems when trying to use the geochat_demo.py file. Although I tried changing the conda environment many times and changing computers and servers, my problem still did not solve. I'm looking for your help here. My problem is:
1. In the "Install" section you wrote "Clone this repository and navigate to LLaVA folder". What is "LLaVA folder"? Do I need to create a new folder myself? If so, what is the directory structure of this folder compared to other files? 2. When I configure the environment according to your instructions and execute the code "python geochat_demo.py --model-path ./models", I will encounter the following error: (geochat) root@Y9000K:/mnt/d/GeoChat# python geochat_demo.py --model-path ./models /root/anaconda3/envs/geochat/lib/python3.8/site-packages/gradio_client/documentation.py:103: UserWarning: Could not get documentation group for <class 'gradio.mix.Parallel'>: No known documentation group for module 'gradio.mix' warnings.warn(f"Could not get documentation group for {cls}: {exc}") /root/anaconda3/envs/geochat/lib/python3.8/site-packages/gradio_client/documentation.py:103: UserWarning: Could not get documentation group for <class 'gradio.mix.Series'>: No known documentation group for module 'gradio.mix' warnings.warn(f"Could not get documentation group for {cls}: {exc}") Initializing Chat Loading checkpoint shards: 50%|████████████████████████████████████████████████████ | 1/2 [02:06<02:06, 126.92s/it] Traceback (most recent call last): File "geochat_demo.py", line 53, in tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, args.load_8bit, args.load_4bit, device=args.device) File "/mnt/d/GeoChat/geochat/model/builder.py", line 125, in load_pretrained_model model = AutoModelForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs) File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained return model_class.from_pretrained( File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2903, in from_pretrained ) = cls._load_pretrained_model( File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/modeling_utils.py", line 3260, in _load_pretrained_model new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model( File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/transformers/modeling_utils.py", line 717, in _load_state_dict_into_meta_model set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs) File "/root/anaconda3/envs/geochat/lib/python3.8/site-packages/accelerate/utils/modeling.py", line 348, in set_module_tensor_to_device raise ValueError( ValueError: Trying to set a tensor of shape torch.Size([577, 1024]) in "weight" (which has shape torch.Size([1297, 1024])), this look incorrect. 3. If I do not use the model from GeoChat-7B, but directly use "facebook/opt-350m", the interface can be loaded, but nothing happens when clicking the "send" button
Sorry to bother you, I haven't been able to solve this problem even though I tried many times. I hope I can get your help.
I also get the error ValueError: Trying to set a tensor of shape torch.Size([577, 1024]) in "weight" (which has shape torch.Size([1297, 1024])), this look incorrect.
Anyone know whats going on with this?
from geochat.
hi, i have renamed the folder containing model weights to contain geochat
and it solved the error 'AttributeError: 'NoneType' object has no attribute 'image_mean'
and the demo images run fine.
However, when i upload my own image, it does not appear. An Error appears in the demo UI too. The following error in terminal appears: AttributeError: 'NoneType' object has no attribute 'rsplit'
. Did anyone encounter this and how can it be resolved? Thanks!
Traceback (most recent call last):
File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/routes.py", line 437, in run_predict
output = await app.get_blocks().process_api(
File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/blocks.py", line 1350, in process_api
inputs = self.preprocess_data(fn_index, inputs, state)
File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/blocks.py", line 1200, in preprocess_data
processed_input.append(block.preprocess(inputs[i]))
File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/components/image.py", line 260, in preprocess
mask_im = processing_utils.decode_base64_to_image(mask)
File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/processing_utils.py", line 51, in decode_base64_to_image
image_encoded = extract_base64_data(encoding)
File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/processing_utils.py", line 42, in extract_base64_data
return x.rsplit(",", 1)[-1]
AttributeError: 'NoneType' object has no attribute 'rsplit'
from geochat.
hi, i have renamed the folder containing model weights to contain
geochat
and it solved the error'AttributeError: 'NoneType' object has no attribute 'image_mean'
and the demo images run fine.However, when i upload my own image, it does not appear. An Error appears in the demo UI too. The following error in terminal appears:
AttributeError: 'NoneType' object has no attribute 'rsplit'
. Did anyone encounter this and how can it be resolved? Thanks!Traceback (most recent call last): File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/routes.py", line 437, in run_predict output = await app.get_blocks().process_api( File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/blocks.py", line 1350, in process_api inputs = self.preprocess_data(fn_index, inputs, state) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/blocks.py", line 1200, in preprocess_data processed_input.append(block.preprocess(inputs[i])) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/components/image.py", line 260, in preprocess mask_im = processing_utils.decode_base64_to_image(mask) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/processing_utils.py", line 51, in decode_base64_to_image image_encoded = extract_base64_data(encoding) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/processing_utils.py", line 42, in extract_base64_data return x.rsplit(",", 1)[-1] AttributeError: 'NoneType' object has no attribute 'rsplit'
hi, i have renamed the folder containing model weights to contain
geochat
and it solved the error'AttributeError: 'NoneType' object has no attribute 'image_mean'
and the demo images run fine.However, when i upload my own image, it does not appear. An Error appears in the demo UI too. The following error in terminal appears:
AttributeError: 'NoneType' object has no attribute 'rsplit'
. Did anyone encounter this and how can it be resolved? Thanks!Traceback (most recent call last): File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/routes.py", line 437, in run_predict output = await app.get_blocks().process_api( File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/blocks.py", line 1350, in process_api inputs = self.preprocess_data(fn_index, inputs, state) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/blocks.py", line 1200, in preprocess_data processed_input.append(block.preprocess(inputs[i])) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/components/image.py", line 260, in preprocess mask_im = processing_utils.decode_base64_to_image(mask) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/processing_utils.py", line 51, in decode_base64_to_image image_encoded = extract_base64_data(encoding) File "/opt/conda/envs/geochat2/lib/python3.10/site-packages/gradio/processing_utils.py", line 42, in extract_base64_data return x.rsplit(",", 1)[-1] AttributeError: 'NoneType' object has no attribute 'rsplit'
i realised it can work for images of size 1024 x 768, but for other squarish images like 1185 x 1189, or 1169 x 1140, the error above keeps appearing. One current workaround is to resize the images.
from geochat.
Related Issues (20)
- How to calculate the metrics [email protected], [email protected], ROUGE and METEOR score in table 7, 8, 9? HOT 6
- get_chunk method in batch_geochat_scene.py seems to be undefined HOT 1
- Minimum memory for the training process
- how to run the lora finetuned model? HOT 5
- metrics about region captioning HOT 1
- training data corrupted HOT 1
- is training necessary ?
- Model for visual grounding
- Calculation of metrics
- Evaluation results about Grounding
- The results of MiniGPT in the paper HOT 2
- when training had an error!
- License for Commercial use
- merge lora
- how to finetune on my custom dataset
- training data corrupt
- Using transformers to use geochat directly
- The error encountered when using ZeRO-2 for training.
- Could you describe the procedure of reproduce the GeoChat?
- Multi images HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geochat.