load model with GPU error about openvino_notebooks HOT 9 CLOSED

@mingLZheng could you share:

1. which notebook sample you used?
2. which OS are you using?
3. what model that causing this error (is it a custom or modified model?)
4. did the issue happen on both CPU & GPU?
5. your OpenVINO version

from openvino_notebooks.

1.notebook is 254-rag-chatbot
2.window 11
3.chatglm3-6b
4.yes issue happen on CPU & GPU
5.openvino 2024.0.0

from openvino_notebooks.

chatglm3-6b model is custom

from openvino_notebooks.

@mingLZheng if possible could you

1. share that custom model for us to test?
2. elaborate further regarding the custom model, eg: static shaped/dynamic shaped, framework: Tensorflow, etc

from openvino_notebooks.

@Iffa-Intel however I use the official ChatGLM3-6B model. the same problem occurs.

from openvino_notebooks.

@mingLZheng I managed to convert and load that model:

ChatGLM3-6B model

Tested with INT4 precision

Selected INT4 for model to run section

Model loaded without any issue

from openvino_notebooks.

@Iffa-Intel It can work on CPU, but not on GPU.

from openvino_notebooks.

@mingLZheng I managed to run it on CPU and GPU

For GPU:

from openvino_notebooks.

Closing issue, feel free to re-open or start a new issue if additional assistance is needed.

from openvino_notebooks.