[254-llm-chatbot] InferRequest.start_async() got an unexpected keyword argument 'shared_memory' about openvino_notebooks HOT 16 CLOSED

@hsulin0806 you may start by looking at https://github.com/openvinotoolkit/openvino.genai/tree/master/text_generation/causal_lm/cpp

Thank you!

from openvino_notebooks.

same

from openvino_notebooks.

Intel® Xeon(R) W-1290E CPU @ 3.50GHz × 20
Ubuntu 20.04.6 LTS
and
13th Gen Intel® Core™ i9-13900E × 32
Ubuntu 22.04.3 LTS
Both environments get the same result.

from openvino_notebooks.

Hi @hsulin0806 could you help to check the version of openvino in your system?
pip freeze | grep openvino

from openvino_notebooks.

你好@hsulin0806你可以幫忙檢查一下你係統中openvino的版本嗎？ pip freeze | grep openvino

Hi

Intel® Xeon(R) W-1290E CPU @ 3.50GHz × 20
Ubuntu 20.04.6 LTS

pip freeze | grep openvino

nncf @ git+https://github.com/openvinotoolkit/nncf@dc928a9e45b2dba3fd6cf8ad8c046b56d5da35f3
openvino-nightly==2024.0.0.dev20240103
openvino-telemetry==2023.2.1
Note: you may need to restart the kernel to use updated packages.

and

13th Gen Intel® Core™ i9-13900E × 32
Ubuntu 22.04.3 LTS

pip freeze | grep openvino

openvino==2023.2.0
openvino-dev==2023.2.0
openvino-nightly==2024.0.0.dev20240103
openvino-telemetry==2023.2.1
Note: you may need to restart the kernel to use updated packages.

from openvino_notebooks.

could you uninstall openvino and openvino-dev, then see if the error still exists ?

from openvino_notebooks.

您可以卸載openvino然後openvino-dev看看錯誤是否仍然存在？

An error will occur.

RuntimeError: Failed to import optimum.intel.openvino because of the following error (look up to see its traceback):
No module named 'openvino.runtime'

from openvino_notebooks.

您可以卸載openvino然後openvino-dev看看錯誤是否仍然存在？

An error will occur.

RuntimeError: Failed to import optimum.intel.openvino because of the following error (look up to see its traceback): No module named 'openvino.runtime'

Just reinstalling optimum-intel shall be ok

from openvino_notebooks.

您可以卸載openvino然後openvino-dev看看錯誤是否仍然存在？

An error will occur.
RuntimeError: Failed to import optimum.intel.openvino because of the following error (look up to see its traceback): No module named 'openvino.runtime'

Just reinstalling optimum-intel shall be ok

Again @@
TypeError: InferRequest.start_async() got an unexpected keyword argument 'shared_memory'

%pip freeze | grep openvino
nncf @ git+https://github.com/openvinotoolkit/nncf@6d6416d631497aaa2cf246c98ab31fb8c97d5847
openvino-nightly==2024.0.0.dev20240103
openvino-telemetry==2023.2.1
Note: you may need to restart the kernel to use updated packages.

from openvino_notebooks.

maybe you can try to install a lower version of openvino-nightly, .e.g. ```pip install openvino-nightly==2023.3.0.dev20231219

from openvino_notebooks.

maybe you can try to install a lower version of openvino-nightly, .e.g. ```pip install openvino-nightly==2023.3.0.dev20231219

Thank u
pip install openvino-nightly==2023.3.0.dev20231219 This is the key!

The internal graphics is working, but the dGPU still cannot execute.

I use Intel Arc-370M dGPU
Kernel Restarting appears.

from openvino_notebooks.

maybe you can try to install a lower version of openvino-nightly, .e.g. ```pip install openvino-nightly==2023.3.0.dev20231219

Thank u pip install openvino-nightly==2023.3.0.dev20231219 This is the key!

The internal graphics is working, but the dGPU still cannot execute.

I use Intel Arc-370M dGPU Kernel Restarting appears.

What is your model's size? a FP16/FP32 model may exceed the maximum dedicated memory on ARC-370m, I believed its 4gGB ?

from openvino_notebooks.

也許你可以嘗試安裝較低版本的 openvino-nightly，例如 ```pip install openvino-nightly==2023.3.0.dev20231219

謝謝 pip install openvino-nightly==2023.3.0.dev20231219 這才是關鍵！
內部顯示卡正在工作，但 dGPU 仍然無法執行。
我使用 Intel Arc-370M dGPU 出現核心重新啟動。

您的模特兒尺寸是多少？FP16/FP32 型號可能超過 ARC-370m 上的最大專用內存，我相信它是 4gGB ？

Yes 4GB 64 bit Memory

You are right!
I tested the small model tiny-llama-1b-chat and it works normally, thank you!

from openvino_notebooks.

Hi, as the owner of most topics related to OV Python, here is the answer to the question: openvinotoolkit/openvino#21979 (comment)

tl;dr you need to replace shared_memory with share_inputs. It works the same. I also encourage you to try out new share_outputs flag, especially with LLMs!

Cheers!

from openvino_notebooks.

Hi, as the owner of most topics related to OV Python, here is the answer to the question: openvinotoolkit/openvino#21979 (comment)

tl;dr you need to replace shared_memory with share_inputs. It works the same. I also encourage you to try out new share_outputs flag, especially with LLMs!

Cheers!

Thank you! I will try 'share_outputs'.
May I ask, does the '254-llm-chatbot' example provide a C++ inference program like the object recognition example?

from openvino_notebooks.

@hsulin0806 you may start by looking at https://github.com/openvinotoolkit/openvino.genai/tree/master/text_generation/causal_lm/cpp

from openvino_notebooks.