Giter VIP home page Giter VIP logo

Comments (16)

rnwang04 avatar rnwang04 commented on September 25, 2024 1

Hi @JJJohnathan
We have not verified on kernel 6.7.1.
Below is a sample output on our Linux MTL:

(base) arda@xiaoxin04-ubuntu:~/ruonan/ipex-llm/python/llm/scripts$ bash env-check.sh
-----------------------------------------------------------------
PYTHON_VERSION=3.10.14
-----------------------------------------------------------------
transformers=4.36.2
-----------------------------------------------------------------
torch=2.1.0a0+cxx11.abi
-----------------------------------------------------------------
ipex-llm Version: 2.1.0b20240611
-----------------------------------------------------------------
ipex=2.1.10+xpu
-----------------------------------------------------------------
CPU Information: 
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      46 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             18
On-line CPU(s) list:                0-17
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Core(TM) Ultra 5 125H
CPU family:                         6
Model:                              170
Thread(s) per core:                 2
Core(s) per socket:                 14
Socket(s):                          1
Stepping:                           4
CPU max MHz:                        4500.0000
CPU min MHz:                        400.0000
BogoMIPS:                           5990.40
-----------------------------------------------------------------
Total CPU Memory: 30.9502 GB
-----------------------------------------------------------------
Operating System: 
Ubuntu 22.04.3 LTS \n \l

-----------------------------------------------------------------
Linux xiaoxin04-ubuntu 6.5.0-35-generic #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May  7 09:00:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
-----------------------------------------------------------------
CLI:
    Version: 1.2.22.20231126
    Build ID: 00000000

Service:
    Version: 1.2.22.20231126
    Build ID: 00000000
    Level Zero Version: 1.14.0
-----------------------------------------------------------------
  Driver Version                                  2023.16.12.0.12_195853.xmain-hotfix
  Driver Version                                  2023.16.12.0.12_195853.xmain-hotfix
  Driver Version                                  2024.17.3.0.08_160000
  Driver UUID                                     32342e30-392e-3238-3731-372e31320000
  Driver Version                                  24.09.28717.12
  Driver Version                                  2024.17.3.0.08_160000
-----------------------------------------------------------------
Driver related package version:
ii  intel-fw-gpu                                   2023.39.2-255~22.04                     all          Firmware package for Intel integrated and discrete GPUs
ii  intel-level-zero-gpu                           1.3.28717.12                            amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero.
ii  level-zero-dev                                 1.14.0-744~22.04                        amd64        Intel(R) Graphics Compute Runtime for oneAPI Level Zero.
-----------------------------------------------------------------
igpu detected
[opencl:gpu:3] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) Graphics OpenCL 3.0 NEO  [24.09.28717.12]
[ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Arc(TM) Graphics 1.3 [1.3.28717]
-----------------------------------------------------------------
xpu-smi is properly installed. 
-----------------------------------------------------------------
+-----------+--------------------------------------------------------------------------------------+
| Device ID | Device Information                                                                   |
+-----------+--------------------------------------------------------------------------------------+
| 0         | Device Name: Intel(R) Arc(TM) Graphics                                               |
|           | Vendor Name: Intel(R) Corporation                                                    |
|           | SOC UUID: 00000000-0000-0200-0000-00087d558086                                       |
|           | PCI BDF Address: 0000:00:02.0                                                        |
|           | DRM Device: /dev/dri/card0                                                           |
|           | Function Type: physical                                                              |
+-----------+--------------------------------------------------------------------------------------+
GPU0 Memory ize=256M
-----------------------------------------------------------------
00:02.0 VGA compatible controller: Intel Corporation Device 7d55 (rev 08) (prog-if 00 [VGA controller])
        Subsystem: Lenovo Device 3cc9
        Flags: bus master, fast devsel, latency 0, IRQ 184, IOMMU group 0
        Memory at 408c000000 (64-bit, prefetchable) [size=16M]
        Memory at 4000000000 (64-bit, prefetchable) [size=256M]
        Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
        Capabilities: <access denied>
        Kernel driver in use: i915
        Kernel modules: i915
-----------------------------------------------------------------
(base) arda@xiaoxin04-ubuntu:~/ruonan/ipex-llm/python/llm/scripts$ sycl-ls
[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2023.16.12.0.12_195853.xmain-hotfix]
[opencl:cpu:1] Intel(R) OpenCL, Intel(R) Core(TM) Ultra 5 125H OpenCL 3.0 (Build 0) [2023.16.12.0.12_195853.xmain-hotfix]
[opencl:cpu:2] Intel(R) OpenCL, Intel(R) Core(TM) Ultra 5 125H OpenCL 3.0 (Build 0) [2024.17.3.0.08_160000]
[opencl:gpu:3] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) Graphics OpenCL 3.0 NEO  [24.09.28717.12]
[opencl:acc:4] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2024.17.3.0.08_160000]
[ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Arc(TM) Graphics 1.3 [1.3.28717]

Maybe you can try with kernel 6.5 following https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html#for-linux-kernel-6-5

from bigdl.

rnwang04 avatar rnwang04 commented on September 25, 2024

Hi @OvaltineSamuel ,
A normal output log on MTL look likes:

llm_load_print_meta: BOS token        = 151643 '<|endoftext|>'
llm_load_print_meta: EOS token        = 151645 '<|im_end|>'
llm_load_print_meta: PAD token        = 151643 '<|endoftext|>'
llm_load_print_meta: LF token         = 148848 'ÄĬ'
llm_load_print_meta: EOT token        = 151645 '<|im_end|>'
[SYCL] call ggml_init_sycl
ggml_init_sycl: GGML_SYCL_DEBUG: 0
ggml_init_sycl: GGML_SYCL_F16: no
found 4 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|                     Intel Arc Graphics|    1.3|    112|    1024|   32| 15482M|            1.3.29283|
| 1|     [opencl:gpu:0]|                     Intel Arc Graphics|    3.0|    112|    1024|   32| 15482M|        31.0.101.5534|
| 2|     [opencl:cpu:0]|                Intel Core Ultra 5 125H|    3.0|     18|    8192|   64| 33945M|2023.16.12.0.12_195853.xmain-hotfix|
| 3|     [opencl:acc:0]|            Intel FPGA Emulation Device|    1.2|     18|67108864|   64| 33945M|2023.16.12.0.12_195853.xmain-hotfix|
ggml_backend_sycl_set_mul_device_mode: true
detect 1 SYCL GPUs: [0] with top Max compute units:112
llm_load_tensors: ggml ctx size =    0.37 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU

It seems your program can't find sycl device and then raise this error.
Could you please let us know your iGPU driver version ?

from bigdl.

OvaltineSamuel avatar OvaltineSamuel commented on September 25, 2024

Yes, I'm using the latest Driver for MTL iGPU

image

from bigdl.

jason-dai avatar jason-dai commented on September 25, 2024

Please run the ENV-Check script in https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/scripts

from bigdl.

rnwang04 avatar rnwang04 commented on September 25, 2024

Hi @OvaltineSamuel
I have verified that on our local MTL, this driver version works:
image

Could you please provide us more env details with https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/scripts and also show us your output of ls-sycl-device ?

from bigdl.

OvaltineSamuel avatar OvaltineSamuel commented on September 25, 2024

So do I need to install OneAPI basetoolkit for using ipex-llm in this case?

from bigdl.

rnwang04 avatar rnwang04 commented on September 25, 2024

So do I need to install OneAPI basetoolkit for using ipex-llm in this case?

You don't need to install OneAPI basetoolkit by yourself.
When you pip install --pre --upgrade ipex-llm[cpp], oneapi 2024.0 has already been installed in your llm-cpp env, please make sure always run your program in llm-cpp env.

from bigdl.

JJJohnathan avatar JJJohnathan commented on September 25, 2024

And I encounter problem like this too, when I follow the guide and run sycl-ls here https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html
image
the result of running ls-sycl-device is :"ls-sycl-device: command not found"
the results of running " ENV-Check script" are as follow
bash env-check.sh

PYTHON_VERSION=3.11.9

transformers=4.36.2

torch=2.1.0a0+cxx11.abi

ipex-llm Version: 2.1.0b20240611

IPEX is not installed.

CPU Information:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 22
On-line CPU(s) list: 0-21
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) Ultra 9 185H
CPU family: 6
Model: 170
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
Stepping: 4
CPU max MHz: 5000.0000
CPU min MHz: 400.0000
BogoMIPS: 6144.00

Total CPU Memory: 30.8876 GB
Memory Type: LPDDR5

Operating System:
Ubuntu 22.04.4 LTS \n \l

Linux odt-huyuan-penvino-ci-77 6.7.1-060701-generic #202401201133 SMP PREEMPT_DYNAMIC Sat Jan 20 11:43:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

CLI:
Version: 1.2.35.20240425
Build ID: 00000000

Service:
Version: 1.2.35.20240425
Build ID: 00000000
Level Zero Version: 1

Driver Version 2023.16.12.0.12_195853.xmain-hotfix
Driver Version 2023.16.12.0.12_195853.xmain-hotfix
Driver UUID 32342e30-392e-3238-3731-372e31320000
Driver Version 24.09.28717.12

Driver related package version:
ii intel-fw-gpu 2024.17.5-329~22.04 all Firmware package for Intel integrated and discrete GPUs
ii intel-i915-dkms 1.24.2.17.240301.20+i29-1 all Out of tree i915 driver.
ii intel-level-zero-gpu 1.3.28717.12 amd64 Intel(R) Graphics Compute Runtime for oneAPI Level Zero.

SYCL Exception encountered: Native API failed. Native API returns: -30 (PI_ERROR_INVALID_VALUE) -30 (PI_ERROR_INVALID_VALUE)

igpu not detected

xpu-smi is properly installed.

+-----------+--------------------------------------------------------------------------------------+
| Device ID | Device Information |
+-----------+--------------------------------------------------------------------------------------+
| 0 | Device Name: Intel(R) Arc(TM) Graphics |
| | Vendor Name: Intel(R) Corporation |
| | SOC UUID: 00000000-0000-0200-0000-00087d558086 |
| | PCI BDF Address: 0000:00:02.0 |
| | DRM Device: /dev/dri/card0 |
| | Function Type: physical |
+-----------+--------------------------------------------------------------------------------------+
GPU0 Memory size=16M

00:02.0 VGA compatible controller: Intel Corporation Device 7d55 (rev 08) (prog-if 00 [VGA controller])
DeviceName: VGA
Subsystem: ASUSTeK Computer Inc. Device 1a63
Flags: bus master, fast devsel, latency 0, IRQ 180
Memory at 5010000000 (64-bit, prefetchable) [size=16M]
Memory at 4000000000 (64-bit, prefetchable) [size=256M]
Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
Capabilities:
Kernel driver in use: i915
Kernel modules: i915

from bigdl.

JJJohnathan avatar JJJohnathan commented on September 25, 2024

@rnwang04 I see. Anyway thx!

from bigdl.

OvaltineSamuel avatar OvaltineSamuel commented on September 25, 2024

Please run the ENV-Check script in https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/scripts

Below is the output of env-check script

Python 3.11.9
-----------------------------------------------------------------
transformers=4.41.2
-----------------------------------------------------------------
torch=2.2.0+cpu
-----------------------------------------------------------------
Name: ipex-llm
Version: 2.1.0b20240607
Summary: Large Language Model Develop Toolkit
Home-page: https://github.com/intel-analytics/ipex-llm
Author: BigDL Authors
Author-email: [email protected]
License: Apache License, Version 2.0
Location: C:\Users\Ovalt\miniforge3\envs\ipex-llm_ollama_v2\Lib\site-packages
Requires:
Required-by:
-----------------------------------------------------------------
IPEX is not installed properly.
-----------------------------------------------------------------
Total Memory: 15.376 GB

Chip 0 Memory: 16 GB | Speed: 5600 MHz
-----------------------------------------------------------------
CPU Manufacturer: GenuineIntel
CPU MaxClockSpeed: 3800
CPU Name: Intel(R) Core(TM) Ultra 7 155H
CPU NumberOfCores: 16
CPU NumberOfLogicalProcessors: 22
-----------------------------------------------------------------
GPU 0: Intel(R) Graphics         Driver Version:  31.0.101.5522
-----------------------------------------------------------------
-----------------------------------------------------------------
System Information

Host Name:                 EXPERTBOOK-B5
OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22631 N/A Build 22631
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Organization:   N/A
Product ID:                00355-61488-69042-AAOEM
Original Install Date:     4/30/2024, 6:54:59 AM
System Boot Time:          6/8/2024, 4:19:15 PM
System Manufacturer:       ASUSTeK COMPUTER INC.
System Model:              ASUS EXPERTBOOK B5404CMA_B5404CMA
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 170 Stepping 4 GenuineIntel ~2280 Mhz
BIOS Version:              ASUSTeK COMPUTER INC. (Licensed by AMI, LLC.) B5404CMA.208, 2/7/2024
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC+08:00) Taipei
Total Physical Memory:     15,745 MB
Available Physical Memory: 6,476 MB
Virtual Memory: Max Size:  42,369 MB
Virtual Memory: Available: 29,067 MB
Virtual Memory: In Use:    13,302 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              \\EXPERTBOOK-B5
Hotfix(s):                 5 Hotfix(s) Installed.
                           [01]: KB5037591
                           [02]: KB5027397
                           [03]: KB5036212
                           [04]: KB5037853
                           [05]: KB5037959
Network Card(s):           3 NIC(s) Installed.
                           [01]: Intel(R) Wi-Fi 6E AX211 160MHz
                                 Connection Name: Wi-Fi
                                 DHCP Enabled:    Yes
                                 DHCP Server:     1.1.1.1
                                 IP address(es)
                                 [01]: 10.174.192.173
                                 [02]: fe80::bbb6:b406:183a:7a63
                           [02]: Intel(R) Ethernet Connection (18) I219-V
                                 Connection Name: Ethernet
                                 Status:          Media disconnected
                           [03]: Bluetooth Device (Personal Area Network)
                                 Connection Name: Bluetooth Network Connection
                                 Status:          Media disconnected
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.
-----------------------------------------------------------------
'xpu-smi' is not recognized as an internal or external command,
operable program or batch file.
xpu-smi is not installed properly.

It says IPEX is not installed properly. Also, I can't run ls-sycl-device without having OneAPI basetoolkit installed.

from bigdl.

OvaltineSamuel avatar OvaltineSamuel commented on September 25, 2024

I tried restarting the laptop and reinstalling the whole conda environment again. Run the same env-check.bat with the same output of "IPEX is not installed properly".

from bigdl.

rnwang04 avatar rnwang04 commented on September 25, 2024

Here is a sample output of our Windows MTL:

(ruonan-cpp) D:\ruonan\ipex-llm\python\llm\scripts>env-check.bat
Python 3.11.9
-----------------------------------------------------------------
transformers=4.41.1
-----------------------------------------------------------------
torch=2.2.0+cpu
-----------------------------------------------------------------
Name: ipex-llm
Version: 2.1.0b20240610
Summary: Large Language Model Develop Toolkit
Home-page: https://github.com/intel-analytics/ipex-llm
Author: BigDL Authors
Author-email: [email protected]
License: Apache License, Version 2.0
Location: C:\Users\arda\miniforge3\envs\ruonan-cpp\Lib\site-packages
Requires:
Required-by:
-----------------------------------------------------------------
IPEX is not installed properly.
-----------------------------------------------------------------
Total Memory: 31.615 GB

Chip 0 Memory: 4 GB | Speed: 7467 MHz
Chip 1 Memory: 4 GB | Speed: 7467 MHz
Chip 2 Memory: 4 GB | Speed: 7467 MHz
Chip 3 Memory: 4 GB | Speed: 7467 MHz
Chip 4 Memory: 4 GB | Speed: 7467 MHz
Chip 5 Memory: 4 GB | Speed: 7467 MHz
Chip 6 Memory: 4 GB | Speed: 7467 MHz
Chip 7 Memory: 4 GB | Speed: 7467 MHz
-----------------------------------------------------------------
CPU Manufacturer: GenuineIntel
CPU MaxClockSpeed: 3600
CPU Name: Intel(R) Core(TM) Ultra 5 125H
CPU NumberOfCores: 14
CPU NumberOfLogicalProcessors: 18
-----------------------------------------------------------------
GPU 0: Intel(R) Arc(TM) Graphics         Driver Version:  31.0.101.5534
-----------------------------------------------------------------
-----------------------------------------------------------------
System Information

主机名:           XIAOXIN02
OS 名称:          Microsoft Windows 11 家庭中文版
OS 版本:          10.0.22631 暂缺 Build 22631
OS 制造商:        Microsoft Corporation
OS 配置:          独立工作站
OS 构建类型:      Multiprocessor Free
注册的所有人:     arda
注册的组织:       暂缺
产品 ID:          00342-31548-11544-AAOEM
初始安装日期:     2023/12/25, 13:28:06
系统启动时间:     2024/6/12, 14:22:58
系统制造商:       LENOVO
系统型号:         83D4
系统类型:         x64-based PC
处理器:           安装了 1 个处理器。
                  [01]: Intel64 Family 6 Model 170 Stepping 4 GenuineIntel ~1200 Mhz
BIOS 版本:        LENOVO MECN40WW, 2023/10/25
Windows 目录:     C:\Windows
系统目录:         C:\Windows\system32
启动设备:         \Device\HarddiskVolume1
系统区域设置:     zh-cn;中文(**)
输入法区域设置:   en-us;英语(美国)
时区:             (UTC+08:00) 北京，重庆，香港特别行政区，乌鲁木齐
物理内存总量:     32,373 MB
可用的物理内存:   24,125 MB
虚拟内存: 最大值: 43,939 MB
虚拟内存: 可用:   34,775 MB
虚拟内存: 使用中: 9,164 MB
页面文件位置:     C:\pagefile.sys
域:               WORKGROUP
登录服务器:       \\XIAOXIN02
修补程序:         安装了 6 个修补程序。
                  [01]: KB5037591
                  [02]: KB5027397
                  [03]: KB5031274
                  [04]: KB5033055
                  [05]: KB5037771
                  [06]: KB5037663
网卡:             安装了 3 个 NIC。
                  [01]: Intel(R) Wi-Fi 6E AX211 160MHz
                      连接名:      WLAN
                      状态:        媒体连接已中断
                  [02]: Bluetooth Device (Personal Area Network)
                      连接名:      蓝牙网络连接
                      状态:        媒体连接已中断
                  [03]: ASIX USB to Gigabit Ethernet Family Adapter
                      连接名:      以太网 3
                      启用 DHCP:   是
                      DHCP 服务器: 10.239.27.228
                      IP 地址
                        [01]: 10.239.158.142
                        [02]: fe80::4481:7641:add8:cc3
Hyper-V 要求:     虚拟机监视器模式扩展: 是
                  固件中已启用虚拟化: 是
                  二级地址转换: 是
                  数据执行保护可用: 是
-----------------------------------------------------------------
'xpu-smi' 不是内部或外部命令，也不是可运行的程序
或批处理文件。
xpu-smi is not installed properly.

You can ignore "IPEX is not installed properly" as ipex is not needed for running ipex-llm[cpp].

And ls-sycl-device doesn't need OneAPI toolkit, it's provided by init-llama-cpp.bat, you can just go to your cpp directory and run this ls-sycl-device.exe .

from bigdl.

OvaltineSamuel avatar OvaltineSamuel commented on September 25, 2024

@rnwang04 Thanks for your clarification on that. However, I'm not getting anything when running ls-sycl-device.exe on my end.

image

from bigdl.

rnwang04 avatar rnwang04 commented on September 25, 2024

Hi @OvaltineSamuel
On our Windows MTL machine, the output of ls-sycl-device.exe looks like:

(ruonan-cpp) D:\ruonan\bmk-llama-cpp>ls-sycl-device
found 4 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|                     Intel Arc Graphics|    1.3|    112|    1024|   32| 15482M|            1.3.29283|
| 1|     [opencl:gpu:0]|                     Intel Arc Graphics|    3.0|    112|    1024|   32| 15482M|        31.0.101.5534|
| 2|     [opencl:cpu:0]|                Intel Core Ultra 5 125H|    3.0|     18|    8192|   64| 33945M|2023.16.12.0.12_195853.xmain-hotfix|
| 3|     [opencl:acc:0]|            Intel FPGA Emulation Device|    1.2|     18|67108864|   64| 33945M|2023.16.12.0.12_195853.xmain-hotfix|

and my pip list is :

(ruonan-cpp) D:\ruonan\bmk-llama-cpp>pip list
Package                 Version
----------------------- --------------
accelerate              0.21.0
bigdl-core-cpp          2.5.0b20240610
certifi                 2024.2.2
charset-normalizer      3.3.2
colorama                0.4.6
dpcpp-cpp-rt            2024.0.2
filelock                3.14.0
fsspec                  2024.5.0
gguf                    0.6.0
huggingface-hub         0.23.1
idna                    3.7
intel-cmplr-lib-rt      2024.0.2
intel-cmplr-lic-rt      2024.0.2
intel-opencl-rt         2024.0.2
intel-openmp            2024.0.2
ipex-llm                2.1.0b20240610
Jinja2                  3.1.4
MarkupSafe              2.1.5
mkl                     2024.0.0
mkl-dpcpp               2024.0.0
mpmath                  1.3.0
networkx                3.3
numpy                   1.26.4
onednn                  2024.0.0
onemkl-sycl-blas        2024.0.0
onemkl-sycl-datafitting 2024.0.0
onemkl-sycl-dft         2024.0.0
onemkl-sycl-lapack      2024.0.0
onemkl-sycl-rng         2024.0.0
onemkl-sycl-sparse      2024.0.0
onemkl-sycl-stats       2024.0.0
onemkl-sycl-vm          2024.0.0
packaging               24.0
pip                     24.0
protobuf                4.25.3
psutil                  5.9.8
PyYAML                  6.0.1
regex                   2024.5.15
requests                2.32.2
safetensors             0.4.3
sentencepiece           0.1.99
setuptools              70.0.0
sympy                   1.12.1rc1
tbb                     2021.12.0
tokenizers              0.19.1
torch                   2.2.0
tqdm                    4.66.4
transformers            4.41.1
typing_extensions       4.12.0
urllib3                 2.2.1
wheel                   0.43.0

Actually your issue is irrelevant with llama.cpp, just your machine can't find sycl device.
Sadly, we have not meet this issue before and we can't reproduce this issue...
Just a suggestion, maybe you can try to install latest driver 5534 (https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html) ?

from bigdl.

OvaltineSamuel avatar OvaltineSamuel commented on September 25, 2024

Got it, will try it out later and let you know if it works. Thanks a lot.

from bigdl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.