Error 101: hipErrorInvalidDevice (Triggered internally at ../c10/hip/HIPFunctions.cpp:113.) about rocm-gfx803 HOT 7 CLOSED

I have the same issue on my environment.

• Linux Mint 20.3 (Based on Ubuntu 20.04)
• RX580
• ROCm 5.2.0
  • sudo amdgpu-install --usecase=rocm
  • sudo dpkg -i rocblas_2.44.0.50200-65_amd64.deb
  • pip3 install torchvision-0.12.0a0+2662797-cp38-cp38-linux_x86_64.whl
  • pip3 install torch-1.11.0a0+git503a092-cp38-cp38-linux_x86_64.whl

Additionally, I had to install libopenmpi-dev, miopen-hip, libopenblas-dev, rocm-libs and rocm-dev to import pytorch.

How did you resolve the issue?

from rocm-gfx803.

@ligun
maybe pci atomic issue, you can run dmesg|grep kfd tko check whether card is added successful.

from rocm-gfx803.

@xuhuisheng
Thank you for the reply. I checked the log.
It seems to be added successfully.

$ dmesg|grep kfd
[    3.098492] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[    3.098647] kfd kfd: amdgpu: added device 1002:67df

from rocm-gfx803.

You can execute rocminfo to check if rocm can run properly.

Right now, I meet a problem that if I installed latest ROCm-5.2.3 dkms - driver, then I won't run rocminfo on gfx803. So I uninstall dkms, using upstream kernel builtin amdgpu driver, then I can run ROCm properly.

from rocm-gfx803.

rocminfo always returned error.

$ rocminfo 
ROCk module is loaded
hsa api call failure at: /long_pathname_so_that_rpms_can_package_the_debug_info/src/rocminfo/rocminfo.cc:1140
Call returned HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.

from rocm-gfx803.

Looks same to me.
My suggestion is uninstall amdgpu-dkms and amdgpu-dkms-firmware and reboot. This will fallback to upstream linux kernel builtin amdgpu-driver. Try rocminfo again, maybe pass.

from rocm-gfx803.

@xuhuisheng
Thank you very much for all your advice. Finally rocminfo ran successfully and torch.cuda.is_available() returned true!
I can use ROCm.

from rocm-gfx803.