[QST] Running cudf terribly slow about cudf HOT 8 OPEN

Hey @Hakan439, thanks for raising this issue! Could you tell me the output of running nvidia-smi in your terminal?

from cudf.

Hi,
here it is:

from cudf.

@Hakan439 Since you say the core runs on pycharm, can you confirm if terminal you are getting the error and pycharm are using the same environment?

You could run which python and share the output of terminal and pycharm.

from cudf.

Which Python?

cuda, cupy, cudf

gpu enabled

Sample code:

results:

from cudf.

Somehow now gpu seems to be enabled but it DOES run very slow. In the sample code above, I reduced the amount of rows in dataframe to 1000 for the test. in cpu dataframe it took 0.12 seconds however in gpu, in 8 seconds. mine running as as eGPU btw I no not know whether it makes a difference or not. What am I missing?

from cudf.

In the pycharm console output I see a different environment from everything else, gputest vs bbx_gpu_env. Not sure if that is significant or intentional. Good to know that it is running now, though. What do you mean by "eGPU"? Regarding performance, what does your data look like? How many columns does it have? Do you observe similar issues if you have a single column?

from cudf.

In the pycharm console output I see a different environment from everything else, gputest vs bbx_gpu_env. Not sure if that is significant or intentional. Good to know that it is running now, though. What do you mean by "eGPU"? Regarding performance, what does your data look like? How many columns does it have? Do you observe similar issues if you have a single column?

I tried with several virtual environments. The initial env was gputest. In mu single column benchmark, it rans slow. In my original python code, When I try to run it via cudf, it gives the error in the first message

from cudf.

I have a python code which calculates lots of numbers for varios custom dataclass objects

If cudf is working now but it is still slow, it is possible that your code is using custom dataclasses in a way that cudf simply doesn't support and so you end up falling back to running everything on the CPU. The relative slowdown you mentioned (0.12 vs 8 seconds) is pretty huge though. Have you tried running your code through the cudf.pandas profiler?

from cudf.