Comments (3)
There might be a way to write a little numba extension code within cudf.pandas
that registers cudf.pandas._wrappers.numpy.ndarray
objects as something numba can unbox into a numpy array or cupy array. If that worked we could probably do the registration at import time. I'll investigate.
from cudf.
Thanks for the report. As your post highlights it looks like the core issue is that cudf.pandas
wraps numpy arrays (to use cupy if possible) and this wrapped array is not compatible with numba
In [1]: import cudf.pandas
...: cudf.pandas.install()
...:
...: import pandas as pd
i
In [2]: import numba
In [3]: @numba.jit(nopython=True, nogil=True)
...: def f(x):
...: return x
...:
In [4]: f(pd.Series([1]).values)
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
Cell In[4], line 1
----> 1 f(pd.Series([1]).values)
File ~/miniforge3/envs/cudf-dev/lib/python3.11/site-packages/numba/core/dispatcher.py:468, in _DispatcherBase._compile_for_args(self, *args, **kws)
464 msg = (f"{str(e).rstrip()} \n\nThis error may have been caused "
465 f"by the following argument(s):\n{args_str}\n")
466 e.patch_message(msg)
--> 468 error_rewrite(e, 'typing')
469 except errors.UnsupportedError as e:
470 # Something unsupported is present in the user code, add help info
471 error_rewrite(e, 'unsupported_error')
File ~/miniforge3/envs/cudf-dev/lib/python3.11/site-packages/numba/core/dispatcher.py:409, in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
407 raise e
408 else:
--> 409 raise e.with_traceback(None)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
During: typing of argument at <ipython-input-3-88a5a2446c8f> (1)
File "<ipython-input-3-88a5a2446c8f>", line 1:
@numba.jit(nopython=True, nogil=True)
^
This error may have been caused by the following argument(s):
- argument 0: Cannot determine Numba type of <class 'cudf.pandas._wrappers.numpy.ndarray'>
Going to repurpose this issue to be about compatibility with numba.
from cudf.
@brandon-b-miller when you have time can you also take a look at how cudf.pandas and numba are interoperating ?
from cudf.
Related Issues (20)
- [BUG] `loc` returning incorrect results for `DatetimeIndex` that is in monotonically decreasing
- [BUG] chunked parquet reader is not factoring empty dataframes with `>0` columns present HOT 2
- [FEA] Make line terminator sequence handling in regular expression engine a configurable option HOT 1
- [BUG] cudf.pandas dataframe.__repr__ slow in jupyterlab for large datasets HOT 1
- [BUG] iloc/loc keeps circular reference to original DataFrame/Series
- [BUG] double free or memory corruption when parsing some JSON HOT 6
- [BUG] Stop allowing floating arrow along minor versions
- [BUG] when using id_vars in `.melt()` , the string of the column name is broken into characters HOT 2
- [BUG] Data corruption and strange CUDA memory address errors at the same row index, despite manipulating data, when using `.stack()` on large, wide dataset
- [FEA] explore using KMP for string matching like operations HOT 1
- [DOC] cudf.Series.to_list is empty
- [BUG] dask-cudf isin errors when passing in a list of values HOT 1
- [FEA] Avoid materializing temporary table in ORC chunked reader
- [FEA] Potential optimization: Batched memset.
- [FEA] Support binary operations between timezone-aware datetime columns
- [BUG] File written by ORC writer cannot be read in Pandas and Spark
- [BUG] `Index.is_monotonic_*` methods not factoring `nan` values
- [BUG] cudf.Series.duplicated returns error 'Series' object has no attribute 'duplicated' HOT 5
- [BUG] Calling `cat.as_ordered` does not work on a sliced column
- [BUG] `uses_custom_row_groups` should not be hardcoded to true in `chunked_parquet_reader`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cudf.