Comments (8)
@ivirshup I have a PR (scverse/rapids_singlecell#60) ready to go once the anndata 0.10.0 comes out. The only things I'll have to do is rerun the notebooks with benchmarks and depreciate cunndata and change the docs with the usage principle. So I think I can have the updated out at most 2 days after you publish.
Edit:
I also updated the tests to work with anndata. Thats why I didn't merge yet.
from anndata.
You can point to the documentation notebooks in https://rapids-singlecell.readthedocs.io/en/latest/notebooks.html
The notebooks will work with anndata once 0.10.0 is released
from anndata.
Responding to:
So I have some more small Ideas that I think would be good. I can also start implementing some of them:
- Check if .nnz for cpx is less than 2^31-1 since cupy only supports int32 indptr
- make a .todevice() like torch. To transform X or .layers from and to GPU. Maybe add an
all
option to dump everything into > RAM.- add a flag/property for .X and layers if its in RAM or VRAM
Originally posted by @Intron7 in #1066 (comment)
- make a .todevice()
What would this do? I was hoping that it would let us move things between CPU and GPU, but I don't think that's consistent with the semantics of cupy and numpy. Maybe it can be handled on the array level for now?
- add a flag/property for .X and layers if its in RAM or VRAM
For now could this just be "whether it's a cupy type or not"? I don't think I'd want to add a property to the AnnData object for this if you can do isinstance
.
- Check if .nnz for cpx is less than 2^31-1 since cupy only supports int32 indptr
Is this in the case where the scipy matrix has 64 bit index types? I think this would make sense there.
from anndata.
Is this in the case where the scipy matrix has 64 bit index types? I think this would make sense there.
Yes there are 64 bit indptr for scipy matrices. If you go above the limit the indptr will become negative and you run into issues with converting the matrix etc.
What would this do? I was hoping that it would let us move things between CPU and GPU, but I don't think that's consistent with the semantics of cupy and numpy. Maybe it can be handled on the array level for now?
Indeed, we could let the user handle this, and the syntax would likely be more similar to pytorch. I really came to like a simple oneliner that moves everything over to RAM and not manually converting every layer.
For now could this just be "whether it's a cupy type or not"? I don't think I'd want to add a property to the AnnData object for this if you can do isinstance.
Using isinstance
is entirely sufficient. I was just considering the scenario where someone wants to run a scanpy function on GPU AnnData or an rsc function with a scipy matrix. I'm currently making all my rapids-singlecell functions compatible with CPU AnnData and was contemplating such a flag.
from anndata.
@Intron7, I'm thinking about how we document this here. I think it would be useful to link out to the rapids-single cell docs. What do you think? I see the latest
build still is referring to CunnData
from anndata.
That’s awesome!
from anndata.
I'll split the others off into new issues, but will point to rapids-single cell for usage examples. Almost closable
from anndata.
Closing as complete in anndata 0.10
from anndata.
Related Issues (20)
- `to_df` docstrings
- dask.array.store but with anndata schema HOT 1
- Indexing With a Dask Array HOT 5
- adata.raw = adata HOT 1
- GPU CI tests failing HOT 2
- Add working `GroupStorageType` type annotation
- 0.10.6 - Cannot write files due to implicit conversion to str dtype. HOT 2
- Saving .h5ad with pd.Series in .uns results in IORegistryError HOT 3
- `read_elem_as_dask` HOT 8
- `FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config'` on CI HOT 2
- Upcoming Scipy 1.13 incompatible with file-backed sparse matrices
- CZI files' `var` count changd
- `/` in column names makes AnnData Zarr object unreadable on windows HOT 12
- String indexes in var lead to UserWarning HOT 1
- Move tests out of the package
- Refactor `BaseCompressedSparseDataset` and subclasses to remove `backed_{csr,csc}_matrix`
- concat_on_disk should use int64 indptrs by default so that it can concatenate large files HOT 4
- TypeError: Can't implicitly convert non-string objects to strings HOT 4
- Error concatenating scRNA with visium dataset
- Error: Expected a python object, received a character HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from anndata.