Comments (6)
Sorry for the long delay in response. There has been a bug that we've been tracking for a long time that causes this problem randomly. We believe we fixed it, although we're not 100% sure. We also have a new branch ("fit_tsne") which implements a version of tSNE using fast fourier transforms. This is both faster and more stable than the current Barnes-Hut implementation.
We will be migrating that to the master branch at some point and also putting it on conda. In the meantime, if you'd like to try cloning the fit_tsne branch and seeing if that works, you may be able to fix the issue that way.
from tsne-cuda.
This problem still exists. GPUassert: an illegal memory access was encountered /home/rmrao/miniconda3/conda-bld/tsnecuda_1538622901232/work/src/util/cuda_utils.cu 55
from tsne-cuda.
Any update on this issue?
from tsne-cuda.
Hello, I've got exactly the same issue:
GPUassert: an illegal memory access was encountered /home/rmrao/miniconda3/conda-bld/tsnecuda_1538622901232/work/src/util/cuda_utils.cu 55
My environment looks like:
OS: Ubuntu 16.04
GPU: TitanX Pascal 12 Go
Python: 3.6.7
cuda90 1.0 h6433d27_0 pytorch
faiss-gpu 1.4.0 py36_cuda9.0.176_1 [cuda90] pytorch
tsnecuda 0.1.1 py36_0 cannylab
The Install is validated with the random(5000,50) dataset as indicated.
The fit_transform gives a result with a random(5583,80) dataset (size of mine); max value 0.999 and min value 3e-7. Takes 4s.
The fit_transform fails (GPUassert ... 55) with my sample dataset(5583,80); max value 0.967 and min value -0.601.
The fit transform fails (GPUassert ... 55) with my dataset.abs(5583,80); max value 0.967 and min value 8.67e-8.
I don't understand what goes wrong with my sample dataset.
My actual dataset is (785000,200).
The biggest random dataset that works is approx (296k, 200).
With (300k, 200), I get the error
terminate called after throwing an instance of 'thrust::system::system_error'
what(): parallel_for failed: out of memory
I don't understand why the computing time for the projection grows up to 2mn31 for 285k and then decreases to 22.0s for 292k and 22.3s for 296k ??
from tsne-cuda.
The fit_tsne
branch has been migrated to master in #39 and is now part of the new release. This has not been pushed to conda yet, but if you compile from source you shouldn't have any issues.
from tsne-cuda.
Binaries for new release on CUDA 9.0 were just pushed to conda. You can install with conda install tsnecuda cuda90 -c cannylab
. This should fix your problem.
from tsne-cuda.
Related Issues (20)
- Feature Request: Custom distance matrix input HOT 3
- CMake error during installation from source HOT 4
- 403 Error when downloading wheels HOT 1
- Failed to cudaFree pointer issue & Memory issue HOT 2
- tsnecuda equivalent for M1 chip HOT 1
- Warning gives a moving target number of training points that could not be satisfied HOT 1
- CUBLAS_STATUS_NOT_INITIALIZED
- CUDA error 209 no kernel image is available for execution on the device HOT 6
- Aborted (core dumped) without any other error messages HOT 1
- TSNE(n_components=1)
- Throws an OSError when initializing HOT 4
- CUBLAS_STATUS_INVALID_VALUE
- CUBLAS_STATUS_NOT_INITIALIZED when importing Datashader *before* running tsnecuda HOT 1
- stuck at Initializing cuda handles HOT 3
- Fail with CUDA error 9. HOT 3
- tsne.fit_transform() prematurely ends before function return in a function call
- Gradient problem HOT 1
- ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory HOT 1
- Is there any way to select the GPU ordinal for the tSNE execution? HOT 1
- random_seed does not seem to work HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tsne-cuda.