wilicc / gpu-burn Goto Github PK
View Code? Open in Web Editor NEWMulti-GPU CUDA stress test
License: BSD 2-Clause "Simplified" License
Multi-GPU CUDA stress test
License: BSD 2-Clause "Simplified" License
I'm getting various errors running gpu_burn
on a newly assembled system.
GPU 0: Tesla K80 (UUID: GPU-6b15ee7a-dc27-9450-1be9-43cff0faef9b)
GPU 1: Tesla K80 (UUID: GPU-0fcbc6b5-7be5-2b6f-7f19-436f8e9a760e)
GPU 2: Tesla K80 (UUID: GPU-95c56dfe-1776-7ac4-5b28-f19ff7376cae)
GPU 3: Tesla K80 (UUID: GPU-603e0fcc-9094-13c0-b775-c15f8e2f3aa9)
Couldn't init a GPU test: Error in "load module":
Initialized device 0 with 11441 MB of memory (11332 MB available, using 10199 MB of it), using FLOATS
Results are 16777216 bytes each, thus performing 635 iterations
read[0] error 0
50.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: 1641686072 (DIED!)- 0 - 0 - 0 temps: 33 C - 29 C - 37 C - 31 C
Summary at: Sat 08 Jan 2022 06:54:33 PM EST
read[0] error 0
50.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: 797249520 (DIED!)- 0 - 0 - 0 temps: 33 C - 29 C - 37 C - 31 C read[0] error 0
50.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: 1594499040 (DIED!)- 0 - 0 - 0 temps: 33 C - 29 C - 37 C - 31 C read[0] error 0
50.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: -1903218736 (DIED!)- 0 - 0 - 0 temps: 33 C - 29 C - 37 C - 31 C read[0] error 0
[===== SNIP 87855 similar lines =====]
100.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - -1 (0 Gflop/s) - -1 (0 Gflop/s) errors: 1720138544 (DIED!)- 0 - 1720138544 (DIED!)- 266762848 (DIED!) temps: 33 C - 29 C - 37 C - 31 C read[0] error 0read[2] error 0read[3] error 0
100.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - -1 (0 Gflop/s) - -1 (0 Gflop/s) errors: -1777579232 (DIED!)- 0 - -1777579232 (DIED!)- 1064012368 (DIED!) temps: 33 C - 29 C - 37 C - 31 C read[0] error 0read[2] error 0read[3] error 0
100.0% proc'd: -1 (0 Gflop/s) - 0 (0 Gflop/s) - -1 (0 Gflop/s) - -1 (0 Gflop/s) errors: -980329712 (DIED!)- 0 - -980329712 (DIED!)- 1861261888 (DIED!) temps: 33 C - 29 C - 37 C - 31 C
Killing processes.. done
Tested 4 GPUs:
GPU 0: FAULTY
GPU 1: OK
GPU 2: FAULTY
GPU 3: OK
Note the snipped output. All of this output happens in the space of a second or two; it does not appear that the tests succeeded, and the "FAULTY" and "OK" results do not seem valid.
This seems potentially related to #7. I tried compiling with different values of COMPUTE
as suggested there, but with similar results.
The machine has dual K80s and nvidia-smi
output is below—any other specs that might be relevant?
Sun Jan 9 14:48:36 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.91.03 Driver Version: 460.91.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla K80 On | 00000000:06:00.0 Off | 0 |
| N/A 27C P8 27W / 149W | 0MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 Tesla K80 On | 00000000:07:00.0 Off | 0 |
| N/A 23C P8 29W / 149W | 0MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 Tesla K80 On | 00000000:84:00.0 Off | 0 |
| N/A 30C P8 26W / 149W | 0MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 Tesla K80 On | 00000000:85:00.0 Off | 0 |
| N/A 24C P8 30W / 149W | 0MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
root@cuda:~/gpu-burn-master# ./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: Tesla T4 (UUID: GPU-0aaee1d0-efc5-d3e6-a13b-f69ac8f6fe36)
Couldn't init a GPU test: Error:
0.0% proc'd: -1 (0 Gflop/s) errors: 0 (DIED!) temps: 68 C
No clients are alive! Aborting
Hello,
If I specify 3600, does it last for 1 hour?
When I run it, it seems to turn off after a few minutes.
Please confirm.
Thank you.
I'm on fedora 31
git clone https://github.com/wilicc/gpu-burn
Cloning into 'gpu-burn'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 66 (delta 9), reused 11 (delta 4), pack-reused 47
Unpacking objects: 100% (66/66), 25.71 KiB | 110.00 KiB/s, done.
[nikolay@localhost Downloads]$ cd gpu-burn
[nikolay@localhost gpu-burn]$ make
bash: make: command not found...
Install package 'make' to provide command 'make'? [N/y] y
* Waiting in queue...
* Loading list of packages....
The following packages have to be installed:
guile22-2.2.6-2.fc31.x86_64 A GNU implementation of Scheme for application extensibility
make-1:4.2.1-15.fc31.x86_64 A GNU tool which simplifies the build process for users
Proceed with changes? [N/y] y
* Waiting in queue...
* Waiting for authentication...
* Waiting in queue...
* Downloading packages...
* Requesting data...
* Testing changes...
* Installing packages...
PATH=/home/nikolay/.local/bin:/home/nikolay/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/nikolay/.composer/vendor/bin:/var/lib/snapd/snap/bin:.:/usr/bin:/home/nikolay/.local/bin:/home/nikolay/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/nikolay/.composer/vendor/bin:/var/lib/snapd/snap/bin /usr/local/cuda/bin/nvcc -I/usr/local/cuda/include -arch=compute_30 -ptx compare.cu -o compare.ptx
/bin/sh: /usr/local/cuda/bin/nvcc: No such file or directory
make: *** [Makefile:10: drv] Error 127
Hi,
I'm trying to compile gpu_burn but I'm getting some errors:
g++ -Wall -O3 -Wno-unused-result -I/usr/local/cuda-11.7/include -c gpu_burn-drv.cpp
gpu_burn-drv.cpp: In member function ‘void GPU_Test<T>::initCompareKernel()’:
gpu_burn-drv.cpp:266:92: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
^
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:267:61: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
^
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:82: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
^
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:95: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
^
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:271:73: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
^
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp: In member function ‘void GPU_Test<T>::compare()’:
gpu_burn-drv.cpp:276:81: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
^
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp: In function ‘void listenClients(std::vector<int>, std::vector<int>, int)’:
gpu_burn-drv.cpp:471:24: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare]
if(res < sizeof(int))
~~~~^~~~~~~~~~~~~
gpu_burn-drv.cpp:484:13: warning: unused variable ‘flops’ [-Wunused-variable]
double flops = (double)processed * (double)OPS_PER_MUL;
^~~~~
gpu_burn-drv.cpp:562:27: warning: comparison of integer expressions of different signedness: ‘time_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
if (startTime + runTime < thisTime)
~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = double; ssize_t = long int]’:
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:741:65: required from here
gpu_burn-drv.cpp:352:4: warning: operation on ‘eventIndex’ may be undefined [-Wsequence-point]
eventIndex = ++eventIndex % maxEvents;
^~~~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = float; ssize_t = long int]’:
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:743:64: required from here
gpu_burn-drv.cpp:352:4: warning: operation on ‘eventIndex’ may be undefined [-Wsequence-point]
gpu_burn-drv.cpp: In instantiation of ‘GPU_Test<T>::GPU_Test(int, bool, bool) [with T = double]’:
gpu_burn-drv.cpp:330:9: required from ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:741:65: required from here
gpu_burn-drv.cpp:288:6: warning: ‘GPU_Test<double>::d_devNumber’ will be initialized after [-Wreorder]
int d_devNumber;
^~~~~~~~~~~
gpu_burn-drv.cpp:286:7: warning: ‘bool GPU_Test<double>::d_doubles’ [-Wreorder]
bool d_doubles;
^~~~~~~~~
gpu_burn-drv.cpp:130:2: warning: when initialized here [-Wreorder]
GPU_Test(int dev, bool doubles, bool tensors) :
^~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘void GPU_Test<T>::compare() [with T = double]’:
gpu_burn-drv.cpp:349:4: required from ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:741:65: required from here
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘GPU_Test<T>::GPU_Test(int, bool, bool) [with T = float]’:
gpu_burn-drv.cpp:330:9: required from ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:743:64: required from here
gpu_burn-drv.cpp:288:6: warning: ‘GPU_Test<float>::d_devNumber’ will be initialized after [-Wreorder]
int d_devNumber;
^~~~~~~~~~~
gpu_burn-drv.cpp:286:7: warning: ‘bool GPU_Test<float>::d_doubles’ [-Wreorder]
bool d_doubles;
^~~~~~~~~
gpu_burn-drv.cpp:130:2: warning: when initialized here [-Wreorder]
GPU_Test(int dev, bool doubles, bool tensors) :
^~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘void GPU_Test<T>::compare() [with T = float]’:
gpu_burn-drv.cpp:349:4: required from ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:743:64: required from here
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:276:31: warning: ‘CUresult cuLaunchGridAsync(CUfunction, int, int, CUstream)’ is deprecated [-Wdeprecated-declarations]
checkError(cuLaunchGridAsync(d_function, SIZE/g_blockSize, SIZE/g_blockSize, 0), "Launch grid");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15260:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuLaunchGridAsync(CUfunction f, int grid_width, int grid_height, CUstream hStream);
^~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘void GPU_Test<T>::initCompareKernel() [with T = double]’:
gpu_burn-drv.cpp:227:3: required from ‘void GPU_Test<T>::initBuffers(T*, T*, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:331:3: required from ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = double; ssize_t = long int]’
gpu_burn-drv.cpp:741:65: required from here
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp: In instantiation of ‘void GPU_Test<T>::initCompareKernel() [with T = float]’:
gpu_burn-drv.cpp:227:3: required from ‘void GPU_Test<T>::initBuffers(T*, T*, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:331:3: required from ‘void startBurn(int, int, T*, T*, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:611:15: required from ‘void launch(int, bool, bool, ssize_t) [with T = float; ssize_t = long int]’
gpu_burn-drv.cpp:743:64: required from here
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:266:28: warning: ‘CUresult cuParamSetSize(CUfunction, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetSize(d_function, __alignof(T*) + __alignof(int*) + __alignof(size_t)), "set param size");
~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14991:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetSize(CUfunction hfunc, unsigned int numbytes);
^~~~~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:267:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, 0, &d_Cdata, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:268:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*), &d_faultyElemData, sizeof(T*)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:269:25: warning: ‘CUresult cuParamSetv(CUfunction, int, void*, unsigned int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuParamSetv(d_function, __alignof(T*) + __alignof(int*), &d_iters, sizeof(size_t)), "set param");
~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:15092:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuParamSetv(CUfunction hfunc, int offset, void *ptr, unsigned int numbytes);
^~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
gpu_burn-drv.cpp:271:33: warning: ‘CUresult cuFuncSetBlockShape(CUfunction, int, int, int)’ is deprecated [-Wdeprecated-declarations]
checkError(cuFuncSetBlockShape(d_function, g_blockSize, g_blockSize, 1), "set block size");
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from gpu_burn-drv.cpp:51:
/usr/local/cuda-11.7/include/cuda.h:14925:36: note: declared here
__CUDA_DEPRECATED CUresult CUDAAPI cuFuncSetBlockShape(CUfunction hfunc, int x, int y, int z);
^~~~~~~~~~~~~~~~~~~
ATH=/usr/local/cuda/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/usr/bin:. /usr/local/cuda-11.7/bin/nvcc -I/usr/local/cuda-11.7/include -arch=compute_50 -ptx compare.cu -o compare.ptx
g++ -o gpu_burn gpu_burn-drv.o -O3 -lcuda -L/usr/local/cuda-11.7/lib64 -L/usr/local/cuda-11.7/lib -Wl,-rpath=/usr/local/cuda-11.7/lib64 -Wl,-rpath=/usr/local/cuda-11.7/lib -lcublas -lcudart```
---
OS:
NAME="Oracle Linux Server"
VERSION="8.6"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="8.6"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Oracle Linux Server 8.6"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:8:6:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://bugzilla.oracle.com/"
ORACLE_BUGZILLA_PRODUCT="Oracle Linux 8"
ORACLE_BUGZILLA_PRODUCT_VERSION=8.6
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=8.6
NVIDIA -- CUDA:
NVIDIA-SMI 515.48.07 Driver Version: 515.48.07 CUDA Version: 11.7
Is this not being updated anymore? I do notice that some of the forks are more updated than this one, but haven't checked all forks to find which has the most changes. Would be great if even the readme was updated to say that this project is no longer updated.
I am trying to install gpu-burn to /usr/bin, but no mattter what I try I get this error since it needs to be ran from the gpu-burn folder ONLY:
Couldn't init a GPU test: Error in "couldn't find file "compare.ptx" from working directory": CUDA_ERROR_NOT_FOUND
I tried symlinks, adding to the PATH. Is there a way I can have it pull from the folder it was executed from or have a shared location of some sort?
Hi Wili, I have used the gpu burn to do stress test on my 2 GPUs. The Tesla K40m failed and showed the following errors. After the system stops running is shows the message "Killing process" and hangs without killing the process. I waited 20 minutes but it never stops and gives me the final summary.
I am not sure what it means and why the script is crashing. I hope you can show some light on it.
I have added all the information below.
Thanks.
$ make && ./gpu_burn -d 1200
GPU 0: Tesla K40m (UUID: GPU-660d987e-5b28-3bf6-2a2d-1d5ecae7f6df)
GPU 1: Quadro M4000 (UUID: GPU-ad447a04-f6f5-3ccf-c0f7-3218b1919de2)
Initialized device 1 with 11441 MB of memory (11330 MB available, using 10197 MB of it), using DOUBLES
Initialized device 0 with 8126 MB of memory (8016 MB available, using 7214 MB of it), using DOUBLES
10.4% proc'd: 446 (74 Gflop/s) - 6636 (423 Gflop/s) errors: 0 - 0 temps: 93 C - 57 C
Summary at: Wed 11 Apr 11:27:59 BST 2018
13.1% proc'd: 669 (74 Gflop/s) - 6952 (154 Gflop/s) errors: 0 - 0 temps: 95 C - 58 C Failure during compute: Error in "Read faultyelemdata": CUDA_ERROR_UNKNOWN
20.5% proc'd: 892 (74 Gflop/s) - -1 (154 Gflop/s) errors: 0 - -1 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:30:00 BST 2018
30.6% proc'd: 1561 (74 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C ^[[B~5B
Summary at: Wed 11 Apr 11:32:01 BST 2018
40.7% proc'd: 2007 (74 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:34:02 BST 2018
50.8% proc'd: 2453 (73 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:36:03 BST 2018
60.8% proc'd: 2899 (74 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:38:04 BST 2018
70.9% proc'd: 3568 (73 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:40:05 BST 2018
74.0% proc'd: 3791 (73 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:41:05 BST 2018
81.0% proc'd: 4014 (74 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:42:06 BST 2018
91.1% proc'd: 4460 (73 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
Summary at: Wed 11 Apr 11:44:07 BST 2018
100.0% proc'd: 5129 (73 Gflop/s) - -1 (154 Gflop/s) errors: 0 - 0 (DIED!) temps: 95 C - 58 C
$ nvidia-smi # Just before the GPU crashed
Wed Apr 11 11:28:13 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.30 Driver Version: 390.30 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K40m On | 00000000:02:00.0 Off | 0 |
| N/A 94C P0 97W / 235W | 10288MiB / 11441MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 1 Quadro M4000 On | 00000000:03:00.0 Off | N/A |
| 52% 58C P0 63W / 120W | 7310MiB / 8126MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 21593 C ./gpu_burn 10266MiB |
| 1 21590 C ./gpu_burn 7289MiB |
+-----------------------------------------------------------------------------+
This is awesome! Just wanted to leave a note of gratitude.
$ make -j48
...
In file included from /usr/lib/cuda/include/cuda_runtime.h:83,
from <command-line>:
/usr/lib/cuda/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported!
138 | #error -- unsupported GNU version! gcc versions later than 8 are not supported!
| ^~~~~
make: *** [Makefile:10: drv] Error 1
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
On some of our systems, we have 8 or more GPUs. I like this tool, but in those cases, the output wraps badly and could use a bit of formatting help. Basically, because you assume terminal width >= line length and your use of \r
to return to the "beginning of the line".
Before I go implementing a solution, are you even interested?
Would a [n]curses approach be appreciated or dismissed?
Other approaches?
Hello,
I have a problem when I do the burn test on my Quadro RTX4000 GPU
I feel like it doesn't really "burn" it since during the burn I can see with nvidia-smi that it only uses 45W/125W which is only one third
When I do it on my K80 GPU it uses 142/149W and 143/149W which is almost the maximum
Do I have to install some other tools to be able to have a complete burn on a RTX4000 GPU ?
Thank you
gpu_burn runs just fine, but segfaults appear in the kernel log (dmesg) as soon as the program terminates
How to test jetson with this
I have compiled the tool and have ran it on a multi-gpu system. I get the following...
GPU 0: TITAN RTX (UUID: GPU-f8cb36a5-7a80-8667-e9c1-66c4acddaeaf)
GPU 1: GeForce RTX 2080 Ti (UUID: GPU-54bf72e7-8427-4d2e-da44-fafcafa8d97e)
GPU 2: GeForce RTX 2080 Ti (UUID: GPU-e2189a61-7a4e-1235-7f17-45c86780f7c8)
GPU 3: GeForce RTX 2080 Ti (UUID: GPU-d0ad5f09-7092-a5fd-10e4-a17238dbe701)
GPU 4: GeForce RTX 2080 Ti (UUID: GPU-13561097-4310-6a40-85e5-1688438e932c)
GPU 5: TITAN RTX (UUID: GPU-95425571-c209-11ed-b978-067923fd639c)
Initialized device 0 with 24190 MB of memory (23919 MB available, using 21527 MB of it), using FLOATS
Couldn't init a GPU test: Error in "load module":
100.0% proc'd: 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: 0 - 0 - 0 - 0 - 0 - 0 temps: 30 C - 28 C - 26 C - 29 C - 28 C - 29 C
Summary at: Wed Jul 31 17:00:39 PDT 2019
Killing processes.. done
Tested 6 GPUs:
GPU 0: OK
GPU 1: OK
GPU 2: OK
GPU 3: OK
GPU 4: OK
GPU 5: OK
I am not sure what to do here. It says device 0 initialized but then says it failed to init the test...
The above was running the tool for 1 second but the same results for longer periods of time.
One thing I notice is that when it starts up I get a spike in GPU usage: Volatile GPU-Util for some of the GPUs hit 24% but then fall. Not sure if this can help you in any way.
Any help would be appreciated.
Hi,
running the test on a dual GPUs system (2x1080 water cooled) I got the following result:
➜ gpu-burn git:(master) docker run --rm --gpus all gpu_burn
GPU 0: NVIDIA GeForce GTX 1080 (UUID: GPU-70f352d1-aa4f-6f8a-2033-f8dff3dca024)
GPU 1: NVIDIA GeForce GTX 1080 (UUID: GPU-8f7dc020-3610-dda7-fa8e-15cdf18a45e2)
11.7% proc'd: 2947 (7124 Gflop/s) - 2664 (6963 Gflop/s) errors: 0 - 0 temps: 56 C - 73 C
Summary at: Sat Sep 25 17:10:59 UTC 2021
23.3% proc'd: 5473 (7020 Gflop/s) - 5772 (6944 Gflop/s) errors: 0 - 0 temps: 57 C - 79 C
Summary at: Sat Sep 25 17:11:06 UTC 2021
35.0% proc'd: 8420 (6951 Gflop/s) - 7992 (6853 Gflop/s) errors: 0 - 0 temps: 59 C - 79 C
Summary at: Sat Sep 25 17:11:13 UTC 2021
46.7% proc'd: 11367 (7109 Gflop/s) - 11100 (6743 Gflop/s) errors: 0 - 0 temps: 62 C - 82 C
Summary at: Sat Sep 25 17:11:20 UTC 2021
58.3% proc'd: 13893 (7088 Gflop/s) - 13764 (6608 Gflop/s) errors: 0 - 0 temps: 62 C - 82 C
Summary at: Sat Sep 25 17:11:27 UTC 2021
68.3% proc'd: 16840 (7100 Gflop/s) - 15984 (6599 Gflop/s) errors: 0 - 0 temps: 62 C - 82 C
Summary at: Sat Sep 25 17:11:33 UTC 2021
80.0% proc'd: 19366 (6981 Gflop/s) - 18648 (6485 Gflop/s) errors: 0 - 0 temps: 63 C - 83 C
Summary at: Sat Sep 25 17:11:40 UTC 2021
91.7% proc'd: 21892 (6931 Gflop/s) - 21312 (6507 Gflop/s) errors: 0 - 0 temps: 62 C - 80 C
Summary at: Sat Sep 25 17:11:47 UTC 2021
100.0% proc'd: 24839 (7065 Gflop/s) - 23532 (6471 Gflop/s) errors: 0 - 0 temps: 64 C - 83 C
Killing processes.. Initialized device 1 with 8119 MB of memory (7931 MB available, using 7138 MB of it), using FLOATS
Results are 16777216 bytes each, thus performing 444 iterations
Freed memory for dev 1
Uninitted cublas
Initialized device 0 with 8116 MB of memory (7537 MB available, using 6783 MB of it), using FLOATS
Results are 16777216 bytes each, thus performing 421 iterations
Freed memory for dev 0
Uninitted cublas
done
Tested 2 GPUs:
GPU 0: OK
GPU 1: OK
what could be the so-different temperatures between the 2 GPUS? is it due to the software stressing in a different way the 2 GPUs or ... my GPU#2 has some sort of thermal issue?
Hello,
First of all, many thanks for this app, it's great!
But, I need to implement an option to get a formatted output.
I would like start by JSON, with a -j <filename.json>
option.
Maybe something like that:
{
"devices": {
"<DEVICE>": {
"model": "<GPU_MODEL>",
"UUID": "<UUID>",
"<GLFOPS>": [<ARRAY_OF_GLOPS_VALUES>],
"<ERROS>": [<ARRAY_OF_ERROS_VALUES>],
"<TEMPS>": [<ARRAY_OF_TEMPS_VALUES>]
}
},
"elapsed": [<ARRAY_OF_SUMMARY_TIMES>]
}
My example with 2 GPUs:
{
"devices": {
"GPU 0": {
"model": "A100-SXM-80GB",
"UUID": "GPU-3bb11c3f-033a",
"Gflop/s": ["17668", "17699", "17705", "17704"],
"errors": ["0", "0", "0", "0"],
"temps (C)": ["56", "59", "60", "60"]
},
"GPU 1": {
"model": "A100-SXM-80GB",
"UUID": "GPU-c1aa22c3a-032b",
"Gflop/s": ["17668", "17699", "17705", "17704"],
"errors": ["0", "0", "0", "0"],
"temps(C)": ["56", "59", "60", "60"]
}
}
"elapsed(s)": ["10", "20", "30", "40"]
}
It gonna be easier to analyze and make statistics, with python
or jq
.
Before starting, what do you think about?
Best,
Adrien
This fails on a system with 16 GPGPUs (Tesla V100 SXM3s)
$ /opt/gpu-burn$ ./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: Tesla V100-SXM3-32GB (UUID: GPU-0baad8c1-90f1-a0cd-bbd6-5bfdaeb6a747)
GPU 1: Tesla V100-SXM3-32GB (UUID: GPU-63cb6f99-dce9-171f-1e51-b4f82ac54a2a)
GPU 2: Tesla V100-SXM3-32GB (UUID: GPU-fd6dee30-31a6-e0d9-5bea-2f10f4f4adf9)
GPU 3: Tesla V100-SXM3-32GB (UUID: GPU-40b208b4-c598-d125-1f4d-428127b62a00)
GPU 4: Tesla V100-SXM3-32GB (UUID: GPU-0c2b7ba6-ca06-dccb-8fa2-a97e4cd3ba9b)
GPU 5: Tesla V100-SXM3-32GB (UUID: GPU-3b3a305f-3e5b-c5cf-031b-d96dfc9f19cc)
GPU 6: Tesla V100-SXM3-32GB (UUID: GPU-178881f5-c6b0-4de3-309a-0eac0a6153bd)
GPU 7: Tesla V100-SXM3-32GB (UUID: GPU-e7e5c623-6b17-9598-056f-cad0ff599286)
GPU 8: Tesla V100-SXM3-32GB (UUID: GPU-2242ae02-9e04-7fcf-f287-b4d17e555d57)
GPU 9: Tesla V100-SXM3-32GB (UUID: GPU-ad843060-d718-47a9-b1d6-117a7870123a)
GPU 10: Tesla V100-SXM3-32GB (UUID: GPU-d36440ed-54fc-c4d0-5288-93ed6375795c)
GPU 11: Tesla V100-SXM3-32GB (UUID: GPU-894366f9-451b-d088-f2cb-ee4037dc19da)
GPU 12: Tesla V100-SXM3-32GB (UUID: GPU-c578fe7a-d3f5-09fd-a847-66d5f6ad3b5c)
GPU 13: Tesla V100-SXM3-32GB (UUID: GPU-e4a6f9ca-77dd-3895-d0ad-29430cc610b3)
GPU 14: Tesla V100-SXM3-32GB (UUID: GPU-9ebe7746-ac18-ba65-6117-319681c32f7c)
GPU 15: Tesla V100-SXM3-32GB (UUID: GPU-d7804840-3d39-7b1b-2dc0-f301be7f2664)
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
*** buffer overflow detected ***: ./gpu_burn terminated
Aborted (core dumped)
At this point, it seems to just keep spawning instances of gpu_burn:
$ ps axf |grep gpu_burn |wc -l
20544
Here's 20,544 instances that I think are all in the Dead state:
24750 ? D 0:00 ./gpu_burn
ce:00.0 Bridge: NVIDIA Corporation Device 1ac2 (rev a1)
Flags: bus master, fast devsel, latency 0, IRQ 928, NUMA node 1
Memory at e1000000 (32-bit, non-prefetchable) [size=16M]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [250] Latency Tolerance Reporting
Capabilities: [258] L1 PM Substates
Capabilities: [128] Power Budgeting
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024
Kernel driver in use: nvidia-nvswitch
These are all connected via NVLink.
Running the latest cuda stack from developer.nvidia.com on Ubuntu 18.04 with the 4.15 kernel.
gpu_burn was pulled from git and compiled just before trying the test.
My setup is to run deeplearning inside docker containers. I assume your app does the same.
Your app finds my GPUs but fails to install CUDA I assume
does not run:
./gpu_burn 10
GPU 0: GeForce GTX 1650 (UUID: GPU-0fccd51a-9bf1-17e7-812c-c1dcf2ca2c42)
Initialized device 0 with 3914 MB of memory (3165 MB available, using 2849 MB of it), using FLOATS
Failure during compute: Error in "SGEMM": CUBLAS_STATUS_EXECUTION_FAILED
10.0% proc'd: -1 (0 Gflop/s) errors: -1 (DIED!) temps: 46 C
No clients are alive! Aborting
using GeForce GTX 1650, Ubuntu 18.04.3 LTS, NVIDIA Driver Version: 430.50, CUDA Version: 10.1
other CUDA code runs fine on my setup!
Hi!
I ran gpu_burn on my work station with two Quadro RTX 8000, cuda version 10.1 and driver version 430.64.
One of the GPU died showing “Failure:Error in "SGEMM": CUBLAS_STATUS_INTERNAL_ERROR ” and got faulty finally, while the other one reported ok. So it didn't seem to be a version problem.
But I still have no idea about this error. Dose it mean there is something wrong with the GPU hardware?
Any help is appreciated!
I have been recently facing some power issues in some machines with 8 Nvidia RTX 3090.
While performing the Tensorflow unit tests (when compiling tensorflow by hand) leads me consistently to kernel panics on this machine (because the power consumption is so huge, one of the gpu dies and the kernel doesn't know what to do).
I just ran gpu-burn test for a couple hours consistently, with and without doubles, with and without tensor cores. The machine runs just fine. I run tensorflow's tests again, the machine has a higher power consumption and goes dead again.
Other than that, great test for the throughput and temperature. Thanks!
Hi,
I Downloaded the latest packages from the website and try to compile the code and give me this error
PATH=/home/bizon/.local/bin:/home/bizon/anaconda3/bin:/home/bizon/anaconda3/condabin:/usr/local/cuda-11.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:.:/usr/bin:/home/bizon/.local/bin:/home/bizon/anaconda3/bin:/home/bizon/anaconda3/condabin:/usr/local/cuda-11.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin /usr/local/cuda/bin/nvcc -I/usr/local/cuda/include -arch=compute_30 -ptx compare.cu -o compare.ptx nvcc fatal : Value 'compute_30' is not defined for option 'gpu-architecture' make: *** [Makefile:10: drv] Error 1
I tried with different options and I am using RTX 3080
make COMPUTE=8.6
Same problem, please help, thanks
There seems to be a compatibility issue between gpu-burn and the 11.3 toolkit/465 driver.
I have replicated the behavior on two different machines, one with 2x 2080 and another with 2x 3090. Reinstalling 11.2.2 with 460 driver solves the issue.
OS: CentOS 7.9.2009 (Core) with ELRepo kernel-lt 5.4.88
CUDA Toolkit 11.3.1:
$ ./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: NVIDIA GeForce RTX 3090 (UUID: GPU-c3832820-5d0a-f9e4-b9c8-fd92111f4b31)
GPU 1: NVIDIA GeForce RTX 3090 (UUID: GPU-ed191d51-aec3-e5bb-4a90-40c629499306)
Initialized device 1 with 24268 MB of memory (23740 MB available, using 21366 MB of it), using FLOATS
Initialized device 0 with 24268 MB of memory (23740 MB available, using 21366 MB of it), using FLOATS
Failure during compute: Error in "SGEMM":
10.0% proc'd: 0 (0 Gflop/s) - -1 (0 Gflop/s) errors: 0 - -1 (DIED!) temps: 39 C - 41 C Failure during compute: Error in "SGEMM":
10.0% proc'd: -1 (0 Gflop/s) - -1 (0 Gflop/s) errors: -1 (DIED!)- -1 (DIED!) temps: 39 C - 41 C
No clients are alive! Aborting
$ nvidia-smi
Thu Jun 10 11:35:38 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01 Driver Version: 465.19.01 CUDA Version: 11.3 |
...
CUDA Toolkit 11.2.2:
$ ./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: GeForce RTX 3090 (UUID: GPU-c3832820-5d0a-f9e4-b9c8-fd92111f4b31)
GPU 1: GeForce RTX 3090 (UUID: GPU-ed191d51-aec3-e5bb-4a90-40c629499306)
Initialized device 1 with 24268 MB of memory (23736 MB available, using 21362 MB of it), using FLOATS
Initialized device 0 with 24268 MB of memory (23736 MB available, using 21362 MB of it), using FLOATS
20.0% proc'd: 0 (0 Gflop/s) - 1333 (8237 Gflop/s) errors: 0 - 0 temps: 39 C - 41 C
Summary at: do jun 10 11:46:51 CEST 2021
40.0% proc'd: 1333 (8165 Gflop/s) - 2666 (15162 Gflop/s) errors: 0 - 0 temps: 39 C - 41 C
Summary at: do jun 10 11:46:53 CEST 2021
70.0% proc'd: 5332 (15355 Gflop/s) - 3999 (15209 Gflop/s) errors: 0 - 0 temps: 53 C - 52 C
Summary at: do jun 10 11:46:56 CEST 2021
90.0% proc'd: 6665 (15408 Gflop/s) - 6665 (15168 Gflop/s) errors: 0 - 0 temps: 53 C - 52 C
Summary at: do jun 10 11:46:58 CEST 2021
100.0% proc'd: 9331 (15338 Gflop/s) - 7998 (15186 Gflop/s) errors: 0 - 0 temps: 55 C - 55 C
Killing processes.. Freed memory for dev 1
Uninitted cublas
Freed memory for dev 0
Uninitted cublas
done
Tested 2 GPUs:
GPU 0: OK
GPU 1: OK
$ nvidia-smi
Thu Jun 10 12:04:06 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |
...
Hi,
The system replies "errors: 1579 (DIED!)", the log is showed below and attached.
If anyone know what happened and the direction to solve it?
Thanks.
If there is an init error on the GPU, gpu-burn does not exist but instead continues and prints "OK" at the end.
$ ./gpu_burn 3
GPU 0: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 1: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 2: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 3: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 4: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 5: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 6: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
GPU 7: GeForce GTX 1080 Ti (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
Couldn't init a GPU test: Error in "init": CUBLAS_STATUS_NOT_INITIALIZED
100.0% proc'd: 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 temps: 30 C - 31 C - 32 C - 29 C - 34 C - 31 C - 32 C - 30 C
Summary at: Wed Dec 4 18:44:29 UTC 2019
100.0% proc'd: 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) - 0 (0 Gflop/s) errors: 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 temps: 30 C - 31 C - 32 C - 29 C - 34 C - 31 C - 32 C - 30 C
Killing processes.. done
Tested 10 GPUs:
GPU 0: OK
GPU 1: OK
GPU 2: OK
GPU 3: OK
GPU 4: OK
GPU 5: OK
GPU 6: OK
GPU 7: OK
It would be better to stop at the point where the error is thrown and not continue until the timer expired.
Enhancement request: If time set to 0 then run infinitely.
Hi.
I am using your program to verify my GPUs.
There are something inconvenient things.
If you don't mind, Please update it :)
Thanks.
Hello!
How can I specify one of the GPUs to run gpu-burn on it, instead of run on all the GPUs in our cluster by using "./gpu_burn"?
Could you please consider adding an option for this?
Thanks!
I'm getting this error when running the burn test.
$ ./gpu_burn 30
Burning for 30 seconds.
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >terminate called after throwing an instance of
'terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator
'
terminate called after throwing an instance of 'terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator
'
terminate called after throwing an instance of 'terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator
'
std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator
std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
'
std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator >'
terminate called after throwing an instance of 'terminate called after throwing an instance of 'terminate called after throwing an instance of 'std::__cxx11::basic_string<ch
ar, std::char_traits, std::allocator >'
std::__cxx11::basic_string<char, std::char_traits, std::allocator >' terminate called after throwing an instance of 'terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits, std::allocator
'
I eventually had to CTRL-C out of this. It's on Ubuntu 22.04 with the latest gpu_burn source and cuda toolkit installed. I'm doing some bug testing of a wrapper I am using, when I hit this.
One of the last few commits includes a change to throw a std::runtime_error
but doesn't actually include <stdexcept>
so make
fails depending on the environment. Some of the commonly used std library headers used to include <stdexcept>
so things like std::runtime_error
may have been implicitly defined.
__@__:~/cave/gpu-burn$ make
g++ -O3 -Wno-unused-result -I/usr/local/cuda/include -c gpu_burn-drv.cpp
gpu_burn-drv.cpp: In function ‘int main(int, char**)’:
gpu_burn-drv.cpp:817:23: error: ‘runtime_error’ is not a member of ‘std’
throw std::runtime_error("No CUDA capable GPUs found.\n");
^
make: *** [gpu_burn-drv.o] Error 1
Is there a place where I can lookup the typical gpu_burn results for various GPUs? For example, I am seeing what I consider to be a super low Gflops/s for K80 on Azure, but I don't know what it's supposed to be.
$ ./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: Tesla K80 (UUID: GPU-b57841d9-9b07-f573-b01c-c2e16694f1ae)
Initialized device 0 with 11441 MB of memory (11351 MB available, using 10216 MB of it), using FLOATS
50.0% proc'd: 636 (2085 Gflop/s) errors: 0 temps: 64 C
Summary at: Fri Aug 9 22:17:04 UTC 2019
100.0% proc'd: 636 (2085 Gflop/s) errors: 0 temps: 64 C
Summary at: Fri Aug 9 22:17:09 UTC 2019
100.0% proc'd: 1908 (2273 Gflop/s) errors: 0 temps: 66 C
It would be cool, if gpu_burn asks the user to publish the results at the end of the test, in exchange of looking up the test results from other users with similar hardware so that the score can be compared.
Or you could simply setup a wiki, or host a result lists and allow people to send PR with results.
100.0% proc'd: 18000 (6311 Gflop/s) - 12600 (6440 Gflop/s) - 11700 (6490 Gflop/s) - 11700 (6457 Gflop/s) - 11700 (6451 Gflop/s) - 10800 (6454 Gflop/s) - 10800 (6446 Gflop/s) - 10800 (6461 Gflop/s) - 10800 (6480 Gflop/s) - 10800 (6411 Gflop/s) - 10800 (6660 Gflop/s) - 11700 (6532 Gflop/s) - 10800 (6523 Gflop/s) - 10800 (6490 Gflop/s) - 10800 (6432 Gflop/s) - 10800 (6439 Gflop/s) errors: 0 - -524288000 (WARNING!)- -524288000 (WARNING!)- -524288000 (WARNING!)- -524288000 (WARNING!)- 0 - 0 - 0 - 0 - -524288000 (WARNING!)- 0 - -524288000 (WARNING!)- 0 - 0 - -524288000 (WARNING!)- 0 temps: 51 C - 45 C - 48 C - 50 C - 49 C - 46 C - 46 C - 50 C - 51 C - 47 C - 46 C - 50 C - 50 C - 47 C - 47 C - 52 C
Killing processes.. done
Tested 16 GPUs:
GPU 0: FAULTY
GPU 1: FAULTY
GPU 2: FAULTY
GPU 3: FAULTY
GPU 4: FAULTY
GPU 5: FAULTY
GPU 6: FAULTY
GPU 7: FAULTY
GPU 8: FAULTY
GPU 9: FAULTY
GPU 10: FAULTY
GPU 11: FAULTY
GPU 12: FAULTY
GPU 13: FAULTY
GPU 14: FAULTY
GPU 15: FAULTY
ubuntu@SYS-9029GP-x16GPU:/opt/gpu-burn$ echo $?
0
So looking at the source code, gpu_burn always exits 0, even when a GPU reports errors in the comparison. I modified compare.cu so that there's always an error logged, and ran it for 20 seconds and found that while all the GPUs are showing as faulty, because of injected errors, the test still exits 0.
Hello,
make NVCCFLAGS=-ccbin
Can you tell me how to specify ?
Thank you.
Hi! Thanks for the great tool
I have been trying to run it with nvidia-docker2 and that's what I get:
sudo docker run --rm --gpus all gpu_burn
./gpu_burn: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./gpu_burn)
./gpu_burn: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./gpu_burn)
However:
strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBC
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_3.4.29
GLIBCXX_3.4.30
GLIBC_2.2.5
GLIBC_2.3
GLIBC_2.14
GLIBC_2.6
GLIBC_2.33
GLIBC_2.25
GLIBC_2.18
GLIBC_2.16
GLIBC_2.32
GLIBC_2.7
GLIBC_2.3.4
GLIBC_2.17
GLIBC_2.3.2
GLIBC_2.34
GLIBC_2.4
GLIBCXX_DEBUG_MESSAGE_LENGTH
When I did try it without docker:
git clone https://github.com/wilicc/gpu-burn
cd gpu-burn
make
./gpu_burn 60
The make produced warnings:
and then the ./gpu_burn 60 shut down the system after initing the GPUs ;(
Here is an output of nvidia-smi:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 520.61.05 Driver Version: 520.61.05 CUDA Version: 11.8 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA RTX A6000 Off | 00000000:31:00.0 Off | Off |
| 30% 32C P8 14W / 300W | 0MiB / 49140MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA RTX A6000 Off | 00000000:4B:00.0 Off | Off |
| 30% 31C P8 8W / 300W | 0MiB / 49140MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA RTX A6000 Off | 00000000:B1:00.0 Off | Off |
| 30% 30C P8 11W / 300W | 0MiB / 49140MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA RTX A6000 Off | 00000000:CA:00.0 Off | Off |
| 30% 31C P8 8W / 300W | 0MiB / 49140MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
And
nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
I'm on the Ubuntu 22.04.1 LTS
Any idea on what is going on here?
Hello!
First of all I'd like to thank every contributor for their work, gpu burn is a very useful tool for us.
I think we found a bug in the code that prints out the final results. We used the tool to test a machine with 3 GPUs for 90 minutes. I'm attaching the stdout of gpu burn: 20221207_190200_gpu_burn_stdout.zip
From the output you can see that around the end of the test (at 92.2%) GPU 1 started to produce some errors:
92.2% proc'd: 2710955 (9344 Gflop/s) - 2744965 (9469 Gflop/s) - 2710060 (9341 Gflop/s) errors: 0 - 0 - 0 temps: 70 C - 70 C - 81 C
92.2% proc'd: 2710955 (9344 Gflop/s) - 2744965 (9469 Gflop/s) - 2710060 (9341 Gflop/s) errors: 0 - 0 - 0 temps: 70 C - 70 C - 81 C
92.2% proc'd: 2710955 (9344 Gflop/s) - 2744965 (9469 Gflop/s) - 2710060 (9341 Gflop/s) errors: 0 - 0 - 0 temps: 70 C - 70 C - 81 C
92.2% proc'd: 2710955 (9344 Gflop/s) - 2744965 (9469 Gflop/s) - 2710060 (9341 Gflop/s) errors: 0 - 0 - 0 temps: 70 C - 70 C - 81 C
92.2% proc'd: 2711850 (9347 Gflop/s) - 2744965 (9469 Gflop/s) - 2710060 (9341 Gflop/s) errors: 1 (WARNING!)- 0 - 0 temps: 70 C - 70 C - 81 C
92.2% proc'd: 2711850 (9347 Gflop/s) - 2744965 (9469 Gflop/s) - 2710955 (9347 Gflop/s) errors: 1 (WARNING!)- 0 - 0 temps: 70 C - 70 C - 81 C
92.3% proc'd: 2711850 (9347 Gflop/s) - 2745860 (9459 Gflop/s) - 2710955 (9347 Gflop/s) errors: 1 (WARNING!)- 0 - 0 temps: 70 C - 70 C - 81 C
92.3% proc'd: 2712745 (9341 Gflop/s) - 2745860 (9459 Gflop/s) - 2710955 (9347 Gflop/s) errors: 1 (WARNING!)- 0 - 0 temps: 70 C - 70 C - 81 C
92.3% proc'd: 2712745 (9341 Gflop/s) - 2745860 (9459 Gflop/s) - 2711850 (9343 Gflop/s) errors: 1 (WARNING!)- 0 - 0 temps: 70 C - 70 C - 81 C
However the final report says:
Tested 3 GPUs:
GPU 0: OK
GPU 1: OK
GPU 2: OK
The problem is that the clientFaulty.at(i) = true;
statement is executed only when the intermediate summary reports are printed.
Lines 619 to 632 in 454d1bf
Which happens after 10%, 20%, ..., 90% of progress. However, if an error happens after 90% of progress - like that happened in our case - then the flag is not set and the final result will be wrong.
'''
$ ./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: NVIDIA GeForce GTX 1060 6GB (UUID: GPU-210cc47f-f02f-c51e-8d2c-4bd28141e785)
Segmentation fault
$ nvidia-smi
Fri Jul 30 14:47:31 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02 Driver Version: 471.41 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:09:00.0 On | N/A |
| 45% 42C P8 13W / 120W | 1584MiB / 6144MiB | ERR! Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
'''
hi,will it work on amd gpu?
With the new CUDA 11, compute_30 is not recognised anymore, simply replacing it with compute_50 in the Makefile was enough.
make
PATH=/home/deployer/bin:/home/deployer/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:.:/usr/bin:/home/deployer/bin:/home/deployer/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games /usr/local/cuda/bin/nvcc -I/usr/local/cuda/include -arch=compute_30 -ptx compare.cu -o compare.ptx
nvcc fatal : Value 'compute_30' is not defined for option 'gpu-architecture'
Makefile:10: recipe for target 'drv' failed
make: *** [drv] Error 1
Edit: I originally had another issue afterwards but that was on me
[bin]$ git clone [email protected]:wilicc/gpu-burn.git
Cloning into 'gpu-burn'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 129 (delta 30), reused 44 (delta 22), pack-reused 73
Receiving objects: 100% (129/129), 34.94 KiB | 420.00 KiB/s, done.
Resolving deltas: 100% (69/69), done.
[bin]$ cd gpu-burn/
[gpu-burn]$ make
g++ -O3 -Wno-unused-result -I/usr/local/cuda/include -c gpu_burn-drv.cpp
gpu_burn-drv.cpp:50:10: fatal error: cuda.h: No such file or directory
50 | #include <cuda.h>
| ^~~~~~~~
compilation terminated.
make: *** [Makefile:32: gpu_burn-drv.o] Error 1
With GTX 2080 Turing graphics, the program errors with the "DIED!" message.
Hello,
I recently build gpu-burn to test a new GPU node on our cluster. It runs and throws the following error:
./gpu_burn
Run length not specified in the command line. Burning for 10 secs
GPU 0: TITAN RTX (UUID: GPU-5ca89547-0a71-fc6d-f749-1651ee078536)
GPU 1: TITAN RTX (UUID: GPU-fd78f358-3e8b-e2bb-abca-a5c099d7e983)
GPU 2: TITAN RTX (UUID: GPU-e617a03a-7fe6-9418-5a58-3c91ee2f1f1d)
GPU 3: TITAN RTX (UUID: GPU-e3abf33b-367a-9704-26cc-4d97b1502511)
GPU 4: TITAN RTX (UUID: GPU-93100465-8c44-b8a0-869c-4a4fc7d50c57)
GPU 5: TITAN RTX (UUID: GPU-d9d50c3c-8fc3-f2f4-2dc8-e609b7e3075a)
GPU 6: TITAN RTX (UUID: GPU-fc8905f2-1b99-30c8-ea2c-f1b8848a3c8e)
GPU 7: TITAN RTX (UUID: GPU-5a739730-efc4-3460-ecf2-d2206dca0348)
terminate called after throwing an instance of 'std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >'
No CUDA devices
These node have gcc 8.3 and cuda 11.0
Thanks!
[root@localhost gpu-burn-master]# ./gpu_burn 1000
GPU 0: NVIDIA A2 (UUID: GPU-da20406d-5c91-2506-a377-c919f8f6dea1)
Initialized device 0 with 14830 MB of memory (14437 MB available, using 12993 MB of it), using FLOATS
启动gpuburn时 会有一段时间处于Initialized 这个状态, 这个是gpuburn的程序启动嘛? 对卡的状态是否有影响? 我GPU卡是否处于活跃状态?
望回复!谢谢!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.