Comments (9)
The reason the variable is there is to defeat broken vendor caches. I don't remember specifics, but in PyOpenCL's early days, I spent a long time tracking down what ended up being a bug in an ICD compiler cache. The ICD compiler did not notice that a header file included by the source was changed, and insisted on using a (stale) cached binary. That variable definition was there to help "convince" ICDs that they're looking at new source code every time, while PyOpenCL's own caching system is (hopefully) less broken than the ones built into the ICD. That said, for some specific ICDs that (competently) do their own caching, PyOpenCL's caching system imposes unnecessary overhead, which we're now thinking of (selectively) removing. See #738 for some discussion.
from pyopencl.
I'm a bit puzzled why this behavior should be different between PyOpenCL and a C++ program calling OpenCL directly. One possible reason that these messages got cached from an old version of the driver. You can check for this by deleting PyOpenCL's build cache:
# Careful! Double check this command before running it, to ensure it does what you intend.
rm -Rf $HOME/.cache/pyopencl
and then rerunning.
from pyopencl.
FWIW, I was not able to reproduce this with CUDA 12.2 on Debian unstable building for a TITAN X.
from pyopencl.
I'm a bit puzzled why this behavior should be different between PyOpenCL and a C++ program calling OpenCL directly. One possible reason that these messages got cached from an old version of the driver. You can check for this by deleting PyOpenCL's build cache:
# Careful! Double check this command before running it, to ensure it does what you intend. rm -Rf $HOME/.cache/pyopencland then rerunning.
Thanks, tried it, but unfortunately didn't work for me.
I also verified that compiled C code and pyopencl are indeed using the same opencl library. With strace
I see that both programs open the following library:
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/x86_64/x86_64/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/x86_64/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/x86_64/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "x86_64/x86_64/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "x86_64/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "x86_64/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=84758, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 84758, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f35713dd000
close(3) = 0
openat(AT_FDCWD, "/usr/local/cuda/targets/x86_64-linux/lib/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = 3
I really have no idea why this message is originated.
from pyopencl.
Finally I found it. After inserting print statements to the pyopencl C++ code, I noticed that the following line was added to the kernel source:
__constant int pyopencl_defeat_cache_14d61c4d6ee748c9a9cef2d50121f8ef = 0;
If I remove such line (modifying pyopencl C++ code) I don't get anymore the compiler warning. Adding such line to my C++ opencl kernel, makes me getting the same build log. So that's the reason in the end, and pyopencl is consistent with C/C++ opencl interface.
Update: this is not the real cause.
from pyopencl.
Interesting! Thanks for tracking this down, I had forgotten about that. :) I still kind of don't understand why having this triggers the warning it does; the warning seems entirely unrelated to that variable definition?
from pyopencl.
Yea, the compiler message seems totally unrelated to that variable, yet it’s anyway triggered by it. Don’t know what nvidia is doing here; we all know OpenCL is not bvidia top priority (to say an euphemism).
Btw, for curiosity, why that constant variable is added to the kernel source code? Looks like it’s related to pyopencl caching system. Is there an easy way to disable it?
from pyopencl.
I now realized I gave a wrong explanation. I've been get confused by (I think) some build caching mechanism that the nvidia ICD compiler is apparently using. From C interface, the ICD compiler builds the kernel, and get non empty build info, only the first time I execute the program, while in all subsequent program executions clBuildProgram
(I think that) uses some cache and clGetProgramBuildInfo
returns an empty message only because (I think that) the build info are not cached.
In conclusion, the line below is not triggering the non-empty build log. I do get a non-empty build log even from the C interface, though only the first time I compile a kernel (I suppose due to the caching mechanism mentioned above).
__constant int pyopencl_defeat_cache_14d61c4d6ee748c9a9cef2d50121f8ef = 0;
In the end, nothing is due to pyopencl.
from pyopencl.
Glad to hear everything got resolved. I'll go ahead and close this issue, LMK if anything else comes up.
from pyopencl.
Related Issues (20)
- test_clmath.py::test_fmod fails on i386 using pocl 3.1 built with llvm 15 HOT 2
- Disable kernel caching from within pyopencl code HOT 1
- Expose create_buffer_gc in a header HOT 4
- Binary operations with Arrays of different memory layout HOT 2
- Windows Intel CL Github CI fails HOT 2
- Fails to build with OpenCL 3.0 headers HOT 3
- UHD Graphics 600 | Calling kernel + enqueue_copy more than once, results in OUT_OF_RESOURCES error or freeze HOT 4
- Compatibility with `numpy2` HOT 3
- move away from deprecated appdirs
- `build program` times increasing with rank count on Mac when caching is enabled HOT 7
- Pocl Mac crashing again HOT 4
- `CMakeLists.txt` file missing in PyPI source dist HOT 2
- `pip install .` causes messy directory layout HOT 3
- Nanobind leak warnings HOT 9
- Kernel evaluation throws exception for SVM objects HOT 1
- pyopencl.compyte missing in pyopencl-2024.2.4.tar.gz HOT 3
- 2024.2.5 is Broken HOT 5
- Wheels built with numpy2 HOT 3
- Invoker cache created and written to even when PYOPENCL_NO_CACHE=1 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyopencl.