Comments (4)
sjeaugey
commented on July 19, 2024
1
The most probable reason is that you still have some NCCL 1.x include somewhere (like in /usr/local/include, or somewhere else) which is picked up by your compilation.
from nccl-tests.
sjeaugey
commented on July 19, 2024
Humm.. could it be you are trying to compile the NCCL tests against a very old NCCL 1.x ?
from nccl-tests.
HeyangQin
commented on July 19, 2024
According to apt I have nccl 2.8.3 yet I still have this issue...
(base) (heyangq)heyangq@node10:~/nccl-tests$ make
make -C src build
make[1]: Entering directory '/home/heyangq/nccl-tests/src'
Compiling all_reduce.cu > ../build/all_reduce.o
Compiling common.cu > ../build/common.o
common.cu:336:38: error: missing binary operator before token "("
#if NCCL_VERSION_CODE >= NCCL_VERSION(2,4,0)
^
Makefile:82: recipe for target '../build/common.o' failed
make[1]: *** [../build/common.o] Error 1
make[1]: Leaving directory '/home/heyangq/nccl-tests/src'
Makefile:17: recipe for target 'src.build' failed
make: *** [src.build] Error 2
(base) (heyangq)heyangq@node10:~/nccl-tests$ apt search nccl
Sorting... Done
Full Text Search... Done
libhttpasyncclient-java/xenial 4.1-1 all
HTTP/1.1 compliant asynchronous HTTP agent implementation
libnccl-dev/unknown,now 2.8.3-1+cuda11.0 amd64 [installed]
NVIDIA Collective Communication Library (NCCL) Development Files
libnccl1/unknown 1.2.3-1+cuda8.0 amd64
NVIDIA Communication Collectives Library (NCCL) Runtime
libnccl2/unknown,now 2.8.3-1+cuda11.0 amd64 [installed,automatic]
NVIDIA Collective Communication Library (NCCL) Runtime
libvncclient1/xenial-updates,xenial-security 0.9.10+dfsg-3ubuntu0.16.04.6 amd64
API to write one's own VNC server - client library
libvncclient1-dbg/xenial-updates,xenial-security 0.9.10+dfsg-3ubuntu0.16.04.6 amd64
debugging symbols for libvncclient
python-ncclient/xenial 0.4.7-1 all
Python library for NETCONF clients
texlive-latex-extra/xenial 2015.20160320-1 all
TeX Live: LaTeX additional packages
xvnc4viewer/xenial 4.1.1+xorg4.3.0-37.3ubuntu2 amd64
Virtual network computing client software for X
from nccl-tests.
HeyangQin
commented on July 19, 2024
The most probable reason is that you still have some NCCL 1.x include somewhere (like in
/usr/local/include, or somewhere else) which is picked up by your compilation.
Thanks for the prompt reply. You are right. There is an ancient nccl.h in my /usr/local/include. After removing it and reinstalling libnccl-dev the problem is gone!
from nccl-tests.
Related Issues (20)
- How to explain Bus Bandwidth in Allreduce Operation?
- How is the maximum number of bytes for all_reduce operation calculated? HOT 3
- Interaction between NCCL_IB_SL and NCCL_IB_ADAPTIVE_ROUTING
- Multi node test hang phenomenon HOT 2
- Performance lack of NCCL Test HOT 5
- Why getBw don't have access to agg_iters ? HOT 1
- Test NCCL failure common.cu:961 'internal error - please report this issue to the NCCL developers / ' HOT 7
- cputime
- misc/ibvwrap.cc:278 NCCL WARN Call to ibv_reg_mr_iova2 failed with error Cannot allocate memory HOT 2
- make failed, error -- unsupported GNU version! gcc versions later than 11 are not supported!
- Test NCCL failure common.cu:954 'unhandled cuda error HOT 1
- The network bandwidth in the alltoall_perf test failed to meet expectations HOT 3
- Differences problems in performance data of HGX A800 single server N GPUs nccl testing
- undefined reference nccl* HOT 1
- H100 all reduce performance is poor HOT 13
- Nccl test seems run seperately on multi nodes HOT 6
- SendRecv Time HOT 2
- NCCL_ALGO on multi-node and multi-GPU HOT 1
- NCCL initialization hangs with 4 GPUs, but works with 2 GPUs HOT 4
- all_reduce_perf hangs; using a single GPU on a 4GPU machine HOT 18
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nccl-tests.