Comments (10)
That's really useful information, thank you!
I have a very similar setup here with the main difference being that I only have a single RTX 3090. This leads me to believe that the automatic GPU architecture detection/treatment in the CMake project file is problematic when multiple GPUs with different architectures are present.
To verify, could you try the following two steps?
- Could you try building with
make VERBOSE=1
and paste the full log here? - Please try replacing the block
include(FindCUDA/select_compute_arch)
cuda_select_nvcc_arch_flags(ARCH_FLAGS)
list(APPEND CUDA_NVCC_FLAGS ${ARCH_FLAGS})
set(CMAKE_CUDA_ARCHITECTURES off)
with the single line
set(CMAKE_CUDA_ARCHITECTURES 86)
in CMakeLists.txt and see if that fixes the problem on your end.
Thank you!
from tiny-cuda-nn.
Hi there, more information about your system would be helpful:
- Which OS are you running? And on which version?
- Which GPUs are installed in the system?
- Which version of CUDA and which version of the NVIDIA driver is installed?
from tiny-cuda-nn.
Another one that might be helpful, actually: which version of CMake are you running? (This is useful, because CMake has recently changed a lot about their CUDA support.)
from tiny-cuda-nn.
Ah yes sorry... That would have been smart of me.
Ubuntu 20.04
RTX 3090
RTX 4000
CUDA 11.5 + 495.44
cmake version 3.21.0-rc3
I note that if I role back to commit cdb419d it builds and runs as expected.
from tiny-cuda-nn.
Replacing the block of code did fix the issue..... Thanks for this great bit of code!
from tiny-cuda-nn.
Hi @rathken . Could you try compiling again with the latest version of this repository? I've tried to fix the multi-GPU issue and would like to make sure. Thanks!
from tiny-cuda-nn.
Will do today.
from tiny-cuda-nn.
It took a while to build, but it built without any issues....
from tiny-cuda-nn.
Amazing, thank you very much for testing!
from tiny-cuda-nn.
Hi everyone.
I have a similar problem,
gmake[2]: *** [dependencies/tiny-cuda-nn/CMakeFiles/tiny-cuda-nn.dir/build.make:216: dependencies/tiny-cuda-nn/CMakeFiles/tiny-cuda-nn.dir/src/fully_fused_mlp.cu.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:288: dependencies/tiny-cuda-nn/CMakeFiles/tiny-cuda-nn.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
but a don't find the block of code that your comment in CMakelist.txt:
include(FindCUDA/select_compute_arch)
cuda_select_nvcc_arch_flags(ARCH_FLAGS)
list(APPEND CUDA_NVCC_FLAGS ${ARCH_FLAGS})
set(CMAKE_CUDA_ARCHITECTURES off)
Only I find the:
set(CMAKE_CUDA_ARCHITECTURES ${TCNN_CUDA_ARCHITECTURES})
and replace for:
set(CMAKE_CUDA_ARCHITECTURES 86)
And the new error is similar to the first one:
gmake[2]: *** [dependencies/tiny-cuda-nn/CMakeFiles/tiny-cuda-nn.dir/build.make:216: dependencies/tiny-cuda-nn/CMakeFiles/tiny-cuda-nn.dir/src/fully_fused_mlp.cu.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:288: dependencies/tiny-cuda-nn/CMakeFiles/tiny-cuda-nn.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
I have:
Ubuntu 22.04 LTS
RTX 3080 ti mobile
NVIDIA-SMI 515.48.07
Driver Version: 515.48.07
CUDA Version: 11.7
CMake version 3.22.1
Thx for your advice
from tiny-cuda-nn.
Related Issues (20)
- Different learning rate per encoding level
- cannot do `pip install` for torch. HOT 1
- 'tiny cuda nn' issue HOT 1
- something to do with tinycudann
- Backward method of the grid encoding
- How to calculate FLOPs?
- Is the RTX4070ti supported?
- install issue HOT 6
- Add auxiliary losses directly imposed on params HOT 1
- README executable instructions include non-executable shell prompts
- Question about the bounding box
- initiailization of hash grid
- tinycudann ImportError: tinycudann_bindings/_80_C.cpython-38-x86_64-linux-gnu.so: undefined symbol: HOT 2
- Enable 5D grids HOT 2
- Link Against tiny-cuda-nn in C++ Program HOT 1
- [Question]: can tiny-cuda-nn build a network with layer's bias=0?
- Problems encountered during installation HOT 1
- Already setted the CUDA_HOME but still:CUDA_HOME environment variable is not set. Please set it to your CUDA install root. HOT 1
- Manual installation with torch extension fails: parameter packs not expanding after cmake build success HOT 1
- Tiny cuda nn compilation issue
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tiny-cuda-nn.