Comments (4)
你好 可以提供你的机器配置和gcc版本吗?
from mononerd.
机器配置: 4 × A100-SXM4-40GB, Driver Version: 530.30.02, CUDA Driver Version: 12.1
gcc: 9.5.0
torch: 1.8.1+cu111
网络构建完成, 可以运行到开始训练部分, 然后就程序就退出了:
epochs: 0%| | 0/60 [00:00<?, ?it/s]{'NAME': 'filter_truncated', 'AREA_RATIO_THRESH': None, 'AREA_2D_RATIO_THRESH': None, 'GT_TRUNCATED_THRESH': 0.98}
filter truncated ratio: null 3d boxes [[ 2.99 -3.87 -0.66499996 4.43 1.84 1.75
-0.2907964 ]] flipped False image idx 890 frame_id 001773
{'NAME': 'filter_truncated', 'AREA_RATIO_THRESH': None, 'AREA_2D_RATIO_THRESH': None, 'GT_TRUNCATED_THRESH': 0.98}
filter truncated ratio: null 3d boxes [[ 2.93 -4.66 -0.73 4.18 1.86 1.48
-1.6307963]] flipped False image idx 1040 frame_id 002080
并且伴随警告:
/home/user/anaconda3/envs/mono3d/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:129: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
from mononerd.
/home/user/anaconda3/envs/mono3d/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 29 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
这个警告可以监测一下运行时cpu和内存利用率,调小dataloader的num_works。或者尝试用
export PYTHONWARNINGS='ignore:semaphore_tracker:UserWarning'
忽略
/home/user/anaconda3/envs/mono3d/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:129: UserWarning: Detected call of
lr_scheduler.step()
beforeoptimizer.step()
. In PyTorch 1.1.0 and later, you should call them in the opposite order:optimizer.step()
beforelr_scheduler.step()
. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call oflr_scheduler.step()
beforeoptimizer.step()
. "
这个警告不影响我们这边的设备正常训练
另外请问你在安装spconv-1.2.1的时候没有遇到问题吗?之前我们也尝试了几次在A100裸机上配环境,一直没有成功;像这样能够正常跑起来的情况倒是第一次见。
你把程序退出时候的详细log发我下,再给我个邮箱,我把我用的dockerfile发你
from mononerd.
已发邮件, 谢谢
from mononerd.
Related Issues (12)
- Help, "Segmentation fault" occur at "x = self.conv_input(input_sp_tensor)" HOT 1
- 运行错误 HOT 15
- Running on Colab HOT 2
- Do you have mono configs for KITTI? HOT 1
- Why monocular methods require the stereo information (of kitti dataset)? HOT 5
- The AP on KITTI val set HOT 1
- How to run on a single gpu HOT 1
- How to run inference on a single gpu? HOT 2
- How to get the score of all classes? HOT 5
- Could you provide the pretrained model and its config file of the Waymo dataset? HOT 1
- 请问在哪里下载ImageSet这个文件夹?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mononerd.