Comments (3)
Hi,
Thanks for letting me know the issue. This seems to be an error related to PyTorch distributed training. Unfortunately, I cannot reproduce this error on my machines and I don't have any ideas regarding why this is happening. Probably you could try to add export PYTHONWARNINGS='ignore:semaphore_tracker:UserWarning'
to the agnews.sh
script as suggested here. Alternatively, you could modify the code to remove the distributed training part as discussed here. If you have a V100 GPU, you probably won't need to train on multiple GPUs.
Thanks,
Yu
from lotclass.
Hi,
Thanks for letting me know the issue. This seems to be an error related to PyTorch distributed training. Unfortunately, I cannot reproduce this error on my machines and I don't have any ideas regarding why this is happening. Probably you could try to add
export PYTHONWARNINGS='ignore:semaphore_tracker:UserWarning'
to theagnews.sh
script as suggested here. Alternatively, you could modify the code to remove the distributed training part as discussed here. If you have a V100 GPU, you probably won't need to train on multiple GPUs.Thanks,
Yu
Hello, when I tried to use the code you provided today, I also reported a ‘bus error’ error. My environment configuration is similar to that of the questioner, single GPU processing. I would like to ask if this program will take up a lot of CPU? When I am running, the CPU usage is often at 100+%. I don't know what is going on? I have reduced the data volume and batch_size. Do you have any suggestions here?
from lotclass.
Hi @wws0815,
The code will not use CPU heavily except at the beginning when preparing the input tensors (you could see the code print out something like "Converting texts into tensors." or "Reading texts from..."). Later when the training begins (i.e., after you see "Constructing category vocabulary."), the code should mainly use GPUs for model training. If you are still seeing high CPU usage, probably you will need to make sure the code is using GPUs for training.
Thanks,
Yu
from lotclass.
Related Issues (20)
- Some issues for duplicating the paper results HOT 3
- CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasCreate(handle)` HOT 2
- [Improvement] More post-processing for the category vocabulary HOT 2
- Category Vocabulary size in paper is 50, the category vocabulary size created by the algorithm is 70 HOT 1
- Torch not compiled with CUDA enabled HOT 3
- In w/o self-training baseline, how are the final test predictions made if the [CLS] hasn't be trained? HOT 2
- pretrained_lm for non english data HOT 1
- Bug exists when using the original data and codes HOT 1
- Error running with test set HOT 3
- Can this method be used in the scenario of multi-label text classification? HOT 1
- Can I train on cpu and control layer freezing ? HOT 1
- AssertionError: Too few (0) documents with category indicative terms found for category 1; try to add more unlabeled documents to the training corpus (recommend) or reduce `--match_threshold` (not recommend) HOT 4
- My performance in reproducing the results of agnews using the code you provided is relatively poor
- Category Vocabulary Creation
- FileNotFoundError: [Errno 2] No such file or directory: 'datasets/agnews/label_names.txt' HOT 3
- is possible to pass muti name for one category HOT 2
- EnvironmentError when running code HOT 1
- RuntimeError: The task could not be sent to the workers as it is too large for `send_bytes`. HOT 3
- RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lotclass.