Comments (14)
Yes, that would fix all multiprocessing related issues at the expense of not using multiprocessing at all. I think you can get away with setting it to 2 on Colab. Setting it to 2 should speed things up a bit but setting it to 1 will ensure that you won't get multiprocessing related errors.
from pytorch-transformers-classification.
I thought I fixed this, sorry.
In this function definition, change the default value of process count to 1 for Google Colab. (Colab has 1 vCPU if I remember correctly).
Edit: Couldn't reproduce the error when I ran the notebook. The Colab notebook specifies a process count of 2 when calling convert_examples_to_features()
from pytorch-transformers-classification.
So this change is only for Colab? If I implement it in my local laptop, can I use the one that is in the repo?
from pytorch-transformers-classification.
Yes, the local version will work fine. By default, the process_count is set to number of CPU cores available - 2
. On a modern computer, you will certainly have more than 2 so it's fine. But for Colab, the number is 1 so that makes the process_count -1
, which throws the error because at least 1 process is needed.
Edit: The cpu_count on Colab is two, and the notebook is configured to use 2 as the process_count.
from pytorch-transformers-classification.
I ran the Colab notebook again, it works without issues. The above change is unnecessary.
from pytorch-transformers-classification.
Mmm and what could it be? I just run it again and got the same error. These are the resources that I have allocated:
from pytorch-transformers-classification.
Try setting process_count to 1 in the call to convert_examples_to_features()
inside the load_and_cache_examples()
function.
features = convert_examples_to_features(examples, label_list, args['max_seq_length'], tokenizer, output_mode,
cls_token_at_end=bool(args['model_type'] in ['xlnet']), # xlnet has a cls token at the end
cls_token=tokenizer.cls_token,
sep_token=tokenizer.sep_token,
cls_token_segment_id=2 if args['model_type'] in ['xlnet'] else 0,
pad_on_left=bool(args['model_type'] in ['xlnet']), # pad on the left for xlnet
pad_token_segment_id=4 if args['model_type'] in ['xlnet'] else 0,
process_count=1)
from pytorch-transformers-classification.
I got this error when I changed it:
Anyway, let me review the code, because I have been disconnected last days from this, so I want to check that I have been following all the steps correctly.
from pytorch-transformers-classification.
Are you using a local copy (local to your Google Drive, that is)? I think this bug was there in the original notebook, but it was fixed later. The function should look like this:
def convert_examples_to_features(examples, label_list, max_seq_length,
tokenizer, output_mode,
cls_token_at_end=False, pad_on_left=False,
cls_token='[CLS]', sep_token='[SEP]', pad_token=0,
sequence_a_segment_id=0, sequence_b_segment_id=1,
cls_token_segment_id=1, pad_token_segment_id=0,
mask_padding_with_zero=True,
process_count=cpu_count() - 2):
""" Loads a data file into a list of `InputBatch`s
`cls_token_at_end` define the location of the CLS token:
- False (Default, BERT/XLM pattern): [CLS] + A + [SEP] + B + [SEP]
- True (XLNet/GPT pattern): A + [SEP] + B + [SEP] + [CLS]
`cls_token_segment_id` define the segment id associated to the CLS token (0 for BERT, 2 for XLNet)
"""
label_map = {label : i for i, label in enumerate(label_list)}
examples = [(example, label_map, max_seq_length, tokenizer, output_mode, cls_token_at_end, cls_token, sep_token, cls_token_segment_id, pad_on_left, pad_token_segment_id) for example in examples]
with Pool(process_count) as p:
features = list(tqdm(p.imap(convert_example_to_feature, examples, chunksize=100), total=len(examples)))
return features
from pytorch-transformers-classification.
Yeah maybe is that. I know is better to fork to your repo so we have the updates/fixes instantly, but I like first to understand the code recreating it in my own notebook. I am using Colab linked to Google Drive. I will check that and I will let you know. Thanks.
from pytorch-transformers-classification.
Understandable! Let me know how it goes.
from pytorch-transformers-classification.
With fresh head is more easy to see clearly. There were two things that I changed and make it worked:
- When making dataframes BERT friendly I didn't pass the parameter
columns=train_df_bert.columns
to theto.csv
function. - In the function
load_and_cache_examples
I was not including the colab add-in parameterundersample_scale_factor=0.1
.
Maybe the first point was the one causing the error? Anyway sorry for my lapsus, I will keep iterating it and let you know if I see any suspicious bug.
from pytorch-transformers-classification.
Weird. Neither of those things should be throwing a "number of processes" error as far as I can tell. That error comes from the multiprocessing used for converting examples to features. Oh well, we don't need to worry about it if it's working!
from pytorch-transformers-classification.
I was facing again this problem with another iteration and I changed this:
process_count = cpu_count() - 2
for this
process_count = 1
in the function convert_examples_to_features
of the utils.py
file, and it worked. I am working on Colab. It makes sense to you?
from pytorch-transformers-classification.
Related Issues (20)
- File path problem HOT 3
- AttributeError: module 'torch.nn.functional' has no attribute 'one_hot' HOT 1
- How to use other models without loading them from internet? HOT 3
- Model performance degrades when moved to Multi-GPU HOT 5
- Not An issue- Adding metadata HOT 3
- Eval results outputs HOT 2
- where is the positional embedding in the Bert model inputs
- Regarding number of train test samples HOT 5
- utils.py [output_mode = regression] HOT 3
- ImportError: cannot import name 'amp' HOT 1
- passing args HOT 2
- Running Inference HOT 2
- 'math' is not defined HOT 1
- ERROR:pytorch_transformers.tokenization_utils:Couldn't reach server to download vocabulary. HOT 2
- Num epoch and other standard configurations HOT 5
- How to make predications HOT 7
- UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. HOT 2
- can I have more column in train set HOT 10
- RuntimeError: Trying to create tensor with negative dimension -1: [-1, 768] HOT 2
- Update prerec packages info: transformer and apex? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-transformers-classification.