I tried to use os.environ['CUDA_VISIBLE_DEVICES'] = '0,1' in my python script.When running the script from the command line, it simply uses 1 GPU only.I also use nn.DataParallel(model), but get error "'DataParallel' object has no attribute 'req_grad_params'"

Excuse me, what is the function of this code?

for batch_idx, pl in enumerate(premise_len):
        h_s[pl:, batch_idx] *= 0.

1. I checked the output of pad_packed_sequence(h_s) and there are zeros for padding. (https://gist.github.com/HarshTrivedi/f4e7293e941b17d19058f6fb90ab0fec)

2. Also, when pl==premise_max_len, it causes "RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)"

tr_loader, _, _ = snlidata.get_dataloaders(batch_size=32, num_workers=0, pin_memory=torch.cuda.is_available())

if I do not set the num_workers as 0, I will get this BrokenPipeError. Does anyone have an idea about it?

I trained according to your training steps, but the difference between the training results and the report is very big and I am puzzled. Thank you.

Sorry, I found that your code did not consider the problem of oov when constructing the vocabulary. When there is a word in the new test variant data that is not in your vocabulary, there is no corresponding unknow idx corresponding to it.

Hi,

I am kind of confused about the use or torch.multiprocessing. As far as my understanding, each batch is trained twice on the model. So training 10 epochs by 2 processes is essentially training 20 epochs. Is this correct? Since I've never used torch.multiprocessing, could you please give me more explanations about the use of it in your code? I'd appreciate it.

Best,
Zihao