• Correct and fill up README.md.
• Variable naming

  • lr confusion
    

    VDSR-pytorch-tf2/pytorch/train.py

    Line 50 in 33618ee

    lr_scheduler = ReduceLROnPlateau(optimizer, 'min')

    
    

    VDSR-pytorch-tf2/pytorch/train.py

    Line 62 in 33618ee

    lr = tr_data['LR'].to(device)

  • etc...

• Choosing best PSNR
  

  VDSR-pytorch-tf2/pytorch/train.py

  Line 55 in 33618ee

  best_psnr = 0

• Efficient validation code

• Remove unused code in ops.py

• Adding super.init inside dataset?

  Not needed. (https://pytorch.org/docs/stable/_modules/torch/utils/data/dataset.html#Dataset)

• Implement dataset lazy loading with resizing. And change custom dataset name.

• Use pytorch native functions for transformation

  Not recommended (Not convenient for SR task)

• Handle scale variable.

• Test detatch function

  https://redstarhong.tistory.com/64

There is a very nasty problem involving pytorch DataLoader. In the log below, you notice that dataloader delay is 6s while the overall epoch takes only 8s. Tensorflow2 only takes 3s for one epoch on same settings(TODO: make tf2 time consumption test code).

Log: (in branch pytorch_dataset_hanging_problem)

[*] Start training

***> dataloader delay: 5.8484
>> end of iteration (time interval: 0.0408) ) 


[2020-04-06 22:22:14]
>> epoch 1       loss: 0.045038 (lr: 1.00e-03)

***> dataloader delay: 6.1289
>> end of iteration (time interval: 0.0431) ) 


[2020-04-06 22:22:26]
>> epoch 2       loss: 0.001379 (lr: 1.00e-03)

Possible scenarios:

1. On every start of epoch, there is a heavy operation done by DataLoader. For example, remaking threads, thus re-copying each resources.

2. Iteration doesn't start until each worker completes on batch(=32).

Useful links:

https://discuss.pytorch.org/t/dataloader-with-num-workers-1-hangs-every-epoch/20323/9
https://discuss.pytorch.org/t/dataloader-resets-dataset-state/27960/4
https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader