Pretrained models about piwise HOT 23 OPEN

@saeedizadi
Thanks! I have tried to set num_classes to 1 and got zeros everywhere in the output mask. It turned out that I forgot to apply the Sigmoid to the output. Now, I got very good results. Thank you a lot!

from piwise.

when I set num_class=1 and use NLLLoss() loss, I will encounter an error:
RuntimeError: Assertion cur_target >= 0 && cur_target < n_classes' failed. at /opt/conda/conda-bld/pytorch_1525812548180/work/aten/src/THNN/generic/SpatialClassNLLCriterion.c:111`
@andrewssobral

class UNet(nn.Module):
    def __init__(self, n_channels=3, n_classes=1):
        super(UNet, self).__init__()
        self.down=resnet18()
        self.inc = inconv(n_channels, 16)
        self.down1 = down(64, 128)
        self.down2 = down(128, 256)
        self.down3 = down(256, 512)
        self.down4 = down(512, 512)'''
        self.up1 = up(96, 64,64,drop=0.2)
        self.up2 = up(80, 32,64,drop=0.2)
        self.up3 = up(48, 16,32,drop=0.1)
        self.up4 = up(32, 16,16,drop=0.1)
        self.outc = outconv(16, n_classes)
        self.reset_params()

        criterion=nn.NLLLoss()
        for ii,(data,mask,_)in enumerate(trainloader):
            #训练模型
            data,mask=data.to(device),mask.to(device)
            mask=mask.long()   
            optimizer.zero_grad()
            out=myNet(data)#numclass=1
            out = F.log_softmax(out, dim=1)
            loss=criterion(out,mask)

from piwise.

@saeedizadi maybe has some?

from piwise.

I've modified the models by @bodokaiser for a binary segmentation problems on medical images (using nn.Sigmoid + BCELoss) and hence my pretrained models may not come helpful in general. However, the training is very straight forward on other datasets.
Let me know if you help.

from piwise.

Hi @saeedizadi , can you share what you changed for the binary segmentation? I have set the number of classes = 2 everywhere and used a Sigmoid + BCELoss. However, I got some weird results. How did you modify the output of the net with shape (2xHxW) to match to the mask's shape (1xHxW)?
Thanks!

from piwise.

Hi @ZweeLe
Yes, I wanna share the modifications but I'm a bit out of time these days.
You must change the num_classes to 1 instead of 2. Then, simply change the last layer activation function to nn.Sigmoid() and employ nn.BCELoss() in your code.

Let me know if you need more information.
Saeed

from piwise.

Hi @saeedizadi ,
I am trying to do the same as @ZweeLe (binary segmentation), but I'm lost.
When you talk about:

You must change the num_classes to 1 instead of 2. Then, simply change the last layer activation function to nn.Sigmoid() and employ nn.BCELoss() in your code.

It means:

#NUM_CLASSES = 22
NUM_CLASSES = 1

and

class CrossEntropyLoss2d(nn.Module):

    def __init__(self, weight=None):
        super().__init__()

        #self.loss = nn.NLLLoss2d(weight)
        self.loss = nn.BCELoss(weight)

    def forward(self, outputs, targets):
        #return self.loss(F.log_softmax(outputs), targets)
        m = nn.Sigmoid()
        return self.loss(m(outputs), targets)

?

from piwise.

Hi @ZweeLe ,
Can you share your modifications for binary segmentation? Thanks

from piwise.

I think you are right that binary segmentation only needs 1 output class / channel with 0 corresponding to first and 1 corresponding to second class.

So something like:

model = Net(1)

criterion = nn.BCEWithLogitsLoss()

outputs = model(inputs)

loss = criterion(outputs, targets)

should work. Further check outputs and targets to check if they are compatible with what BCEWithLogitsLoss expects.

from piwise.

Hi @andrewssobral , you just need to follow the above modifications of @bodokaiser. For calculating the loss, the outputs and targets must be 1D arrays:
loss = criterion(outputs.view(-1), targets.view(-1))
To use the nn.BCEWithLogitsLoss() you have to build Pytorch from source. For me I used nn.Sigmoid() and nn.BCELoss() but they are less numerically stable.
For the evaluating code, you will need another post processing code:

outputs = model(inputs)
mask = sigmoid(outputs) * 255
mask[mask < thres] = 0
mask[mask >= thres] = 255

"thres" is the threshold pixel value you set to make a binary mask. You can find out the best value with validation (mine is 120). If you use the default transform then you don't need to multiply with 255.

from piwise.

Thank you @ZweeLe !
When you say:

To use the nn.BCEWithLogitsLoss() you have to build Pytorch from source. For me I used nn.Sigmoid() and nn.BCELoss() but they are less numerically stable.

It means?

class CrossEntropyLoss2d(nn.Module):

    def __init__(self, weight=None):
        super().__init__()

        #self.loss = nn.NLLLoss2d(weight)
        #self.loss = nn.BCEWithLogitsLoss(weight)
        self.loss = nn.BCELoss(weight)

    def forward(self, outputs, targets):
        #return self.loss(F.log_softmax(outputs), targets)
        #return self.loss(outputs, targets)
        return self.loss(nn.Sigmoid(outputs), targets)

from piwise.

@andrewssobral Yes, they are similar. However, the BCELoss only accepts 1D tensors so you have to reshape your tensors or you can define a new BCELoss2d function.

from piwise.

Thank you @ZweeLe

I did your tips, and now the algorithm is running, but the loss becomes negative, please see:

loss: 1.2735891342163086 (epoch: 1, step: 0)
save: segnet-001-0000.pth (epoch: 1, step: 0)
loss: -1.1411563158035278 (epoch: 1, step: 1)
loss: -3.6113040447235107 (epoch: 1, step: 2)
loss: -4.822791635990143 (epoch: 1, step: 3)
loss: -6.884974336624145 (epoch: 1, step: 4)
loss: -7.026113708813985 (epoch: 1, step: 5)
loss: -8.921686070305961 (epoch: 1, step: 6)
loss: -10.695589691400528 (epoch: 1, step: 7)
loss: -11.778627316157023 (epoch: 1, step: 8)
loss: -13.278458523750306 (epoch: 1, step: 9)
loss: -14.816673040390015 (epoch: 1, step: 10)
loss: -16.244035343329113 (epoch: 1, step: 11)
loss: -17.84725154363192 (epoch: 1, step: 12)
loss: -19.353035739489965 (epoch: 1, step: 13)
loss: -21.03988642692566 (epoch: 1, step: 14)
loss: -22.63021345436573 (epoch: 1, step: 15)
loss: -23.58169169986949 (epoch: 1, step: 16)
loss: -25.269151012102764 (epoch: 1, step: 17)
loss: -26.97707804880644 (epoch: 1, step: 18)
loss: -28.725287783145905 (epoch: 1, step: 19)
loss: -29.03264051391965 (epoch: 1, step: 20)
loss: -30.018881505185906 (epoch: 1, step: 21)
loss: -31.15997344514598 (epoch: 1, step: 22)
loss: -32.05112152298292 (epoch: 1, step: 23)
loss: -33.00953503608704 (epoch: 1, step: 24)
loss: -35.36375263104072 (epoch: 1, step: 25)
loss: -36.99868559837341 (epoch: 1, step: 26)
loss: -39.33454824345453 (epoch: 1, step: 27)
loss: -41.40287765141191 (epoch: 1, step: 28)

this is normal?

from piwise.

@andrewssobral That is weird, the loss must be positive and decrease over time. You should check your outputs of the model and the targeted masks before applying BCELoss. All values should be in range of 0~1. Did you apply the transformation to the labels?

from piwise.

Hi @ZweeLe ,
I solved this issue by changing this:

input_transform = Compose([
    CenterCrop(256),
    ToTensor(),
    Normalize([.485, .456, .406], [.229, .224, .225]),
])
target_transform = Compose([
    CenterCrop(256),
    ToLabel(),
    Relabel(255, 21),
])

to this:

input_transform = Compose([
    Scale((64,64)),
    ToTensor(),
])
target_transform = Compose([
    Scale((64,64)),
    ToTensor(),
])

Now, the algorithm is learning well, the loss is decreasing and the preliminary results are very good! Thank you! and thanks to @bodokaiser for your advices ;-)

from piwise.

@andrewssobral Yes. The ToLabel() and Relabel() are not necessary for binary segmentation. Can you share your training time? For me, it took nearly 5 secs for an iteration and 0.5s to evaluate an image, which is a bit slow since I have a large dataset.

from piwise.

@ZweeLe as I have only 126 images and I'm using Amazon EC2 with p2x.large instance (NVIDIA K80 with 12Gb of RAM), the training time is very fast (less than 5 minutes). In my laptop with only CPU, it is much more slow, I stopped the training berfore 10 epochs. I think the training will take around (or more than) 30min for the whole dataset.

from piwise.

Can I directly use nn.CrossEntropyLoss() for binary segmentation? Need I set num_class=1 ? And I want use weighed loss. @andrewssobral @duylp @saeedizadi @bodokaiser

from piwise.

You would suggest to use Sigmoid() + BinaryCrossEntropy for Binary Segmentation.

from piwise.

@saeedizadi I don't understand how to set the parameter weight for :
class torch.nn.BCELoss(weight=None, size_average=None, reduce=None, reduction='elementwise_mean')

from piwise.

@woaichipinngguo You should see the definitions for the parameters. But, for the initial try, just try to use the default values, and see the results, then, you can customise them toward your taste

from piwise.

Hi @woaichipinngguo , you can find here an example of training binary segmentation on pytorch here:
https://github.com/andrewssobral/deep-learning-pytorch/blob/master/segmentation/train_binseg.py
this code was based on bodokaiser/piwise repository. However, in the code I don't make use of weights, and some improvements (e.g. model selection, prediction, etc) are needed, I still working to improve it.

from piwise.

@woaichipinngguo You may have a label index overpassing the limit. Cross-entropy needs at least 2 classes.

from piwise.