Blank output maps! about piwise HOT 9 OPEN

Hi,
That's so weird! so so weird!!
Although there are some problems with your UNet implementation, but I found that the blank issue does not arise from UNet, or generally model since I used the implementations by others while using the rest of your code, and the problem exists!!

I'm going to spend some time on the code. Will update you on this

Thanks
Saeed

from piwise.

I finally found the problem!!
For the last set of convolutions, that is 128-> 64 -> 64 -> 1, the activation function should not be used!
The activation function causes the values to vanish!

I just removed the nn.ReLU() modules on top of these convolution layers and now everything works like a charm!

Saeed

from piwise.

Hi,

yes see this issue.

You can also check out semseg as stated in the README.

I cannot say why there is so bad convergence with my implementation(s) - if I would know I would fix it believe me. That said different researchers tried their luck and did not find the problem - though I don't want to preclude that it could be something really simple (i.e. wrong sign on the loss).

As this is not really my research area - I initially just wrote it together to cross check something else - there is low probability that I will find a solution to this in the near future. I also lack on hardware resources at the moment.

I would be really happy if someone would find the mistake or just PR a complete rewritten u-net, however I am aware that also other people are limited on time.

So to cut a long story short: checkout semseg and/or double check my u-unet implementation if you find something (on u-net or the loss).

Good luck!

from piwise.

from piwise.

Sure,
I will do a pull request

from piwise.

@saeedizadi I am now working on a different project which utilizes the u-net architecture and found some solutions:

• Use leaky relus in the encoder/downsample part
• Small learn rate (i.e. 1e-4)
• Use transposed Convolutions

More changes which may improve performance:

• Use per image standardization or normalize image to
• Use Xavier initialization on convolution weights

from piwise.

@bodokaiser thank you for your information :)

from piwise.

Hi, Saeedizadi

I seems like I got the similar problem with yours. What's the meaning of your: blank output maps?
My forward result will be into one label at every iteration, like: all pixels have the same prediction label.

from piwise.

Hi JiayuanSternLi
By blank image, I meant an image in which all pixel labels are 0 or very small values, e.g. 0.01 in the range of [0,1].

from piwise.