Apply to 5D tensor? about pytorch-msssim HOT 18 CLOSED

@John1231983 Just use a view with the batch and channel dimensions merged together. Tensor.view(-1, D, H, W) will work.

from pytorch-msssim.

@John1231983 Are you using merged batch and channel dimensions or merged width and height dimensions? You must use merged batch and height or your results will be incorrect. The 2D convolution applies to the last 2 dimensions, while working in parallel for the other dimensions.
Using merged height and width will return nonsense outputs, probably explaining the negative values.

Also, a view does not copy memory, so there is almost no extra time needed for returning a new view.

Finally, please note that using merged batch and channel dimensions will give a SSIM for each slice of depth. It will be the average of the SSIM of the 2D images for the 3D volume. If you desire a SSIM for the depth/height axis or depth/width axis, you must reshape your 5D tensor appropriately.

from pytorch-msssim.

Hi,
I'm not very familiar with 4D images, but MSSSIM & SSIM modules require channel as a parameter.
I hope the following example is useful.

from pytorch_msssim import *
import torch 

s = SSIM(data_range=1., channel=2)

a = torch.randint(0, 255, size=(10, 2, 32, 32, 32), dtype=torch.float32) / 255.
b = a * 0.8
B,c,d,h,w = a.size()
a = a.view(B,c,d,-1)
b = b.view(B,c,d,-1)
print(s(a, b))

outputs:

tensor(0.9520)

from pytorch-msssim.

@John1231983 you should take -1*ssim or 1-ssim as loss rather than ssim.
I made a mistake in previous comments. The correct loss should be like this:

loss1 = 1-msssim(output+0.5, input+0.5, data_range=1.0) # out and target are in range of [-0.5, 0.5]
loss2 = other_loss(output, ...)

from pytorch-msssim.

@VainF : Perfect! it worked
@veritas9872 : Which one should be faster merge batch and channel or merge height and width

view(-1,D,H,W)

or

view(B,C,D,-1)

from pytorch-msssim.

@VainF : I applied your loss to the cyclegan that applied in cycleloss. The range of image is [-0.5,0.5], I have change

s = SSIM(data_range=0.5, channel=2)

But the loss becomes a negative value, sometimes positive. Is it normal?

from pytorch-msssim.

@VainF : I applied your loss to the cyclegan that applied in cycleloss. The range of image is [-0.5,0.5], I have change

s = SSIM(data_range=0.5, channel=2)

But the loss becomes a negative value, sometimes positive. Is it normal?

data_range=0.5 means [0, 0.5]
you can change the range by adding 0.5 to outputs and set data_range=1.0

from pytorch-msssim.

Is it possible to change data_range to accept from [-0.5, 0.5] in the ssim loss. I cannot add 0.5 because it will change the other losses that are using the range [-0.5,0.5]

from pytorch-msssim.

I think adding 0.5 will not affect other losses.

loss1 = msssim(output+0.5, input+0.5, data_range=1.0) # out and target are in range of [-0.5, 0.5]
loss2 = other_loss(output, ...)

from pytorch-msssim.

Thanks. I have tried it. But the loss still negative value. Any reason? Not that, other losses are normal

from pytorch-msssim.

Nice comment @veritas9872 . I will try your way. Could you tell me which win_size do you often use? I use 11 and the SSIM hurt my performance (performance down although the loss always positive value)

from pytorch-msssim.

@John1231983 Sorry for the late replay.
What do you mean by the performance down? In ssim, win_size only determines the size of guassian blur. Did you directly use ssim value as loss?

from pytorch-msssim.

@VainF yes. I used ssim in cyclegan for cycleloss. Win size is 11 as tuning param. Without ssim, my accuracy is 0.7 and it is 0.65 while adding ssim

from pytorch-msssim.

@VainF : Good cash. I think it should be

loss1 = 1-msssim(output...

Because range of ssim is from 0 to 1. I will check the new one

from pytorch-msssim.

😧 A mistake again.

from pytorch-msssim.

@John1231983 Most people seem to use window size of 11. However, the best outcome depends on what you want as your outcome. There is a paper on the effects of using different window sizes that I refer to here.

from pytorch-msssim.

Hi. I used ssim in cyclegan and the loss D tends to zero

Anyone try to apply ssim in cyclegsn? I found someone used it and it worked but my case is not

from pytorch-msssim.

@John1231983 , you can scale the ssim value by 100x.
I used this repo to train my image compression model and the loss was 100 * (1-msssim)

Code:

class MS_SSIM_Loss(MS_SSIM):
    def forward(self, img1, img2):
        return 100*( 1 - super(MS_SSIM_Loss, self).forward(img1, img2) )

See Experiments part of Conditional Probability Models for Deep Image Compression

we use the multiscale structural similarity index (MS-SSIM) [23] as measure of distortion d(x, xˆ) = 100 · (1 − MS-SSIM(x, xˆ))
for our models

from pytorch-msssim.