cyclomon / unsb Goto Github PK

Hi,
When I train a model with the default --direction (AtoB), will using --direction BtoA at inference work properly for translating some domain B image to domain A, or does the model need to instead be trained with --direction BtoA? I.e., is the model trained for both directions?

I was unsure upon noticing that the training only logs realA->fakeB translations.

Thanks!

I noticed that the compute_G_loss() function is called twice during the optimization.
https://github.com/cyclomon/UNSB/blob/main/models/sb_model.py#L143-L144

Hello,
Thank you for your awesome paper and code! I trained your model for breast MRI translation, and the translations looks great overall, but I do notice a bit of blurriness/loss of fine detail in the outputs, even if the image overall looks good. See for example an input image from domain A (left) and the translated output for domain B (right); note the blurriness in the output.

I was wondering if you suggest ways to help with the bluriness, especially with this translation task that requires only very subtle changes to the image. From your comment here #11 (comment), I think maybe a solution is to use a higher-resolution bottleneck, by using fewer downsampling/upsampling layers? What do you think?

Thanks!

Hi! why two datasets are used for training and testing?

Hello,

I was pretty amazed by the results you and your team created in your work "Unpaired Neural Schrödinger Bridge".
Is there any possibility that the code will be available?

Best regards,

hi I found that the following command can better install the visdom package.

conda install -c "conda-forge/label/cf201901" visdom

please consider to update the readme.

I have enjoyed reading your paper, while I was implementing your work on my dataset, I have increased the default batch size from 1 to 8 in base options. And it showed this error:

Traceback (most recent call last):
File "train.py", line 44, in
model.data_dependent_initialize(data,data2)
File "sb_model.py", line 106, in data_dependent_initialize
self.forward() # compute fake images: G(A)
File "sb_model.py", line 184, in forward
for t in range(self.time_idx.int().item()+1):
ValueError: only one element tensors can be converted to Python scalars"

Thank you for sharing this excellent work in the field of unpaired image translation. It introduces a new approach by utilizing SB without any intermediate computation, which is commendable.

However, I have a few questions regarding the details of the training forward process. It appears that you activate the evaluation mode during the training phase.

UNSB/models/sb_model.py

Line 183 in 5d22e49

self.netG.eval()

Consequently, the normalization layer is never updated throughout the training. I am curious about the rationale behind choosing this strategy.

I appreciate your response in advance.

Hello, thank you for your amazing work. What is the training and testing code of Summer2Winter? The introduction on the paper is "In the experiments on Summer→Winter, we used pre-trained VGG16 network as our feature selection source, following the strategy in the previous work [37].” How do I need to modify the code? Thank you!

Could you please a pre-trained model for a quick exploration？

Batch Size more than 1 breaks the code, tensor sizes mess up

I have a large dataset. Is it possible to train with a batch size larger than 1?
This fails if the batch size is larger than 1
https://github.com/cyclomon/UNSB/blob/main/models/sb_model.py#L178

Hi, I want to train our approach on two gpu. So I have provided the GPU IDs in train options. But I am getting this error
"
raceback (most recent call last):
File "train.py", line 44, in
model.data_dependent_initialize(data,data2)
File "models/sb_model.py", line 106, in data_dependent_initialize
self.forward() # compute fake images: G(A)
File "models/sb_model.py", line 184, in forward
for t in range(self.time_idx.int().item()+1):
ValueError: only one element tensors can be converted to Python scalars
"

Can you help me in this regard

Hello, thank you a lot for the work you’ve done on this project. I’ve been using your model and it has been working very well. I have a question that I hope you could help clarify.

I understand that the vgg directory contains code to utilize the pretrained VGG model. However, I noticed that the training process is conducted with sc_model.py. Could you please explain how sc_model.py differs from sb_model.py?

I believe sb_model.py is the model described in the paper. If this is the case, could you provide some insight into why sc_model.py is specifically used for the summer2winter dataset?

The related code is here -> https://github.com/cyclomon/UNSB/blob/main/models/sb_model.py#L307-L315

    if self.opt.lambda_SB > 0.0:
            XtXt_1 = torch.cat([self.real_A_noisy, self.fake_B], dim=1)
            XtXt_2 = torch.cat([self.real_A_noisy2, self.fake_B2], dim=1)
            bs = self.opt.batch_size
            ET_XY    = self.netE(XtXt_1, self.time_idx, XtXt_1).mean() - torch.logsumexp(self.netE(XtXt_1, self.time_idx, XtXt_2).reshape(-1), dim=0)
            self.loss_SB = -(self.opt.num_timesteps-self.time_idx[0])/self.opt.num_timesteps*self.opt.tau*ET_XY
            self.loss_SB += self.opt.tau*torch.mean((self.real_A_noisy-self.fake_B)**2)

self.loss_SB += self.opt.tau*torch.mean((self.real_A_noisy-self.fake_B)**2)

Shouldn't the self.opt.tau be replaced by the weight self.opt.lambda_SB?

Firstly, it is a great work that helps me catch up with SB concept. Anw, I am quite curious whether your method can be applied to standard image generation from pure noise. Have you tried it before, it is highly appreciated if you can share some trial results.

The UNSB project is quite impressive. During the actual code execution process, we used the code for underwater image translation experiments and noticed that several results exhibited flaws resembling "holes." Could you suggest possible improvements to address this issue? Thank you.

I have followed your default setting to train the horse2zebra dataset, and test on the epoch 400's model. Generate 120 images(which is the same num as the test dataset) and use NFE=5 to calculate FID, only got 231.22809380899474.
Is it possible for u to share how to get FID 35.7