amjltc295 / free-form-video-inpainting Goto Github PK

Thanks for sharing your work. but，when I use the FVI dataset, root_masks_dir="../dataset/FVI/Train/object_masks/"
Displays: NotImplementedError: Image mode P
And, after using root_masks_dir="../dataset/FVI/Train/random_masks/"
The number of JPEGImages trained in the epoch is 1940, and the random_masks are greater than 2,000, so a complete epoch cannot be completed.

Hi Authors,

Thanks for sharing your interesting work. I follow the way you mentioned in paper to train(pre-train the generator without the
TSMGAN loss until convergence, and then fine-tune with the TSMGAN). But I can not reproduce the result in the paper. It would be a great help if you can mention the weight of different losses (i.e. perceptual, style, reconstruction and adversarial) you used while training. Thanks for your helping.

Best,
Hank

Hi, authors. Thanks for your splendid works.
when i run train.py with test mode i get this error:

Traceback (most recent call last):
File "F:\py\Free-Form-Video-Inpainting-master\src\trainer\trainer.py", line 426, in evaluate_test_set
output_root_dir=output_root_dir, epoch=epoch)
File "F:\py\Free-Form-Video-Inpainting-master\src\trainer\trainer.py", line 98, in _evaluate_data_loader
for batch_idx, data in enumerate(data_loader):
File "E:\Anaconda3\envs\opn\lib\site-packages\torch\utils\data\dataloader.py", line 819, in iter
return _DataLoaderIter(self)
File "E:\Anaconda3\envs\opn\lib\site-packages\torch\utils\data\dataloader.py", line 560, in init
w.start()
File "E:\Anaconda3\envs\opn\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "E:\Anaconda3\envs\opn\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "E:\Anaconda3\envs\opn\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "E:\Anaconda3\envs\opn\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "E:\Anaconda3\envs\opn\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
File "F:\py\Free-Form-Video-Inpainting-master\src\utils\directory_IO.py", line 54, in getattr
f"{attr} not in root_dir_names {self.output_root_dirs}")
KeyError: "getstate not in root_dir_names {'masked_frames': '../VOS_resized2\\masked_frames', 'result_frames': '../VO
S_resized2\\result_frames', 'optical_flows': '../VOS_resized2\\optical_flows'}"
--Call--

e:\anaconda3\envs\opn\lib\multiprocessing\util.py(182)call()
-> def call(self, wr=None,
(Pdb) Traceback (most recent call last):
File "", line 1, in
File "E:\Anaconda3\envs\opn\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "E:\Anaconda3\envs\opn\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Hello, when I tried to test my own testing data and set the type to super resolution, I always got a bad result. Could you please tell me how to set the json file when I want to use super resolution type to test my video? Thanks!

Hi,

I have some confusion about the loss design.

As the mentioned by Yu et al., SN-PatchGAN has encoded the patch-level information, so they omit the perceptual loss.
In your paper, except Temporal-PatchGAN, perceptual loss and style loss are also adopted to train the network. I want know why you add the perceptual and style loss.

Look forward to your answer. Thanks a lot.
Best wishes.

Thanks for your great work and sharing!

I'm sorry to re-submit an issue about the flownet_wrapper package. I need to use this package for evaluation. Could you please more details about this package? I didn't find it in the internet.

Thanks!

Hello,I have met a bug on evaluate.py
I read the script_usage.md and I have already run the code and get the result and I want to use the evaluate.py to evaluate other result.
So I use the original result as a experiment,but get some bugs.

python evaluate.py -rgd ../dataset/test_20181109/JPEGImages -rmd ../dataset/random_masks_vl20_ns5_object_like_test/ -rrd saved/VideoInpaintingModel_v0.3.0_l1_m+a_mvgg_style_1_6_0.05_120_0_0_all_mask/0102_214744/test_outputs -mask
I found that The code"-mask" doesn't exist ,is it possible that you modify the code?
And I meet another bug that I can't read the result_image.

I'd appreciate it if you could answer my questions.

Could I use a 1080ti GPU to train your model? Thank you

Thanks for sharing your code. I try to reproduce inference results with pre-trained weights (weights for the BMVC 2019 work) and get only 7 FPS on GTX 1080Ti. In the paper you report 80 FPS for LGTMS solution, can you tell me what conditions these results were obtained?

Hi,
I used the config in config.json and FVI_all_masks.json to train a 3d gated model in FVI.
I modified the loss_gan_temporal_weight to 1.0 and pretrained the model for 3days without gan loss first.
The model has been trained more than 5 days on 4 GPUs with batch size 8 and has been convergent already. However, the performance is far away as good as the value in your paper.

So, How can we modify the config to obtain the performance as good as the value in your paper?
Are the losses' weights in my setting correct? (loss_recon: 1.0, loss_masked_recon: 1.0, loss_vgg: 1.0, loss_style: 10, loss_gan_spatial_weight: 1.0)

Thank you!

Hello,

Thank you for this great work!

While going through your code, I noticed that the weight for the GAN loss is 0: code. It basically means that the generator is trained with the reconstruction loss, perceptual loss, etc, without the supervision of the discriminator.

I am not sure if it is a mistake or your have trained your model in this way. I will try to train the model again with the GAN loss integrated.

Any explanation or remark from you would be highly appreciated. Thank you in advance!

@amjltc295 @Nash2325138 Can anyone help me with the inference of both methods, TPatchGAN and LGTSM? The code seems to be confusing and I need to test both models on custom face dataset.

Hi Authors : )

Thanks for your interesting work. It would be really appreciated if you could give some information about your work on training epochs and how do you select the video for training!

Best,
Xueyan

I am having a problem with the following line:

where

Since AttentionDownSampleModule is the basically the same class as DownSampleModule, its forward() method returns a tuple of size 3.

However, the RefineNet.forward() is trying to unpack a tuple of size 2 and I get ValueError: too many values to unpack (expected 2).

Hi, thanks you work!
I test your pretrained model on my dataset . (608*544 per frame) when I just put 11 frame in test folder , it can get good performance . But when I put more frames (like 30 , 1441 frames), it's GPU out of memory , what's the problem?

Thanks,

Hi,

Thanks for sharing your code. I am working on the same dataset FVI. I just wanted to know for how long did you trained the network and you have mentioned that you have used 1940 video and then applied a few data augmentation techniques. Did you apply data augmentation on all the videos? Can you please mention the final size of the dataset after augmentation? Also, It would be a great help if you can mention the weight of different losses (i.e. perceptual, style, reconstruction and adversarial) you used while training.

Looking for your help

Hi, thanks for sharing great codes!

I have a question about the mode (training / testing) you have used for I3D networks for evaluating VFID score.

When I evaluate your model without converting I3D into eval mode, I was able to reproduce the similar results in the paper.
However, when I turned the model into evaluation mode (because using .eval() is common for using the model as feature extractor), I found the VFID score goes up to over 1.0.

Is there any reason why the I3D network is used as training mode?
Thank you for reading! Have a nice day.

Hi, authors. Thanks for your splendid works.

I want to test your code with your provided pretrained weights, but I am confused of their naming. I saw two weights in the shared Google drive folder, but which one is for the FaceForensics dataset? Which for the FFVI dataset?

Hi Ya-Liang, thanks for your wonderful sharing. I was a little confused during the fine-tune stage. Both spatial_discriminator and temporal_discriminator should be involved？ There seems to be a dimension error when initializing the spatial-discriminator network.
please refer to:

Secondly，should the non-gan losses (weight) settings in the pretrained stage keep the same in the fine-tune stage?

Thanks~

Where can I find the supplementary material for your BMVC work 'Learnable Gated Temporal Shift Module for Deep Video Inpainting'?

Hi,tahnks for your sharing.
When I download your dataset, I discover that the FVI and FaceForensics dataset link to the same website,so could you please share your dropped dataset?
Thanks a lot!

Hey guys, thanks for the clean code and open sourcing your work
I'm looking into face inpainting techniques for some work in occlusion removal for faces, and I wanted to see if using your technique would be able to provide more temporal stability
When I tried using the Face Forensics Weights (Forensics_L1_maskL1_vgg_style_1_6_1_10_allMasks_0311_092041_e200.pth ) with the normal testing script ( with some modification to load my custom data )

I am getting a shape mismatch error.

Am I expected to use a different model config file for the Face Foresnics model , If so which one?

Regards
-Ashwath

In your training procedure, you updated G first and then updated D.
I think the reason is to save GPU memory. But It may be wrong against the principle of discriminating learning.
Updating D net first to find the minimal or optimal value that measure the divergence of distribution between real and fake data.
But in your procedure, you update G first by latest D net's parameters. It may trains successfully with LSGAN, but gets enormous value with hinge loss in my experiments.
Though the results are not too bad with LSGAN, the temporal GAN loss will be constant after some iterations.
I think the reason is what I have pointed and I have done some experiments to verify.

Hi,

Thanks very much for providing the research community with your code.

In the LGTSM paper, it says "Note that the TSM module could be easily modified to online settings (without peeking future frames) for real-time applications."

Does the code allow real-time usage? And are the weights trained for certain a future horizon parameter, or are they independent of the future horizon?

Thanks, Mo

Hi Ya-Liang, thanks for your wonderful works and sharing.
According to your code framework, I just want to modify the "weight of TemporalWarpingLoss" just like the following code:

{ "type": "TemporalWarpingLoss", "weight": 1, "nickname": "loss_warp", "args": { "alpha": 50 } }

However, I can not found the "libs.flownet2_pytorch.flownet_wrapper" package in this repo.
Would you please tell me where can we found the flownet_wrapper function. I search the official version of FlowNet2 in https://github.com/NVIDIA/flownet2-pytorch, but there is no function named as "flownet_wrapper" in official repo.

I am not clear about the two version used in you TSM,
(1)What's the mearning of of Zero and Circulant?
different pading style?
(2)The the learnable can be understant in your code?
I am a greenhand in code and this field, thank you very much and I am looking forward your reply and patience.
That's all~

Hello, I learned a lot from your codes and your paper, thank you! However, when I tried to run your codes and evaluate the FID score, I found I was forbidden by the website you leave for downloading the i3d model (http://www.cmlab.csie.ntu.edu.tw/~zhe2325138/i3d_rgb_imagenet.pt), do you have other link for downloading this model? Thanks and I'm looking for your reply.

Hi！
Can you share dataset? I would strongly appreciate any helps from you!

Thanks for your wonderful sharing.
I try to train the model by the dataset you provided but I met some error.
There is a file here "FaceForensics\FaceForensics\Train\videos\mx7Yv0zAtAk_0_nQcD9LbDm1Y_0" is empty.
The following is the error message.

[2020-07-30 06:08:52,315] {trainer.py:384} INFO - Epoch: 1 [0/854 (0%)] loss_total: 1.860, BT: 2.97s
[2020-07-30 06:10:41,876] {trainer.py:384} INFO - Epoch: 1 [80/854 (9%)] loss_total: 2.344, BT: 2.73s
[2020-07-30 06:12:31,386] {trainer.py:384} INFO - Epoch: 1 [160/854 (19%)] loss_total: 1.278, BT: 2.73s
[2020-07-30 06:14:20,886] {trainer.py:384} INFO - Epoch: 1 [240/854 (28%)] loss_total: 2.026, BT: 2.73s
[2020-07-30 06:16:07,748] {dataset.py:115} WARNING - len frames 0 reader 0 < sample_length 32 dir ../dataset/FaceForensics/Train/videos/mx7Yv0zAtAk_0_nQcD9LbDm1Y_0
[2020-07-30 06:16:10,395] {trainer.py:384} INFO - Epoch: 1 [320/854 (37%)] loss_total: 1.831, BT: 2.73s
Traceback (most recent call last):
File "train.py", line 166, in
main(config, args.resume, args.output_root_dir, args.pretrained_path)
File "train.py", line 112, in main
trainer.train()
File "/storage/Free-Form-Video-Inpainting-master/src/base/base_trainer.py", line 98, in train
result = self._train_epoch(epoch)
File "/storage/Free-Form-Video-Inpainting-master/src/trainer/trainer.py", line 319, in _train_epoch
for batch_idx, data in enumerate(self.data_loader):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/usr/local/lib/python3.6/dist-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/storage/Free-Form-Video-Inpainting-master/src/data_loader/dataset.py", line 171, in getitem
return self._process_vds(vds)
File "/storage/Free-Form-Video-Inpainting-master/src/data_loader/dataset.py", line 120, in _process_vds
gt_frames = self._transform(gt_frames)
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 61, in call
img = t(img)
File "/storage/Free-Form-Video-Inpainting-master/src/data_loader/transform.py", line 22, in call
w, h = img_group[0].size
IndexError: list index out of range

Thanks for your help.

Hi @amjltc295 ,

I saw that you have released an implementation of the model proposed in ICCV2019. video-inpainting-model.py.
I trained it on YouTube-VOS with object-like masks. However, the results are very blurred and I'm unable to reproduce the results shown in your paper.

Would you mind sharing completed training codes (especially weights of different losses)?
I would strongly appreciate any helps from you!

Thanks!

Hi, Thanks for your code.
I can't find the code which related to the learnable temporal shift kernel. I only find the original TSM Code from Song Han, can you tell me where is?
Thanks very much

Hi Ya-Liang, after going through the issues, you have mentioned in #20 that there are pretraining and finetuning stage of your model. Also, when I train your model from scratch, I found your GAN loss is set to zero. Could you please explain the detailed training schedule of the pretraining and finetuning stage and loss type/weight? Thanks in advance : )

Hi, Thanks for your wonderful work!

I was trying to train the network with your code, but it seems the link in the code does not work any more. The error I got is:

Traceback (most recent call last):
File "train.py", line 166, in
main(config, args.resume, args.output_root_dir, args.pretrained_path)
File "train.py", line 103, in main
pretrained_path=pretrained_path
File "/media/wenhua/disk_wenhua/workspace/Free-Form-Video-Inpainting/src/trainer/trainer.py", line 53, in init
init_i3d_model()
File "/media/wenhua/disk_wenhua/workspace/Free-Form-Video-Inpainting/src/evaluate.py", line 44, in init_i3d_model
urllib.request.urlretrieve('http://www.cmlab.csie.ntu.edu.tw/~zhe2325138/i3d_rgb_imagenet.pt', i3d_model_weight)
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 1345, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/home/wenhua/anaconda3/envs/free_form_video_inpainting/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 110] Connection timed out>

Thanks!

Hi,

I was going through the code but was unable find where you have implemented learnable shift of TSM. I was able to find normal shift in src/models/tsm_utils.py. I would be very grateful if you can indicate the part of code implementing learnable shift in TSM.

Thanks,
Harsh

hellodear authorI wanna know where is self.flownet.infer_video code in src/model/loss.py?!

Dear authors,

Thank you for sharing!
I have a question that, do you test all frames in the test set for each sample?
According to dataset.py and inference.config, it seems that you only evaluate on former 15 frames of each video.

BTW, the model takes n frames as input and outputs n frames, so for each video how do you deal with overlapping frames?

when I enter this command conda env create -f environment.yaml
I got

Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

• ffmpeg==4.0.2=ha0c5888_2
• libxml2==2.9.8=h422b904_5
• libxcb==1.13=h470a237_2
• xorg-renderproto==0.11.1=h470a237_2
• libgfortran-ng==7.3.0=hdf63c60_0
• libgfortran==3.0.0=1
• pixman==0.34.0=h470a237_3
• xorg-libxdmcp==1.1.2=h470a237_7
• xorg-libx11==1.6.6=h470a237_0
• tornado==5.1.1=py37h7b6447c_0
• py-opencv==3.4.2=py37hb342d67_1
• fontconfig==2.13.1=h65d0f4c_0
• numpy==1.15.4=py37h1d66e8a_0
• libffi==3.2.1=hd88cf55_4
• zlib==1.2.11=h7b6447c_3
• xorg-kbproto==1.0.7=h470a237_2
• libgcc-ng==8.2.0=hdf63c60_1
• cffi==1.11.5=py37he75722e_1
• torchvision==0.2.1=py37_1
• xorg-libice==1.0.9=h470a237_4
• xorg-libxext==1.3.3=h470a237_4
• harfbuzz==1.9.0=h04dbb29_1
• gmp==6.1.2=hfc679d8_0
• scipy==1.1.0=py37hfa4b5c9_1
• nettle==3.3=0
• sip==4.18.1=py37hf484d3e_2
• freeglut==3.0.0=hfc679d8_5
• mkl_fft==1.0.6=py37h7dd41cf_0
• dbus==1.13.2=h714fa37_1
• freetype==2.9.1=h8a8886c_1
• opencv==3.4.2=py37h6fd60c2_1
• expat==2.2.6=he6710b0_0
• libiconv==1.15=h470a237_3
• libopencv==3.4.2=hb342d67_1
• libtiff==4.0.9=he85c1e1_2
• openh264==1.8.0=hd28b015_0
• xorg-libxrender==0.9.10=h470a237_2
• xorg-libsm==1.2.3=h8c8a85c_0
• tk==8.6.8=hbc83047_0
• openssl==1.1.1a=h7b6447c_0
• pthread-stubs==0.4=h470a237_1
• gstreamer==1.14.0=hb453b48_1
• pyqt==5.6.0=py37h22d08a2_6
• graphite2==1.3.12=hfc679d8_1
• libpng==1.6.35=hbc83047_0
• glib==2.56.2=h464dc38_1
• libglu==9.0.0=hfc679d8_0
• pillow==5.3.0=py37hc736899_0
• xz==5.2.4=h14c3975_4
• scikit-image==0.14.0=py37hf484d3e_1
• xorg-xproto==7.0.31=h470a237_7
• kiwisolver==1.0.1=py37hf484d3e_0
• gst-plugins-base==1.14.0=hbbd80ab_1
• xorg-xextproto==7.3.0=h470a237_2
• sqlite==3.25.3=h7b6447c_0
• pytorch==1.0.0=py3.7_cuda9.0.176_cudnn7.4.1_1
• mkl_random==1.0.1=py37h4414c95_1
• icu==58.2=hfc679d8_0
• pywavelets==1.0.1=py37hdd07704_0
• cairo==1.14.12=h276e583_5
• hdf5==1.10.2=hc401514_2
• jasper==2.0.14=h07fcdf6_1
• ninja==1.8.2=py37h6bb024c_1
• xorg-libxau==1.0.8=h470a237_6
• bzip2==1.0.6=h470a237_2
• libstdcxx-ng==8.2.0=hdf63c60_1
• matplotlib==2.2.2=py37hb69df0a_2
• gnutls==3.5.19=h2a4e5f8_1
• ncurses==6.1=he6710b0_1
• python==3.7.1=h0371630_3
• libedit==3.1.20170329=h6b74fdf_2
• gettext==0.19.8.1=h5e8e0c9_1
• numpy-base==1.15.4=py37h81de0dd_0
• pcre==8.42=h439df22_0
• x264==1!152.20180717=h470a237_1
• jpeg==9c=h470a237_1
• readline==7.0=h7b6447c_5
• qt==5.6.3=h8bf5577_3

What might be the issues?

It is ok to use pre-training weights to run on THE GPU during the test. But I can only run on the CPU but not on the GPU during the training?Can you give me some advice?

hi !
when i run train.py with test mode i get this error:

Traceback (most recent call last):
File "train.py", line 166, in
main(config, args.resume, args.output_root_dir, args.pretrained_path)
File "train.py", line 103, in main
pretrained_path=pretrained_path
File "/Users/momo/Desktop/code/inpainting/Free-Form-Video-Inpainting-master/src/trainer/trainer.py", line 33, in init
pretrained_path
File "/Users/momo/Desktop/code/inpainting/Free-Form-Video-Inpainting-master/src/base/base_trainer.py", line 67, in init
self._resume_checkpoint(resume)
File "/Users/momo/Desktop/code/inpainting/Free-Form-Video-Inpainting-master/src/base/base_trainer.py", line 194, in _resume_checkpoint
self.model.load_state_dict(checkpoint['state_dict'])
File "/Users/momo/miniconda2/envs/CRAFT/lib/python3.6/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for VideoInpaintingModel:
Missing key(s) in state_dict: "generator.coarse_net.downsample_module.conv1.featureConv.weight", "generator.coarse_net.downsample_module.conv1.gatingConv.weight", "generator.coarse_net.downsample_module.conv2.featureConv.weight", "generator.coarse_net.downsample_module.conv2.gatingConv.weight", "generator.coarse_net.downsample_module.conv3.featureConv.weight", "generator.coarse_net.downsample_module.conv3.gatingConv.weight", "generator.coarse_net.downsample_module.conv4.featureConv.weight", "generator.coarse_net.downsample_module.conv4.gatingConv.weight", "generator.coarse_net.downsample_module.conv5.featureConv.weight", "generator.coarse_net.downsample_module.conv5.gatingConv.weight", "generator.coarse_net.downsample_module.conv6.featureConv.weight", "generator.coarse_net.downsample_module.conv6.gatingConv.weight", "generator.coarse_net.downsample_module.dilated_conv1.featureConv.weight", "generator.coarse_net.downsample_module.dilated_conv1.gatingConv.weight", "generator.coarse_net.downsample_module.dilated_conv2.featureConv.weight", "generator.coarse_net.downsample_module.dilated_conv2.gatingConv.weight", "generator.coarse_net.downsample_module.dilated_conv3.featureConv.weight", "generator.coarse_net.downsample_module.dilated_conv3.gatingConv.weight", "generator.coarse_net.downsample_module.dilated_conv4.featureConv.weight", "generator.coarse_net.downsample_module.dilated_conv4.gatingConv.weight", "generator.coarse_net.downsample_module.conv7.featureConv.weight", "generator.coarse_net.downsample_module.conv7.gatingConv.weight", "generator.coarse_net.downsample_module.conv8.featureConv.weight", "generator.coarse_net.downsample_module.conv8.gatingConv.weight", "generator.coarse_net.upsample_module.deconv1.conv.featureConv.weight", "generator.coarse_net.upsample_module.deconv1.conv.gatingConv.weight", "generator.coarse_net.upsample_module.conv9.featureConv.weight", "generator.coarse_net.upsample_module.conv9.gatingConv.weight", "generator.coarse_net.upsample_module.deconv2.conv.featureConv.weight", "generator.coarse_net.upsample_module.deconv2.conv.gatingConv.weight", "generator.coarse_net.upsample_module.conv10.featureConv.weight", "generator.coarse_net.upsample_module.conv10.gatingConv.weight", "temporal_discriminator.conv1.featureConv.weight", "temporal_discriminator.conv2.featureConv.weight", "temporal_discriminator.conv3.featureConv.weight", "temporal_discriminator.conv4.featureConv.weight", "temporal_discriminator.conv5.featureConv.weight".
Unexpected key(s) in state_dict: "generator.coarse_net.downsample_module.conv1.featureConv.weight_v", "generator.coarse_net.downsample_module.conv1.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv2.featureConv.weight_v", "generator.coarse_net.downsample_module.conv2.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv3.featureConv.weight_v", "generator.coarse_net.downsample_module.conv3.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv4.featureConv.weight_v", "generator.coarse_net.downsample_module.conv4.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv5.featureConv.weight_v", "generator.coarse_net.downsample_module.conv5.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv6.featureConv.weight_v", "generator.coarse_net.downsample_module.conv6.gatingConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv1.featureConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv1.gatingConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv2.featureConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv2.gatingConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv3.featureConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv3.gatingConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv4.featureConv.weight_v", "generator.coarse_net.downsample_module.dilated_conv4.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv7.featureConv.weight_v", "generator.coarse_net.downsample_module.conv7.gatingConv.weight_v", "generator.coarse_net.downsample_module.conv8.featureConv.weight_v", "generator.coarse_net.downsample_module.conv8.gatingConv.weight_v", "generator.coarse_net.upsample_module.deconv1.conv.featureConv.weight_v", "generator.coarse_net.upsample_module.deconv1.conv.gatingConv.weight_v", "generator.coarse_net.upsample_module.conv9.featureConv.weight_v", "generator.coarse_net.upsample_module.conv9.gatingConv.weight_v", "generator.coarse_net.upsample_module.deconv2.conv.featureConv.weight_v", "generator.coarse_net.upsample_module.deconv2.conv.gatingConv.weight_v", "generator.coarse_net.upsample_module.conv10.featureConv.weight_v", "generator.coarse_net.upsample_module.conv10.gatingConv.weight_v", "temporal_discriminator.conv1.featureConv.weight_v", "temporal_discriminator.conv2.featureConv.weight_v", "temporal_discriminator.conv3.featureConv.weight_v", "temporal_discriminator.conv4.featureConv.weight_v", "temporal_discriminator.conv5.featureConv.weight_v".
(CRAFT) bogon:src momo$ python3 train.py -r /Users/momo/Desktop/code/inpainting/Free-Form-Video-Inpainting-master/v0.2.3_GatedTSM_inplace_noskip_b2_back_L1_vgg_style_TSMSNTPD128_1_1_10_1_VOR_allMasks_load135_e135_pdist0.1256.pth --dataset_config other_configs/inference_example.json -od test_outputs -p ../vgg16-397923af.pth

what shold do for this?

Hi,
it seems that there is no corresponding pretrained model to the latest work LGTSM, just the pretrained model in ICCV2019's work. Could you please provide the pretrained LGTSM model?

Thanks!

Hi,

Many thanks for sharing the code.

I found the pretrained model (v0.2.3_GatedTSM_inplace_noskip_b2_back_L1_vgg_style_TSMSNTPD128_1_1_10_1_VOR_allMasks_load135_e135_pdist0.1256.pth) is for gated TSM but not LGTSM.

May I ask could you please share your trained model for LGTSM?

Many thanks!

Hi,
This url is forbidden. Could you please check it and update?
https://github.com/amjltc295/Free-Form-Video-Inpainting/blob/master/src/evaluate.py#L44