question on evaluate.py about free-form-video-inpainting HOT 3 CLOSED

That doc could be outdated. @Nash2325138 could you take a look?

from free-form-video-inpainting.

@amjltc295 Hello, I have a new question about evaluation, I do some evaluation on video inpainting. My input frames size is 256256. My input mask is 100100 in the center of the frames. About 15% of the size of frames. I found that the value of PSNR is really high, which is over 36. But the paper I read about image inpainting the value of PSNR in their dataset almost between 25 and 30.
Such as this table.

I read the code of evaluation. I don't mean that your code is wrong, I just what to find why the value is so high, I white the issue because I think that maybe you can help me.

from free-form-video-inpainting.

@amjltc295 Hello, I have a new question about evaluation, I do some evaluation on video inpainting. My input frames size is 256_256. My input mask is 100_100 in the center of the frames. About 15% of the size of frames. I found that the value of PSNR is really high, which is over 36. But the paper I read about image inpainting the value of PSNR in their dataset almost between 25 and 30.
Such as this table.

I read the code of evaluation. I don't mean that your code is wrong, I just what to find why the value is so high, I white the issue because I think that maybe you can help me.

I'm not sure what you did for the evaluation, there may be some bugs if the PSNR is a lot higher than other methods.
Also we did not train our models with bounding boxes, so if you measure the results with bounding boxes, it may be worse (in our paper, we don't have a lot improvement for MSE as well.)
Other than wrong code/evaluation process, there is a trade-off between perceptual quality and distortion measures (see http://openaccess.thecvf.com/content_cvpr_2018/papers/Blau_The_Perception-Distortion_Tradeoff_CVPR_2018_paper.pdf)
In the inpainting task we focus on perceptual quality more (with the help of GAN) but may lose the distortion scores (recover videos different from original ones).

from free-form-video-inpainting.