jiaxi-jiang / fbcnn Goto Github PK
View Code? Open in Web Editor NEWOfficial Code for ICCV 2021 paper "Towards Flexible Blind JPEG Artifacts Removal (FBCNN)"
Home Page: https://arxiv.org/abs/2109.14573
License: Apache License 2.0
Official Code for ICCV 2021 paper "Towards Flexible Blind JPEG Artifacts Removal (FBCNN)"
Home Page: https://arxiv.org/abs/2109.14573
License: Apache License 2.0
Hi, I test the result on classic5 and live1 in gray with pre-trained weight
However, I find some mistakes caused by rgb2ycbcr function
Here is my offline test result
where classic5 is ok because it is gray input, and the live1 result is strange because of the rgb2ycbcr function.
The GT I compare is produced by Matlab version YCbCr, which is different from OpenCV version which is utilized in your training and eval process.
if n_channels == 3: img_L = cv2.cvtColor(img_L, cv2.COLOR_RGB2BGR) _, encimg = cv2.imencode('.jpg', img_L, [int(cv2.IMWRITE_JPEG_QUALITY), quality_factor]) img_L = cv2.imdecode(encimg, 0) if n_channels == 1 else cv2.imdecode(encimg, 3)
The logging result in live1_qf10 is
21-12-23 15:43:59.031 : Average PSNR/SSIM/PSNRB - live1_fbcnn_gray_10 -: 28.96$\vert$0.8254$\vert$28.64.
which is a little higher than 26.97 but much lower than 29.75, because the model is for OpenCV version Y, not Matlab version
However, the common setting in deblocking is using Matlab version Y
I test ICCV21-Learning Dual Priors for JPEG Compression Artifacts Removal, the offline results are
It is similar to the results produced in paper
I hope it may be helpful.
I'm excited to watch the demo video.
Please release the code for Google colab so that we can try it easily.
I am looking forward to your next update.
Thank you.
Hello,
I tried running 'main_test_fbcnn_color.py' on a real JPEG image using one 16 GB V100 but the code threw ´Out of Memory´ error. Any idea on how to use this code with large images, say 12 MPix or more?
run default code, show me error:
model.load_state_dict(torch.load(model_path), strict=True):
error in loading state_stict for FBCNN:
Missing keys in state_stict :
......
but when run fbcnn_color.py , it is ok.
why ?
Hi,
The README says that code will be released here https://github.com/cszn/KAIR/
Any idea when?
Thanks!
Greetings,
Sorry for bothering, but how can I run the trained model?
I tried to create another .py script for trained color.pth model (similar to Color/Real scripts)
But getting an error when tryin to run it.
File "main_test_fbcnn_latest_g.py", line 105, in
main()
File "main_test_fbcnn_latest_g.py", line 55, in main
model.load_state_dict(torch.load(model_path), strict=True)
File "D:\Users\Vasily\anaconda3\envs\FBCNN\lib\site-packages\torch\serialization.py", line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "D:\Users\Vasily\anaconda3\envs\FBCNN\lib\site-packages\torch\serialization.py", line 763, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: could not find MARK
How to solve this issue, could you, please, help?
Thank you in advance~
Hi,
Great works! I test your model(fbcnn_color.pth) in 'testset/Real' dataset. The results are not so remarkable as the picture in this master. The output of model(fbcnn_color.pth) without qf_input are as follow(left is input, right is output):
I don't kown if there are something wrong with my results. And the output of model(fbcnn_color.pth) with qf_input are also not so good. When zoom out, I can find obvious artifacts. Hope for your reply.
AssertionError: ../trainsets/Data_DIV2K_Flickr2K is not a valid directory在训练过程中出现这样一个问题,请问作者训练数据集的架构分布是如何的。
Sorry to bother you, but could you share any pretrained models for direct inference?
hello, it seems the results of the real dataset are not run from the fbcnn_color.pth ? Would you mind providing the corresponding fbcnn model for the real dataset?
Thank you!
Hi, It is a nice job to enable flexible QF embedding in JPEG deblocking!
I want to re-produce the results following the directions in the paper, However, there are some questions.
input = batch["degree"].type(Tensor)
label = batch["label"].type(Tensor)
qf_label = batch["qf"].type(Tensor)
qf_label = (1-qf_label/100) # 0-1
out, qf_pred = model(input)
mse = mse_pixelwise(out, label)
cls_loss = l1_pixelwise2(qf_pred, qf_label)
loss = l1_pixelwise(out, label) * 0.5 + mse * 0.5 + cls_loss * 0.1
However, during the training phase, the cls_loss is not changed at all
2021-10-09 22:38:47,481 - __main__ - INFO -
[QF 10 lr 0.000100 cls_loss:0.23448 loss:0.04852 Epoch 0/120] [Batch 19/203] [psnr: 19.300194 ] ETA: 6:18:37.362892 [mse: 0.005858 ] [mse_original: 0.000583 ]
2021-10-09 22:39:08,319 - __main__ - INFO -
[QF 10 lr 0.000100 cls_loss:0.20090 loss:0.03645 Epoch 0/120] [Batch 39/203] [psnr: 21.449274 ] ETA: 7:02:01.486693 [mse: 0.003679 ] [mse_original: 0.000384 ]
2021-10-09 22:39:29,365 - __main__ - INFO -
2021-10-10 05:11:26,158 - __main__ - INFO -
[QF 10 lr 0.000020 cls_loss:0.20545 loss:0.02607 Epoch 119/120] [Batch 159/203] [psnr: 34.804854 ] ETA: 0:00:41.660994 [mse: 0.000345 ] [mse_original: 0.000475 ]
2021-10-10 05:11:45,303 - __main__ - INFO -
[QF 10 lr 0.000020 cls_loss:0.22516 loss:0.02851 Epoch 119/120] [Batch 179/203] [psnr: 34.770472 ] ETA: 0:00:22.570621 [mse: 0.000385 ] [mse_original: 0.000513 ]
2021-10-10 05:12:04,498 - __main__ - INFO -
[QF 10 lr 0.000020 cls_loss:0.18704 loss:0.02385 Epoch 119/120] [Batch 199/203] [psnr: 34.775089 ] ETA: 0:00:03.771435 [mse: 0.000276 ] [mse_original: 0.000377 ]
2021-10-10 05:12:12,970 - __main__ - INFO - QF: 10 [PSNR: live1: 29.005289 classic: 29.159697]
[SSIM: live1: 0.806543 classic: 0.795164]
[max PSNR in live1: 29.005289, max epoch: 119]
The cls_loss is always at 2, and when I test the qf estimation with input image in qf=[10,20,..,90], the qf estimation is always 53,
which shows that the cls_loss does not work well. It is reasonable that qf estimation is 50 since it is good for L1 loss when not trained well.
Due to the GPU limitation, I set the Batchsize = 96, and training with DIV2K patches for about 2.5W iters
Is there anything wrong in my implementation, thanks!
Hello @jiaxi-jiang, Sorry to bother you,
I plan apply FBCNN
for my non-photo content jpeg(2d drawing),
but I have some question about tool implement,
could you teach me about those question?
FBCNN
,FBCNN
?I find a
split_imageset
function inFBCNN
utils_image.py
,
I try to invoke this function,
but I haven't find a good method to implement split and merge function.
Thank you produce this amazing tool. :)
Hi, Exist a possibility about up this repo to https://replicate.com/ for best use? or Can I up it?
Thank you so much for your impressive work, I tried many tools but their performance were not like yours. But I spent a lot of time to figure out how I can use test_split()
but I did not figure out where I should use it, could you please tell me in which line I should use it in main_test_fbcnn_color_real.py
what should be the input parameters.
Hello, thanks for your interest in our work!
The function you mentioned is mainly used for generating small patches for fast training. For testing large images, please try to utilize the function test_split
from utils_model.py
: https://github.com/jiaxi-jiang/FBCNN/blob/main/utils/utils_model.py#L214
For fair comparisons, we follow the convention of prior work to use 8 bit depth image for training and testing. Intuitively training with 16 bit depth images should be better for testing 16 bit depth images, but I think not necessary for testing 8 bit depth images. We can discuss more if you are interested in this part.
Feel free to ask me if you still have questions.
Originally posted by @jiaxi-jiang in #12 (comment)
Hello @jiaxi-jiang, Sorry to bother you,
I test some non-photo content jpeg(2d drawing),
high FBCNN
QF
can preserve detail and noise,
but I notice some area have slight color(chroma) brightness change.
In this case, FBCNN
QF 70
have slight brightness change on dark red area(red circle),
could you teach me how to improve color accurate for non-photo content?
original image (jpeg q75 420),
Other sample(QF 30,50,70)
ruti
thanks
where is the train dataset?
AssertionError: ../trainsets/Data_DIV2K_Flickr2K is not a valid directory During the training process, a question arises, what is the architecture distribution of the training dataset?
请问作者,主页的样例上的退化图是由由Jpeg退化函数从高清图获取的吗?我想获得类似的退化图像具体应该怎么操作?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.