cchen156 / learning-to-see-in-the-dark Goto Github PK
View Code? Open in Web Editor NEWLearning to See in the Dark. CVPR 2018
Home Page: http://cchen156.web.engr.illinois.edu/SID.html
License: MIT License
Learning to See in the Dark. CVPR 2018
Home Page: http://cchen156.web.engr.illinois.edu/SID.html
License: MIT License
paper has sRGB input comparision, I modify the last two level conv10(12 channel ) to output 3channel for the rgb input image generated by rawpy.
the training loss is down to 0.025, the result is not bad, but some image is not real. is this correct way?
I don't understand why you set the batch_size=1, i.e., only 1 picture will be trained in one iteration. Have you ever tried other batch_sizes?
I think that train_Sony.py at line 129
allfolders = glob.glob('./result/*0')
and train_Sony.py at line 136
if os.path.isdir("result/%04d" % epoch):
are wrong.
The correct programs are
allfolders = glob.glob(result_dir + '*0')
if os.path.isdir(result_dir + "%04d" % epoch):
, right?
Is there any evaluation network?
Hi,
Thanks for sharing this work and the result is awesome ! @cchen156
I find comparison with hdrnet in the project page , so are you using HDRnet prettrained model from @mgharbi 's repo or you trained the comparison model youself?
I want to reproduce the comparison but seems no night related pretrained model is found for hdrnet, would you like to share the comparison pretrained model ? Thanks
The official website contains a low light image and its result when run on SID model (http://web.engr.illinois.edu/~cchen156/SID/examples/6s.html). I could not find the weights for this in the repository. Is it possible to upload the weights for when trained on iPhone 6s?
I downloaded the two zip files for myself, and with my host (wasabi) I pay for storage but not bandwidth, so I'm willing to share the links to the datasets here:
https://s3.wasabisys.com/datasets/Learning-to-See-in-the-Dark/Fuji.zip
https://s3.wasabisys.com/datasets/Learning-to-See-in-the-Dark/Sony.zip
Feel free to use them, or get back in touch if you want me to stop making them public. I'll be keeping them up for the foreseeable future.
A question in the test_Sony.py is that, the scipy.io
is really slow and cv2.imwrite
is really fast. In the 8-bit saving way, do this two methods inference the finnaly PSNR & SSIM?
@cchen156 Hi, congrats first. In your training settings now, you aim to learn white balance within the network. Have you tried to apply white balance on input(those short exposure images) before feeding them into networks? If so, how does the result look like? THX!
The gpu memory is occupied when I run the train_Sony.py, so I set
tfconfig = tf.ConfigProto(allow_soft_placement=True) tfconfig.gpu_options.allow_growth=True sess = tf.Session(config=tfconfig)
But there is still the question...?
It's hard to ignore that in your paper Table 3, the psnr of Sony rgb input is 17.4 while another psnr is 25.11. I couldn't understand why they differ so much when both are rgb images. I mean,if input rgb images, they must have gone through the same process before put into the network.
And would you mind to explain how you convert your raw datasets to rgb images, and how to understand this difference?
I tried to compare the output generated by SID (O-sid) and photoshop (O-ps, with all auto from DNG to PNG), and according to what I saw:
1/ when the light is extremely low, O-sid is better than O-ps.
2/ but when the light is low and not extremely, O-ps is better than O-sid.
Also, O-sid loses quite a few texture compared to O-ps.
I wonder is this as expected?
Hi @cchen156 i test psnr of ur model on test samples 1-49 in ur test_list.txt , and the psnr is ~32dB ,is that the same with urs ? i didnt change any of ur codes
My PSNR Calculation :
PSNR(img1,img2):
mse = np.mean((img1 - img2)^2)
return 10*np.log10(255^2 / mse);
Hi,
Thanks for your novel work. I have some questions. I noticed that you restrict the value of ground truth image around 0 to 1, then force the model to generate output image which is also around 0 to 1. I am confused that why not just use the original ground truth image as the label and produce the output image directly without multiplying 255. Is there any difference between these two ways? Could you please help answer this question?
Thanks,
Amazing results, congrats!
Is it possible to download from somewhere the pre-trained model?
Thanks a lot!
@cchen156 Do you have the "Croplayer" crop operation in this fuctions,
It only have 'DECONVOLUTION' ans 'CONCAT',
I just tried to train Sony dataset after converting the raw data into jpg image and found the result quite terrible. So are there any tricks when training 8bit data?
P.S.:
The channel number of input is 12, which results from packing each channel of rgb into four.
why can not i download the data?
The website always return:
Bad Request
Error 400
I don't have enough RAM,can you give me some suggestions to revise the code. Thanks for a lot.
I reimplemented the paper with pytorch,but the result is very bad.I am not sure whether pytorch has changed the effect of tensorflow in this paper.
Hi, thanks for your great work.
This issue is just a suggestion of possible enhancement.
When testing the model, currently the output images are ground truth, the output from the neural network, and the input scaled up to the same mean as ground truth.
My suggestion is to add
scipy.misc.toimage(input_full[0,:,:,:]*255, high=255, low=0, cmin=0, cmax=255).save(result_dir + 'final/%5d_00_%d_in.png'%(test_id,ratio))
to output the initial input image, to show how unusable the original photo is.
Can you get the inference freeze.pb file for your model?
what is your input node name ? output node name?
The model is nice, but how can I use the model to convert my own single jpg(or png) img ? Is there any Introduction?
Hi Chen,
Thanks for your code, it is very useful!
I just wonder could you please provide the script for downloading Sony 12GB? It will help me a lot!
Thanks,
Hi, I am work on the project which can detect and track pedestrian in low light or night. Then how can I use your model for my dataset?
I noticed that the README suggests it needs Python 2.7 but from looking at the libraries, nothing jumps out that would be dependent on 2.7, so I'd have expected it to run with 3.
Is it simply that 2.7 is what you happened to develop / test with, or is there something in your code that specifically needs 2.7?
Hello,@cchen156,I used the cpkt file you provided to test PNSR and SSIM, but the measured value is still lower than the original text, PNSR=28.639, SSIM=0.7682, is it different from the method I used to test pnsr and ssim? Can you provide the method or code for testing pnsr and ssim?
@cchen156 Hi, nice work and congratulations. I am wondering why you use np.maximum(xx,0) in pack_raw() function to ignore negative values. Will it affect the result?
How long did the training on Titan X take? What was the final loss?
I have try to test the Sony Image Data:
/home/ljz/桌面/DB3065DB.png
If we only have rgb input images, how should I retrain the model and test it?
Thanks
download_models.py
Hi @cchen156 ,
This work is very exciting. But I have a question about the quantitative results reported in the paper Table3.
I downloaded the source code and trained model, and tested Sony dataset. I generate GT by Rawpy and apply Gamma transfer after it, since you apply Gamma transfer on the output of the network in your test code. I found that the results looks fine, but the average PSNR/SSIM in the testset is 25.44/0.713, which is quite lower than 28.88/0.787 reported in Table3 in your paper.
If I have made some mistake? How can I get the reported results.
Thanks,
Wenbo
after download and extract raw Sony data in dir dataset/Sony.
run test_Sony.py
got error at line 151:
im = raw.postprocess(use_camera_wb=True, half_size=False, no_auto_bright=True, output_bps=16)
rawpy._rawpy.LibRawFatalError: b'Unsufficient memory'
could you have met this?
Appreciate for answer.
I downloaded the sony data of 22 GB, the training and testing programs were executed, the folder named 'result_sony' is created, with some files in it, but those files are not executable and there were no images to be seen, can you please tell me that was result was expected once the program is executed completely.
Hi:
I found I cannot download the dataset, could you pls give another URL?
https://docs.google.com/uc?export=download
Bad Request
Error 400
why black level equal 512 and why is 16383 ,What is meaning of the number 16383?
@kkruups @prabodhw96 @cchen156 Hi, congrats for your work. In your paper and code, during training, input and corresponding ground truth are first scaled between [0-1], and then input are scaled by ratio but the ground truth is still between [0-1]. Have you tried not to scale input by ratio and keep input and gt both in range [0-1] to train the network? Will it get better result and what's your opinion? THX!
Hi!
I was able to run the examples for Sony raw data and the results are amazing.
I would like to experiment with Canon and Nikon raws but with Canon's raws the model seems to never converge. I am able to run it but i don't get any output. I modified the test_Sony file changing the ARW extension in CR2 (Canon's raw). Any advice?
Thank you!
when I run the download_models.py
there is no response. can i download the models in other ways?
GREAT WORK!
There's a certain app you mentioned in your paper, which can set parameters and export raw data.
Could you please tell me what's the name of the app?
I only know we can access YUV data through Apple's API.
Thanks~!
What`s this probably mean?Here is the log:
Traceback (most recent call last):
File "test_Sony.py", line 117, in
out_image = network(in_image)
File "test_Sony.py", line 76, in network
up6 = upsample_and_concat(conv5, conv4, 256, 512)
File "test_Sony.py", line 50, in upsample_and_concat
deconv_output = tf.concat([deconv, x2], 3)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.
Hello,thank you for your help.Because my computer configuration is too low, RAM is only 4G, gpu memory 2G, although you provide sonny(12G) data set, but still a bit big for me and I can't download it .so could you provide training test data set within 2G. I just want to use a small amount of data to test the model.
Thank you !
The download_models.py always got a HTTP connection error, could you please provide a available download links? Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.