elliottwu / deephdr Goto Github PK
View Code? Open in Web Editor NEWThis is the implementation for Deep High Dynamic Range Imaging with Large Foreground Motions (ECCV'18)
License: MIT License
This is the implementation for Deep High Dynamic Range Imaging with Large Foreground Motions (ECCV'18)
License: MIT License
Hi, thanks for your work.
I have a problem about the PNSR in your work.
the code about psnr is following one in mode.py
curr_psnr = [compute_psnr(tonemap_np(res_samples[i]), tonemap_np(batch_ref_HDR[i])) for i in range(batchSz)]
I understand it is calculation of PNSR_T
. While I tested the pretrained model in Kalantari's dataset and got a average PSNR_T
(41.9977) which is different of the PSNR_T
in your paper (40.81 for unet ver. and 41.65 for resnet ver.). I wonder if this is normal or the code of calculate PSNR_T
is something other?
The detail of my testing in following:
First I cloned the projection and downloaded the pretrained model and Kalantari's dataset (from the link in readme.md)
Then I have the test dataset and pretrained model
$ ls dataset/test
001 002 003 004 005 006 007 008 009 010 BarbequeDay LadySitting ManStanding PeopleStanding PeopleTalking
$ ls pretrained_ckp/
checkpoint pretrained.data-00000-of-00001 pretrained.index pretrained.meta
I made a test by the command
$ CUDA_VISIBLE_DEVICES=3 python test.py --dataset ./dataset/test --num_shots 3 --checkpoint_dir pretrained_ckp --batch_size 1 --results_dir results_pretrained --test_h 960 --test_w 1440
the log is following:
$ CUDA_VISIBLE_DEVICES=3 python test.py --dataset ./dataset/test --num_shots 3 --checkpoint_dir pretrained_ckp --batch_size 1 --res
ults_dir results_pretrained --test_h 960 --test_w 1440
2019-11-30 14:11:41.696733: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2019-11-30 14:11:43.051910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:
name: Tesla V100-PCIE-16GB major: 7 minor: 0 memoryClockRate(GHz): 1.38
pciBusID: 0000:db:00.0
totalMemory: 15.78GiB freeMemory: 15.36GiB
2019-11-30 14:11:43.051957: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2019-11-30 14:11:43.606928: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-11-30 14:11:43.606977: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2019-11-30 14:11:43.606986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2019-11-30 14:11:43.607332: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14877 MB memory) -> physical GPU (device: 0, name: Tesla V100-PCIE-16GB, pci bus id: 0000:db:00.0, compute capability: 7.0)
WARNING:tensorflow:From /data/nonu_data/pyvenv/.venv/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
WARNING:tensorflow:From /data/nonu_data/pyvenv/.venv/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py:118: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
[*] Reading checkpoints...
batch no. 1:
time: 4.8884
PSNR: 48.6350
batch no. 2:
time: 0.7845
PSNR: 44.5190
batch no. 3:
time: 0.2106
PSNR: 41.9392
batch no. 4:
time: 0.2477
PSNR: 42.9087
batch no. 5:
time: 0.2121
PSNR: 43.0685
batch no. 6:
time: 0.2460
PSNR: 38.9594
batch no. 7:
time: 0.2231
PSNR: 39.3789
batch no. 8:
time: 0.2155
PSNR: 34.9346
batch no. 9:
time: 0.2113
PSNR: 42.4318
batch no. 10:
time: 0.2115
PSNR: 37.4935
batch no. 11:
time: 0.2123
PSNR: 39.1859
batch no. 12:
time: 0.2146
PSNR: 41.9842
batch no. 13:
time: 0.2251
PSNR: 49.1725
batch no. 14:
time: 0.2523
PSNR: 43.7752
batch no. 15:
time: 0.2158
PSNR: 41.5797
Average PSNR: 41.9977
Hi,
Thanks for your job. I have some questions . The paper describes that the radiometric calibration and gamma correction are needed if the LDR is not in RAW format, however I don't see them in the code. Is the input of the network is in rgb format? Otherwise, If the input image is raw format, how to use the network? is demosaicking needed to convert RAW to RGB?
Thank you for your work. in function conv2d, why do you calculate pad_top, pad_bottom, pad_left, pad_right. why not use d_h=2, d_w=2 directly????
Hi,
Thank you for your nice work.
I found that in the paper the data is normalized to 0-1 but in your source code the range is from -1->1.
Can you give me an idea about this difference?
Thanks!
I am confuse about how to transfer a LDR image to HDR domain.
In your paper, it was finished by gamma correction, (consistent with Kalantari's paper)
Hi = (Ii ** γ)/ti; γ=2.2
, where ti
is the exposure time of image Ii
.
In your code,
def LDR2HDR(img, expo): # input/output -1~1
return (((img+1)/2.)**GAMMA / expo) *2.-1
and
in_exps_path = os.path.join(scene_dir, 'input_exp.txt')
in_exps = np.array(open(in_exps_path).read().split('\n')[:ns]).astype(np.float32)
in_exps -= in_exps.min()
...
in_HDRs[:,:,c*i:c*(i+1)] = LDR2HDR(img, 2.**in_exps[i])
And the Kalantari's dataset include exposure bias {−2.0, 0.0, +2.0}
or {−3.0, 0.0, +3.0}
.
My confusion are follow:
Dose in_exps -= in_exps.min()
mean that ti
in formulation is the the exposure time relative to low exposure image ?
What is the exposure bias? Why exposure time is equal to 2**exposure_bias
?
I search the wiki and find a formulation which may be related,
EV = log2( N**2 / t )
where EV
is exposure value, N
is f-number and t
is exposure time. It can transfer to t = (N**2)/(2**EV)
. While it also can not explain the relationship between exposure time and exposure_bias.
Dose the exposure_bias mean the exposure compensation ? If dose, then expousre_time = 2**exposure_compensation
?
Hello,
Have you encountered outofrangeerror?
OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 8, current size 3) [[node shuffle_batch (defined at /mnt/Data/wk_linux/DeepHDR_T/DeepHDR/load_data.py:85) ]]
file "/mnt/Data/wk_linux/DeepHDR_T/DeepHDR/model.py", line 120, in build_model self.in_LDRs, self.in_HDRs, self.ref_LDRs, self.ref_HDR, _, _ = load_data(filename_queue, config)
My system is ubuntu 16.04, tensorflow 1.13.1, python 3.5
I used my own dataset which has higher resolution, it contains 80 sets of images. The images are .png format.
Looking forward to hearing some solutions from you, thanks.
Hello! It seems that the link to download the dataset is deprecated. Is that possible to provide another link?
Hi,
I just have one question.
When I am trying to test the network with some random captures, I don't know how to create the correct ground-truth for 3 static images I captured.
I simply used the triangle method in Kalatari's work, but my output is probably different from yours.
If I just read ref_hdr_aligned.hdr and convert it to png using gamma = 2.2, the output image looks very dark.
Can you share with me how you created the ground-truth for the test data you captured?
Thanks a lot.
Thank you for your work.
I have two question about the code.
1.In your paper, you have trained data by GPU.But when I run the code which you provide,the GPU is not work.
2.I don`t know how to align my pictures use homography transformation,can you provide to me your code?
HDR-VDP-2 is used to measure perceived visual quality of the HDR image in the actual luminance domain (with the unit of cd/m^2) instead of the normalized domain. In this work, since the luminance maps are normalized, how to compute HDR-VDP-2?
matlab HDR-VDP-2:
function res = hdrvdp( test, reference, color_encoding, pixels_per_degree, options ).
Which color_encoding did you choose?
Thanks.
Hi, thanks for providing the way of downloading Kalantari's dataset.
I find that there are three static LDR images named ref_i.tif
in each training scene. While I also down dataset from Kalantari's project web, and static LDR images are not provided.
So, are these pictures the original static images for generating the ground truth HDR? Or are them produced from ground truth HDR by apply inverse gamma correction?
Hi,
I a Masters student researching at a Signals Processing lab. In your paper, you mention homography alignment used prior in data preparation. However I could not find the corresponding code for this data preprocessing part. Could you please provide me with the code if it is available?
Thank you!
First, thank you for your great contribution to the HDR.
Your code is tremendously cool and very clear so it is easy to use even like me.
Comparing with other HDR codes based on deep learning, I think distinctive feature of yours is you use 3 inputs with different exposure so output image can easily reconstitute almost-white and almost-black area of input images.
However, I wonder how to speed up the process.
With 3 800x600 jpg inputs, it takes 0.39s per one output. (i7-7700 3.60Hz, GTX1080 8G)
(Only 5% of my GPU is used)
I want at least 5 FPS
For poor newbie, please give me some advice.
1 when i set the --test_h 682 --test_w 1024
ValueError: Dimension 1 in both shapes must be equal, but are 170 and 171 for 'generator/decoder/concat_1' (op: 'ConcatV2') with input shapes: [1,170,256,128], [1,171,256,128]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.