ameraner / dsen2-cr Goto Github PK

Hi, @ameraner
I have a small issue about the dataset import, so I would like to ask for your help.
The error below appeared, but it appears that the import was not imported properly.
""
Model compiled successfully!
Getting file lists
Number of train files found: 3
Number of validation files found: 3
Number of test files found: 3
Training model name: DSen2-CR_01
Generator initialized
Generator initialized
Generator initialized
Training starts...

Epoch 1/8
Traceback (most recent call last):
File "dsen2cr_main.py", line 179, in
run_dsen2cr(args.predict_file, args.resume_file)
File "dsen2cr_main.py", line 168, in run_dsen2cr
input_shape, max_queue_size, use_multi_processing, workers)
File "/home/soyeon/Others/DSen2-CR-main/Code/dsen2cr_tools.py", line 131, in train_dsen2cr
workers=workers)
File "/home/soyeon/DSen2CR/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 1296, in fit_generator
steps_name='steps_per_epoch')
File "/home/soyeon/DSen2CR/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_generator.py", line 301, in model_iteration
aggregator.finalize()
File "/home/soyeon/DSen2CR/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_utils.py", line 138, in finalize
raise ValueError('Empty training data.')
ValueError: Empty training data.
""
I completed the composition of the input data as written along ReadMe.
Datasetfilelist.csv was constructed, and three folders named S1, S2, and S2_cloudy were created in the path where the csv file exists. Each image TIFF file is stored in each folder.

Example of Datasetfilelist.csv : 1, S1, S2, S2_cloudy, ROIs1970_fall_1_p32.tif

May I know what I missed?

Hi,

I have this error:

NotImplementedError: Cannot convert a symbolic Tensor (lambda_17/strided_slice:0) to a numpy array.

in function DSen2CR_model in file dsen2cr_network.py when try to create lambda conversion in line 66

x = Lambda(concatenate_array)(x)

Hi, I'm using my own Input-Data and I'm becoming the: Index Error: list out of range.

Here is the full issue:

Traceback (most recent call last):
File "dsen2cr_main.py", line 179, in
run_dsen2cr(args.predict_file, args.resume_file)
File "dsen2cr_main.py", line 160, in run_dsen2cr
max_val_sar, scale)
File "C:\Users\maria\Documents\GitHub\dsen2-cr\Code\dsen2cr_tools.py", line 169, in predict_dsen2cr
for i, (data, y) in enumerate(predict_generator):
File "C:\Users\maria\anaconda3\envs\dsen2cr\lib\site-packages\keras\utils\data_utils.py", line 372, in iter
for item in (self[i] for i in range(len(self))):
File "C:\Users\maria\anaconda3\envs\dsen2cr\lib\site-packages\keras\utils\data_utils.py", line 372, in
for item in (self[i] for i in range(len(self))):
File "C:\Users\maria\Documents\GitHub\dsen2-cr\Code\tools\dataIO.py", line 280, in getitem
self.random_crop_paramy[indexes])
File "C:\Users\maria\Documents\GitHub\dsen2-cr\Code\tools\dataIO.py", line 308, in __data_generation
random_crop_paramy_temp, data_type=3)
File "C:\Users\maria\Documents\GitHub\dsen2-cr\Code\tools\dataIO.py", line 407, in get_batch
data_image = self.get_data_image(ID, data_type, random_crop_paramx_temp[i], random_crop_paramy_temp[i])
File "C:\Users\maria\Documents\GitHub\dsen2-cr\Code\tools\dataIO.py", line 354, in get_data_image
data_path = os.path.join(self.input_data_folder, ID[data_type], ID[4]).lstrip()
IndexError: list index out of range

The output folder is also created but don't get any output images.
It wolud be very nice if someone could give me an advice for this error.

Does the input have to be cloudy img and SAR img?
Can I just use a single 3-channel jpg as the input image?

When using the model_SARcarl.hdf5 model directly for prediction, does s2_cloudy have to include all 13 bands? Do the sizes of s1 and s2_cloudy also have to be exactly the same? In addition, what format should be set for the names of subfolders and images in the input_data_folder path? Thanks~

Hi , Is it possible to deploy the model?
We always giving triplet data,But how can we will predict only on Real test data without Groundtruth?
Could you please suggest me on this.
Thank you

When I run the dsen2cr_main.py , I encounter this error. Could you please advise on how to resolve it?

Hi,

I am trying to use your trained model to do some predictions. I have preprocessed my S1 and S2 images to the same same size as your inputs but still get the error message below. Could you please provide some samples of your datasets for us to understand how to preprocess our own data?

Error message: "tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,13,128,128] vs. shape[2] = [16,1,128,128]"

BTW, when I select the testing model, do I use the s2_cloudy and s1 images automatically?

Thanks in advance!

Hi, I want to test the pretrained model model_noSARcarl.hdf5 with this image from your dataset :

I also modify this variable :

and this

but i get the following error :

I tried to change this:

to this :

but doesn t work...

Can you help me figure it out?

Hello @ameraner,

After testing the pre-trained CARL model on the hold-out test dataset in train_val_test_patches.csv (7899 images) with a batch_size of 1, I see that a csv file is generated with each line containing the metrics for a specific batch. How would I achieve a single number metric as seen in table 1 of your paper? Would I just have to calculate the column-wise average?

After plotting the values from that csv file in a histogram I see that in the MAE and RMSE categories, there are a few testing images where the model achieves equal or better results to the ones recorded in the paper, but in general MAE and RMSE seem to fall around .11 and 0.198 as opposed to .0290 and .0366 seen in the paper, could there be a reason for this?

Thank you

In file dsen2cr_pytorch_model.py line 7 "from .base_model import BaseModel", where is the BaseModel class?

I had tried to load the .hdf5 trained model checkpoint from DSen2-CR model trained by optimizing the CARL loss and model trained on a plain L1 loss in tensorflow and pytorch but in both cases, the weights are empty.

Is it a download problem or is the file is corrupt or maybe something else? Can u please check?

Hello,

Could you provide how you split your data into train/val/test？
I would like to train the model only on RGB and NIR channels.

Thank you.

I downloaded a small part of the SEN12MS-CR dataset for experiments.For example, the file names in the three folders (s1, s2_cloudFree, s2_cloudy) are ROIs1158_spring_s1_1_p30.tif, ROIs1158_spring_s2_1_p30.tif, ROIs1158_spring_s2_cloudy_1_p30.tif.
Then, for convenience, I rename the files in each folder to 1.tiff, 2.tiff, 3.tiff.... The corresponding datasetfilelist.csv has also been changed.
But the final cloud removal prediction picture inputred.png is strange. It can achieve the effect of cloud removal, but it looks like the enlargement of an area of inputtarg.png (groundtruth). The features of inputred.png and inputtarg.png (groundtruth) do not correspond. As shown in the picture below.
I don't know where my problem is. I think it is in the data file reading, but I don't understand. Has anyone ever had a similar problem? What should I do to get real results? thank you.

Hello @ameraner,
After changing n_gpus=2 for model training, when performing prediction tasks, I encountered the issue 'len(layer_names) != len(filtered_layers)'. The layer_names in the weight file .hdf5 only contains 8 layers, which is significantly different from the model structure. Do you know where the problem lies, or do I need to change other parameters besides n_gpus when using multiple GPUs for training?

After installing said libraries and formatting csv file as it's described in the readme file I managed to make the code run, however, the predicted output is exactly like the photo with cloud I have inputted. Has anyone else encountered something similar? What should I do to get the actual result? Thank you

Hello @ameraner ,
You have a wonderful project here, and I am interested in replicating the results that your team has achieved as a first step.

In the paper I see that you are using an adam optimizer with momentum of 0.5.
Does that correspond with the three parameters beta_1, beta_2, and epsilon here?

Also, in the paper I see a multiplicative factor used to decay the learning rate.
I was thinking of training the first 5 epochs first with schedule_decay=0, then to use --resume for each epoch afterwards with schedule_decay=0, but multiplying the multiplicative factor to the previous lr. Is that similar to the process you used or is there a better way to do that?

Thank you

Hello @ameraner, what is the pre-trained model that I can test with input only a Sentinel-2 image without SAR image? Thanks :)

Hi ameraner,
I'm new to Ubuntu LST and also cloud removal. I have tried to run the main code but I have encountered this problem:

Model compiled successfully!
Getting file lists
Number of train files found: 3
Number of validation files found: 3
Number of test files found: 3
Training model name: DSen2-CR_001
Traceback (most recent call last):
File "dsen2cr_main.py", line 179, in
run_dsen2cr(args.predict_file, args.resume_file)
File "dsen2cr_main.py", line 168, in run_dsen2cr
input_shape, max_queue_size, use_multi_processing, workers)
File "/home/atk/dsen2-cr/Code/dsen2cr_tools.py", line 19, in train_dsen2cr
out_path_train = make_dir(os.path.join(base_out_path, model_name))
File "/home/atk/dsen2-cr/Code/tools/dataIO.py", line 18, in make_dir
os.makedirs(dir_path)
File "/home/atk/miniconda3/envs/dsen2cr_env/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/atk/miniconda3/envs/dsen2cr_env/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/atk/miniconda3/envs/dsen2cr_env/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 1 more time]
File "/home/atk/miniconda3/envs/dsen2cr_env/lib/python3.7/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/path'

How can I fix this ?
Waiting for your reply. Thank you very much.

I have tested the model model_SARcarl.hdf5 with this .csv:

having previously made the changes:

and

but I got the following result, which doesn't seem good to me... am I wrong?

Where is dataIO library from? I can not find anything with the functions used in the dsen2cr_tools.py code.

Best regards,
Raminta

Hello,
Could you please update the link to the train_val_test_patches.csv file?
Currently, the link to access the CSV file is not available.
Thank you.

Hello,

Could you please tell me the details about the 10 ROIs for validation and 10 ROIs for testing?

Regards.

I'm trying to use the pre-trained model for prediction using the following command:

python dsen2cr_main.py --predict model_SARcarl.hdf5

But I'm getting the following error. Do you have any idea why this is?

Traceback (most recent call last):
File "dsen2cr_main.py", line 198, in
run_dsen2cr(args.predict_file, args.resume_file)
File "dsen2cr_main.py", line 145, in run_dsen2cr
include_sar_input=include_sar_input)
File "C:\Users\Catalin\Desktop\cloud removal\dsen2cr_network.py", line 52, in DSen2CR_model
x = Add()([x, input_opt])
File "C:\Users\Catalin\Soft\anaconda3\dsen2cr_env\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 824, in call
self._maybe_build(inputs)
File "C:\Users\Catalin\Soft\anaconda3\dsen2cr_env\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 2146, in _maybe_build
self.build(input_shapes)
File "C:\Users\Catalin\Soft\anaconda3\dsen2cr_env\lib\site-packages\tensorflow_core\python\keras\utils\tf_utils.py", line 306, in wrapper
output_shape = fn(instance, input_shape)
File "C:\Users\Catalin\Soft\anaconda3\dsen2cr_env\lib\site-packages\tensorflow_core\python\keras\layers\merge.py", line 111, in build
output_shape = self._compute_elemwise_op_output_shape(output_shape, shape)
File "C:\Users\Catalin\Soft\anaconda3\dsen2cr_env\lib\site-packages\tensorflow_core\python\keras\layers\merge.py", line 82, in _compute_elemwise_op_output_shape
'together with shapes ' + str(shape1) + ' ' + str(shape2))
ValueError: Operands could not be broadcast together with shapes (13, 128, 13) (13, 128, 128)

Hello, I noticed that when running your model for prediction, the resulting output is concatenated with the optical imagery, followed by a lambda operation. In the end, the output shape of the model is (16, 27, 128, 128) instead of (None, 13, 128, 128). Is there any issue with this?