sthalles / deeplab_v3 Goto Github PK
View Code? Open in Web Editor NEWTensorflow Implementation of the Semantic Segmentation DeepLab_V3 CNN
License: MIT License
Tensorflow Implementation of the Semantic Segmentation DeepLab_V3 CNN
License: MIT License
tensorflow.python.framework.errors_impl.OutofRangeError: End of sequence
Nice work done. Just curious, how much time does it take to train the model? Some other models, such as Mask R-CNN, take more than one days to train on a 8-GPU machine using standard COCO dataset. I wonder if DeepLab has any advantages of training over other models, since I want to training it personally if the computation cost is not too expensive. Thank you.
I would like to train the network on a mine dataset, how do the masks have to be prepared?
I have RGB images where every pixel is:
[0,0,0] -> background
[1,1,1] -> dog
[2,2,2] -> cat
are they okay?
Hi,
Thanks for your open-source code of your project. I have few questions when I use my private data to train the model. My data only have person and background, the foreground of the ground-truth is 255, the background of the ground-truth is 0, and I modify number_of_classes=1
, but when I use resnet_v2_50 to train or your 16645 model to finetune, I get Average train loss: 0.0 Global Validation Avg Loss: 0.0 MIoU: 1.0
when I begin.
How can I fix that?
Looking forward to your reply!
Thanks for great work.
I have a question about the inference.
In the original Deeplab_v3 paper, there are some comments about running inferences with Cityscapes dataset. They said "We adopt the same training protocol as before except that we employ 90K training iterations, crop size equal to 769, and running inference on the whole image, instead of on the overlapped regions as in [11]."
I found that running the codes in this repository produces the inference results with the cropped size(513x513). I want to know whether I could also run inference on the whole image, using the model trained on the cropped images.
Hi and thanks for your well-written blog post and code!
I think I encountered a bug in the script for testing. Seems like the argument "resnet_model" is not available in test.py. I'm currently on TF 1.5 so it might be related to this, but as far as I can tell from a quick glance through the code I think it's just a missing argument.
Calling the script as described in the README using:
python test.py --model_id=16645
yields an error:
resnet = getattr(resnet_v2, args.resnet_model)
TypeError: getattr(): attribute name must be string
I was able to fix it by replacing all occurrences of args.resnet_model
with "resnet_v2_50"
(which is a horrible hack tbh), so I assume the argument is actually just missing in test.py atm.
Cheers
Lukas
Hi, thanks for sharing your code.
I wish to know the results(mIOU) compared with the original paper.
I can not download the dataset from the Semantic Boundaries Dataset and Benchmark using the link, could you please share the dataset directly? @sthalles
Thanks for this great share!
When I attempt to download the datasets, it's in a drive called MEGA, and it sets many barriers to download this datasets.... after several hours, I'm waiting for the free quota during the download process.
Could you please consider to make a copy of datasets to google drive or other free online drives?
Thank you very much,
Xiao
I don't know what mean "label_image.shape[0]==263169",why it is 263169?
When I run the command: python test.py --model_id=16645, it throws the # TypeError:scale_image_with_crop_padding() missing 1 required positional argument :'crop_size'.
so where is the wrong thing?thx.
Thanks for your working in Deeplab. Could you tell me how to prepare dataset and run your code? Thanks
GTX 1080 8G is not enough.
How do I train this on a custom dataset?
When I am trying to train, I get the following error:
2018-07-15 22:31:21.210544: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 750000 values, but the requested shape has 187500
[[Node: annotation_reshape = Reshape[T=DT_UINT8, Tshape=DT_INT32](DecodeRaw_1, annotation_reshape/shape)]]
This seems to be related to the shape in tfrecord file. I used the code provided in the repository to create those files. Should I update them?
I'm editing CreateTfRecord file so I can use my own dataset. When I run the last 2 cells I have the following error message "TypeError: 'NoneType' object is not iterable". I uploaded a text file with the whole thing.
error.txt
image_level_features = tf.reduce_mean(net, [1, 2], name='image_level_global_pool', keep_dims=True)
image_level_features = slim.conv2d(image_level_features, depth, [1, 1], scope="image_level_conv_1x1",
activation_fn=None)
image_level_features = tf.image.resize_bilinear(image_level_features, (feature_map_size[1], feature_map_size[2]))
The dim of input tensor net is [ batch,height,width,deepth], so after resuce_mean it became to [batch,1,1,deepth], afetr 1X1 conv, it became to[batch,1,1,256], I want to know why it can to be[batch,heihgt,width,256] after resize?
Hi,
So basically I just launched your code using Tensorflow 1.7.0 on your data (I downloaded them from the media fire link), and I tried to retrain the network using the resnet v2 50 and I get NaN losses when testing... I figured it has something to do with the is_training placeholder:
-If set to False the output from the network (softmax, the "logits_tf" op (?)) is a 513x513x21 matrix filled with NaN values.
-if set to True it is okay.
I am training on a GTX 1080Ti, and I used the following command:
python train.py --starting_learning_rate=0.00001 --batch_norm_decay=0.997 --crop_size=513 --gpu_id=0 --resnet_model=resnet_v2_50
Do you know why it does that, feels like it's batchnorm related as always with tensorflow...
I also tried with tensorflow 1.8, it did not work.
Thanks in advance,
Antoine
@sthalles Hi, thanks for your blog and codes.
I have some questions since I am a green hand. I followed the README.md and finished downloading the datasets. But I don't know how to go on with next work. Could you please tell me the next steps? I don't understand the steps in README.md very much.
Thank you!
My model has finished training.
How to output the predicted image as shown in the figure below your article?
Thank you!
Should test.tfrecords be in the same format as train.tfrecords and validation.tfrecords? I was able to generate train and validation for a new data set and successfully train a model. However, when trying test.py, I have been getting reshaping errors like this one: ValueError: cannot reshape array of size 195453 into shape (381,795)
. Should test.tfrecords be created differently?
How to predict my own pictures?Looking forward to your guidance
I am trying to run this on my own custom dataset. I have followed the steps and created tfrecords. When I run the training after about ten minutes I get the following error:
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
[[Node: IteratorGetNext = IteratorGetNext[_class=["loc:@map/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], output_shapes=[[?,513,513,3], [?,513,513], [?], [?]], output_types=[DT_FLOAT, DT_INT32, DT_INT32, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorFromStringHandle)]]
Caused by op 'IteratorGetNext', defined at:
File "train.py", line 95, in <module>
batch_images_tf, batch_labels_tf, _ = iterator.get_next()
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 370, in get_next
name=name)), self._output_types,
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 1466, in iterator_get_next
output_shapes=output_shapes, name=name)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
op_def=op_def)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
OutOfRangeError (see above for traceback): End of sequence
[[Node: IteratorGetNext = IteratorGetNext[_class=["loc:@map/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], output_shapes=[[?,513,513,3], [?,513,513], [?], [?]], output_types=[DT_FLOAT, DT_INT32, DT_INT32, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorFromStringHandle)]]
Any idea what would cause this?
Thanks
Can you provide a pre-train model for test?
Thanks for great work.
I found that checkpoints files were written successfully. I used these files with test.py to load the saved model.
On the other hand, I want to know whether I could load the saved checkpoints files for retraining.
Hello, it is possible to start test for separate images not for tfrecords file? I want to use with network for segmentation object on a video. I have a problem with tensorflow. How I need to edit test.py? Thank you for help.
Hi
When I run the train.py, there are errors
InvalidArgumentError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to get matching files on ./resnet/checkpoints/resnet_v2_50.ckpt: Not found: FindFirstFile failed for: ./resnet/checkpoints : ϵͳ\udcd5Ҳ\udc
Where should I download the resnet_v2_50.ckpt file?
tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
[[Node: IteratorGetNext = IteratorGetNext_class=["loc:@map/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], output_shapes=[[?,456,456,3], [?,456,456], [?], [?]], output_types=[DT_FLOAT, DT_INT32, DT_INT32, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Caused by op 'IteratorGetNext', defined at:
File "train.py", line 95, in
batch_images_tf, batch_labels_tf, _ = iterator.get_next()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/data/ops/iterator_ops.py", line 373, in get_next
name=name)), self._output_types,
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_dataset_ops.py", line 1745, in iterator_get_next
output_shapes=output_shapes, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3414, in create_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1740, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
OutOfRangeError (see above for traceback): End of sequence
[[Node: IteratorGetNext = IteratorGetNext_class=["loc:@map/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], output_shapes=[[?,456,456,3], [?,456,456], [?], [?]], output_types=[DT_FLOAT, DT_INT32, DT_INT32, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Hi
I am instantiating your model from the checkpoint that you have provided and I am looking to add some additional layers to the end.
I am using it in the following way:
path = os.path.join(model_path, model_name)
saver = tf.train.import_meta_graph(path + '.meta')
graph = tf.get_default_graph()
trained_output = graph.get_tensor_by_name("cond/DeepLab_v3/logits/Conv2D:0")
final_conv_layer = tf.layers.conv2d(trained_output, num_classes, 1,
kernel_regularizer=tf.contrib.layers.l2_regularizer(0.001),
kernel_initializer=tf.truncated_normal_initializer(mean=0, stddev=0.01),
name='final_conv_layer')
final_layer = tf.image.resize_bilinear(final_conv_layer, image_shape, name='final_layer')
When I print out all the tensors in the network, I see that you have two placeholders called Placeholder(line:15 in output1.txt) and Placeholder_1(line:38 in output1.txt).
Can you provide some info on what each of these placeholders needs as an input ?
The code that i used to look at the all the operation names is as follows:
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver.restore(sess, save_path="./models/model.ckpt")
op = sess.graph.get_operations()
op_list = [m.name for m in op]
for operation in op_list:
print(operation)
I am attaching the output1.txt file which contains the names of the tensors and the .py file I use to generate it (deeplab_test.txt).
Thanks for your help!
output1.txt
deeplab_test.txt
P.S - Thanks for the medium article. It was helpful in understanding how DeepLab works.
Last feature has size (32x32) and there is a convolution with kernel size 3 with dilation rate 18. So in order that the feature map would be valid one the feature size should be (2*18+3 =39). Is that correct?
./resnet/checkpoints/resnet_v2_50.ckpt: Not found: ./resnet/checkpoints; No such file or directory
thinks very much
Hi, it seems that the code do not contain multi-grid part in it, or maybe I miss something..?
I'm confusing about which block should apply multi grid, in the paper. Block 4 ? or Block 4- 5,6,7..(deeper)?
Or, apply multi-grid to all ASPP parallel layers?
Many thanks!
Global step: 100 Average train loss: 3.046896914243698 Global Validation Avg Loss: nan MIoU: 0.10376305785030127
Global step: 200 Average train loss: 2.418999447822571 Global Validation Avg Loss: nan MIoU: 0.04654880184680223
is it possible to fine tune the model on a custom dataset which might have different number of classes?
of course a pre-trained weights file must exist either
Hi sthalles, Thanks for your great work!
I get a little confusion about CreateTfRecord.ipynb.
In CreateTfRecord.ipynb, I found the two types of annotation are different.
1,
If "annotation_np = imread(os.path.join(annotations_dir_voc, image_name.strip() + ".png"))",
then the result label is :
annotation_np.dtype : uint8, annotation_np.shape: (w,h,4), np.max(annotation_np): 255, np.min(annotation_np)): 0.
2,
If "annotation_np = read_annotation_from_mat_file(annotations_dir_aug_voc, image_name)",
then the result label is :
annotation_np.dtype : uint8, annotation_np.shape: (w,h), np.max(annotation_np): 15, np.min(annotation_np)): 0.
What is the meaning of the annotation_np when its shape is (w,h,4)? If it will cause something wrong when train the model?
Hi there, i am student who is interested in this project,
I just want to know where to get the "the pre-trained resnet-50 or resnet-101 models"
The link given in the README leads to a paper, which means I should ask the author for the models?
Or is there a download link please?
when running the deeplab_client.ipynb
notebook, i got No module named 'grpc'.
`ModuleNotFoundError Traceback (most recent call last)
in ()
1 from future import print_function
2 from PIL import Image
----> 3 from grpc.beta import implementations
4 import tensorflow as tf
5 import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'grpc'install both
$ python -m pip install grpcio
$ python -m pip install grpcio-tools`
but the problem remains.
hello , after creating the custom tfrecord file i tried to run train.py
but it gives me the following errors
ResNet checkpoints file successfully found.
Tensoboard folder: ./tboard_logs\12028
2018-08-20 08:44:58.956467: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Model checkpoits for resnet_v2_50 restored!
Traceback (most recent call last):
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 1322, in _do_call
return fn(*args)
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 34799360 values, but the requested shape has 8699840
[[Node: annotation_reshape = Reshape[T=DT_UINT8, Tshape=DT_INT32](DecodeRaw_1, annotation_reshape/shape)]]
[[Node: IteratorGetNext = IteratorGetNext[_class=["loc:@map/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], output_shapes=[[?,900,900,3], [?,900,900], [?], [?]], output_types=[DT_FLOAT, DT_INT32, DT_INT32, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorFromStringHandle)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train.py", line 190, in <module>
handle: training_handle})
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 900, in run
run_metadata_ptr)
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 1316, in _do_run
run_metadata)
File "C:\python3.6\lib\site-packages\tensorflow\python\client\session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 34799360 values, but the requested shape has 8699840
[[Node: annotation_reshape = Reshape[T=DT_UINT8, Tshape=DT_INT32](DecodeRaw_1, annotation_reshape/shape)]]
[[Node: IteratorGetNext = IteratorGetNext[_class=["loc:@map/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], output_shapes=[[?,900,900,3], [?,900,900], [?], [?]], output_types=[DT_FLOAT, DT_INT32, DT_INT32, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorFromStringHandle)]]```
Up till now 4 editions of DeepLabs papers have been released.
The blog post mentions the paper no. 1706.05587, while the repo Read-me mentions the previous edition numbered 1606.00915.
Which edition of DeepLab is the implementation in this repo based on?
I am aiming to train on my custom dataset and want to refer to the paper to get a better understanding of your implementation.
I used your pre-trained model on the test data, I got mIOU 70%, not 74%. Why?
This is not a issue,but a question,sorry to put it here.
I want to do human body segmentation on Android phones using the frontal camera,the picture from camera,its width/height is 9/16=0.5625,does the input image's width/height ratio influence the segmentation results?Or should I resize the image to width/height =1.0?And how much training image is need to get a good segmentation result?I am new to image segmentation,and now I can get only about 1k images,Thank you very much if you can help me!
Can this project run at lower version TensorFlow?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.