when i run "python yad2k.py cfg/yolo.cfg yolo.weights model_data/yolo.h5" ,the yolo.weights is 194MB.

Can you give a mini-tutorial on how to train this model and use it to predict bounding-boxes, on your own data. (given 20,000 images with bounding boxes, of objects not in ImageNet/coco). Just a few lines to get started. Do you have to first use the original Draknet code to train the model?

I wanted to achieve the benchmark of the yolov2 paper, which is a 78% mAP on voc2007+voc2012. I used retrain_yolo.py to train on voc2007+2012 and test on voc2007_test, but I only got a mAP of 37%. Did anyone recurrent the result successfully?

For voc, with 20 classes and 5 anchors, the final output layer is 13x13x125 tensor. I want the index of this output tensor containing class probability for class 0 (aeroplane),

is the index 5, 30, 55, 80, 105 ? i.e. for each output number, the first four indices are for the box, the fifth for the confidence and then 20 for class probablity?

or is the index 25,26,27,28,29? i.e the output number for each output of yolo? so first 5 are for box x, then next 5 for box y and so on...

In the original Darknet framework, I can process videos. Is it possible to process videos without opencv in this framework?

It takes over 60 seconds to load the h5 file. It is possible to reduce the time needed for this process?
Thank you

Hi, I want to try this Keras version of YOLO for training on ImageNet model for 1000 classes, What changes do I have to make? Like for YOLOv1 they have mentioned about it here: https://pjreddie.com/darknet/yolov1/ in Training YOLO section.
Any help for do that in this Keras version YOLO.

Thanks!

System : Ubuntu 16.04
Install :
python 3.5 and using alternative to set 3.5 is default.
-> update-alternatives --install /usr/bin/python python /usr/bin/python3.5 1
Install ( global ) : pip install -user packages...

In Quick Start.
./yad2k.py yolo.cfg yolo.weights model_data/yolo.h5
-> well done.

However,

./test_yolo.py model_data/yolo.h5

Using TensorFlow backend.
2017-06-19 10:41:16.146412: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146543: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146612: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146673: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146731: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Traceback (most recent call last):
File "./test_yolo.py", line 194, in
_main(parser.parse_args())
File "./test_yolo.py", line 79, in _main
yolo_model = load_model(model_path)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/models.py", line 246, in load_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/models.py", line 314, in model_from_config
return layer_module.deserialize(config, custom_objects=custom_objects)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/layers/init.py", line 54, in deserialize
printable_module_name='layer')
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/utils/generic_utils.py", line 140, in deserialize_keras_object
list(custom_objects.items())))
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/engine/topology.py", line 2450, in from_config
process_layer(layer_data)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/engine/topology.py", line 2419, in process_layer
custom_objects=custom_objects)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/layers/init.py", line 54, in deserialize
printable_module_name='layer')
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/utils/generic_utils.py", line 142, in deserialize_keras_object
return cls.from_config(config['config'])
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/engine/topology.py", line 1242, in from_config
return cls(**config)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/layers/advanced_activations.py", line 38, in init
self.alpha = K.cast_to_floatx(alpha)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/backend/common.py", line 108, in cast_to_floatx
return np.asarray(x, dtype=_FLOATX)
File "/home/ubuntu/.local/lib/python3.5/site-packages/numpy/core/numeric.py", line 531, in asarray
return array(a, dtype, copy=False, order=order)
TypeError: float() argument must be a string or a number, not 'dict'

I got this...

Hi @allanzelener ,

Thank you for your great work. I wonder if the current Loss Function works for non-square images?

In file keras_yolo.py, I see this:

    feats      = K.reshape(feats, [-1, conv_dims[0], conv_dims[1], num_anchors, num_classes + 5])
    conv_dims  = K.cast(K.reshape(conv_dims, [1, 1, 1, 1, 2]), K.dtype(feats))

    box_xy          = K.sigmoid(feats[..., :2]) 
    box_wh          = K.exp(feats[..., 2:4])
    box_confidence  = K.sigmoid(feats[..., 4:5])
    box_class_probs = K.softmax(feats[..., 5:])

I am trying to figure out if breaking box_xy --> box_x and box_y would allow me to use non-square image. Is there any other places in your code I need to be careful if I do that?

Thanks, :)

I'm having a problem testing with the tiny YOLO:

./yad2k.py cfg/tiny-yolo-voc.cfg tiny-yolo-voc.weights model_data/tiny-yolo-voc.h5
./test_yolo.py model_data/tiny-yolo-voc.h5

Are the postprocessings different for the yolo and tiny yolo? Could you have a look? Thanks very much for your sharing of this project, it's very helpful!

@allanzelener hello，when I run ./yad2k.py yolo.cfg yolo.weights model_data/yolo.h5
It takes error. like this:

section-counters defaultdict(<type 'int'>, {})
Traceback (most recent call last):
File "yad2k.py", line 269, in
_main(parser.parse_args())
File "yad2k.py", line 89, in _main
unique_config_file = unique_config_sections(config_path)
File "yad2k.py", line 60, in unique_config_sections
output_stream.write(line)
TypeError: unicode argument expected, got 'str'

How could I resolve it ？thank you .

How can I setup the number of images to process in batch?

Hi,
I'm trying to use your example code for classification within a Jupyter Notebook. Unfortunately, once I executed the cell with the code below, the kernel simply dies without any error message. The console looks like this:
´
I tried both with Tensorflow 1.0.1 and 1.1 (Python 3.5) , an error reported on Tensorflow killing the Jupyter Kernel was supposedly fixed in TF 0.9 already. Is there anything I can do to debug this issue?

[I 21:05:04.686 NotebookApp] Kernel restarted: 9fb36248-695a-44bf-9e19-7a604abf4d1b
2017-05-29 21:05:16.629087: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-29 21:05:16.629116: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-29 21:05:16.629125: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-05-29 21:05:16.629132: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-29 21:05:16.629138: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-05-29 21:05:16.912397: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-05-29 21:05:16.912739: I tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with properties: 
name: GeForce 920M
major: 3 minor: 5 memoryClockRate (GHz) 0.954
pciBusID 0000:04:00.0
Total memory: 1.96GiB
Free memory: 1.94GiB
2017-05-29 21:05:16.912767: I tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0 
2017-05-29 21:05:16.912775: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0:   Y 
2017-05-29 21:05:16.912785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce 920M, pci bus id: 0000:04:00.0)
[I 21:05:19.686 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel 9fb36248-695a-44bf-9e19-7a604abf4d1b restarted

sess = K.get_session()  # TODO: Remove dependence on Tensorflow session.
yolo_model = load_model(MODEL_PATH)
num_classes = len(class_names)
num_anchors = len(anchors)
model_output_channels = yolo_model.layers[-1].output_shape[-1]
assert model_output_channels == num_anchors * (num_classes + 5), \
        'Mismatch between model and given anchor and class sizes. ' \
        'Specify matching anchors and classes with --anchors_path and ' \
        '--classes_path flags.'
print('{} model, anchors, and classes loaded.'.format(MODEL_PATH))

    # Check if model is fully convolutional, assuming channel last order.
model_image_size = yolo_model.layers[0].input_shape[1:3]
is_fixed_size = model_image_size != (None, None)

# Generate colors for drawing bounding boxes.
hsv_tuples = [(x / len(class_names), 1., 1.) for x in range(len(class_names))]
colors = list(map(lambda x: colorsys.hsv_to_rgb(*x), hsv_tuples))
colors = list(map(lambda x: (int(x[0] * 255), int(x[1] * 255), int(x[2] * 255)),
            colors))
random.seed(10101)  # Fixed seed for consistent colors across runs.
random.shuffle(colors)  # Shuffle colors to decorrelate adjacent classes.
random.seed(None)  # Reset seed to default.

# Generate output tensor targets for filtered bounding boxes.
# TODO: Wrap these backend operations with Keras layers.
yolo_outputs = yolo_head(yolo_model.output, anchors, len(class_names))
input_image_shape = K.placeholder(shape=(2, ))
boxes, scores, classes = yolo_eval(yolo_outputs,input_image_shape,score_threshold=.3,iou_threshold=.5)

    
image = Image.open(SAMPLE_IMAGE)
if is_fixed_size:  # TODO: When resizing we can use minibatch input.
    resized_image = image.resize(tuple(reversed(model_image_size)), Image.BICUBIC)
    image_data = np.array(resized_image, dtype='float32')
else:
    # Due to skip connection + max pooling in YOLO_v2, inputs must have
    # width and height as multiples of 32.
    new_image_size = (image.width - (image.width % 32), image.height - (image.height % 32))
    resized_image = image.resize(new_image_size, Image.BICUBIC)
    image_data = np.array(resized_image, dtype='float32')
    print(image_data.shape)
image_data /= 255.
image_data = np.expand_dims(image_data, 0)  # Add batch dimension.

out_boxes, out_scores, out_classes = sess.run(
            [boxes, scores, classes],
            feed_dict={yolo_model.input: image_data,
                input_image_shape: [image.size[1], image.size[0]],
                K.learning_phase(): 0})

print('Found {} boxes for {}'.format(len(out_boxes), SAMPLE_IMAGE))

font = ImageFont.truetype(font=BASE_DIR +'/font/FiraMono-Medium.otf',size=np.floor(3e-2 * image.size[1] + 0.5).astype('int32'))
thickness = (image.size[0] + image.size[1]) // 300

for i, c in reversed(list(enumerate(out_classes))):
    predicted_class = class_names[c]
    box = out_boxes[i]
    score = out_scores[i]
    label = '{} {:.2f}'.format(predicted_class, score)

    draw = ImageDraw.Draw(image)
    label_size = draw.textsize(label, font)

    top, left, bottom, right = box
    top = max(0, np.floor(top + 0.5).astype('int32'))
    left = max(0, np.floor(left + 0.5).astype('int32'))
    bottom = min(image.size[1], np.floor(bottom + 0.5).astype('int32'))
    right = min(image.size[0], np.floor(right + 0.5).astype('int32'))
    print(label, (left, top), (right, bottom))

    if top - label_size[1] >= 0:
        text_origin = np.array([left, top - label_size[1]])
    else:
        text_origin = np.array([left, top + 1])
    # My kingdom for a good redistributable image drawing library.
    for i in range(thickness):
        draw.rectangle([left + i, top + i, right - i, bottom - i], outline=colors[c])
    draw.rectangle([tuple(text_origin), tuple(text_origin + label_size)],fill=colors[c])
    draw.text(text_origin, label, fill=(0, 0, 0), font=font)
    del draw

display(image)
sess.close()

I followed your readme exactly with your environment.yaml. I got the following error. I figure it's a problem when you initially created the weights and the current weights right? Do you know what the potential issue is and if there's an easy fix?

Parsing section reorg_0
Traceback (most recent call last):
File "./yad2k.py", line 270, in
_main(parser.parse_args())
File "./yad2k.py", line 236, in _main
name='space_to_depth_x2')(prev_layer))
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/keras/engine/topology.py", line 578, in call
output = self.call(inputs, **kwargs)
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/keras/layers/core.py", line 659, in call
return self.function(inputs, **arguments)
File "/home/ubuntu/Code/YAD2K/yad2k/models/keras_yolo.py", line 32, in space_to_depth_x2
return tf.space_to_depth(x, block_size=2)
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3399, in space_to_depth
block_size=block_size, name=name)
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 494, in apply_op
raise err
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 491, in apply_op
preferred_dtype=default_dtype)
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 702, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 110, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 99, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 441, in make_tensor_proto
tensor_proto.string_val.extend([compat.as_bytes(x) for x in proto_values])
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 441, in
tensor_proto.string_val.extend([compat.as_bytes(x) for x in proto_values])
File "/home/ubuntu/anaconda2/envs/yad2k/lib/python3.6/site-packages/tensorflow/python/util/compat.py", line 65, in as_bytes
(bytes_or_text,))
TypeError: Expected binary or unicode string, got Elemwise{add,no_inplace}.0

@shadySource @allanzelener does retrain_yolo.py shows the mAP score aswell...if not how can i calculate it?

@shadySource I tried running your code but due to the variable length of boxes i get an error saying that it expected input of 3 dimensions but got array of size (1,)

Hi,

h5py is necessary to run this command:

./yad2k.py cfg/yolo.cfg yolo.weights model_data/yolo.h5

and miniconda does not include the package by default.

However, conda install h5py works like a charm, so please consider adding the dependency to the README or environment.yml

Thank you for the project!

Convert the "office weight"(yolov2) from yolo website, I find the mAP nearly 0.57+, worse than the office give '0.768'，do you test the mAP in your model on the voc dataset?
Could you tell me the mAP of your test? (If you test the mAP). thank you~!

Hi !

I'm working on retraining this model on a new dataset to extract attributes from clothes (e.g sleeves type, neck type etc).

I have a set with images and for each image the attributes, but I don't have the bound boxes of each attribute on the image as it is in the PASCAL VOC dataset.

Do I need the bound boxes to retrain it ?

I run the convert yolo model to h5py model, but got this error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: dlopen(/usr/local/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so, 10): Library not loaded: @rpath/libcublas.8.0.dylib
  Referenced from: /usr/local/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
  Reason: image not found

I am using Tensorflow 1.1 and keras the newest version, macOS, I am not sure it's error about keras or tensorflow or simply about conda (I am not using Anaconda at all), could you help out what's going wrong? This prevent me dive into this very good project, I really like to try yolo-v2 on tensorflow. Many thanks.

Hello @allanzelener. Appreciating you in advance for your great implementation in keras. I have got one question. In some parts of your code, you use tf.space_to_depth function to transfer a tesnor of size (None, 38,38,64) to (None, 19,19,256). Could you please give me some hints or a numpy code to illustrate how this function works. Because as far as I know there is not such a function in pytorch and I would like to make it for myself. Could you please help?

Can someone point me to some resources so I can understand how to generate this file?
Thanks

@allanzelener ,when I run like this,but mat that error.It maybe is the keras ,I use keras1.1.1.

yad2k.py yolo.cfg yolo.weights model_data/yolo.h5
Using TensorFlow backend.
Loading weights.
Weights Header: [ 0 1 0 5133312]
Parsing Darknet config.
section-counters defaultdict(<class 'int'>, {})
Creating Keras model.
Parsing section net_0
Parsing section convolutional_0
conv2d bn leaky (3, 3, 3, 32)
Traceback (most recent call last):
File "yad2k.py", line 269, in
_main(parser.parse_args())
File "yad2k.py", line 186, in _main
padding=padding))(prev_layer)
TypeError: init() missing 1 required positional argument: 'nb_col'

The following code doesn't work

    if 'region' in cfg_parser.sections():
    #    print('generating anchors file')
        with open('{}_anchors.txt'.format(output_root), 'w') as f:
            print(anchors, file=f)

In def darknet19 I see logits = DarknetConv2D(1000, (1, 1), activation='softmax')(body)
Should I insert code to this place to get Softmax layer or it already exists?

Thanks.

Hey there!

I want to use pascal_classes.txt. (default is coco)

But I get an error with the anchors and don't quite understand what's going on or how to fix it. Any help would be awesome! Note: Everything works perfectly with coco.

AssertionError: Mismatch between model and given anchor and class sizes. Specify matching anchors and classes with --anchors_path and --classes_path flags.

in preprocess_true_boxes
compute box area box_area = box[2] * box[3]
but the box is relative value to original
To compute box area on final conv dim need multiply with conv_dim?
box_area = (box[2] *conv_width) * (box[3] * conv_height)

I am using yolo.cfg and yolo.weights, and then tried some fish images collected from imagenet. However, I found the fishes were both detected (which is excellent) but were classified as birds. Is that because I should use other cfg or weights, or should the model be retrained? Thanks a lot.

Hi @allanzelener
Could you please tell me, did you disable batchnorm layer in forward pass when you want to test an input image?
I have got one problem in transfered pytorch model. When I don't disable the batchnorm layer, the networks work well but when i disable it, I face with overflow issue in `yolo_head' for sigmoid function.

Hi @allanzelener,

Thanks for you great implementation. It was just like a tutorial. I appreciate your efforts. Finally, I have transfered your model to pytorch and reproduced your results.
I have got one more question. Did you validate your keras model on Pascal VOC test data and MSCoco test data? I mean did you calculate the mean average precision metric?

highly appreciating you in advance for your reply!

System : Ubuntu 16.04
Install :
python 3.5 and using alternative to set 3.5 is default.
-> update-alternatives --install /usr/bin/python python /usr/bin/python3.5 1
Install ( global ) : pip install -user packages...

In Quick Start.
./yad2k.py yolo.cfg yolo.weights model_data/yolo.h5
-> well done.

However,

./test_yolo.py model_data/yolo.h5

Using TensorFlow backend.
2017-06-19 10:41:16.146412: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146543: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146612: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146673: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-19 10:41:16.146731: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Traceback (most recent call last):
File "./test_yolo.py", line 194, in
_main(parser.parse_args())
File "./test_yolo.py", line 79, in _main
yolo_model = load_model(model_path)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/models.py", line 246, in load_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/models.py", line 314, in model_from_config
return layer_module.deserialize(config, custom_objects=custom_objects)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/layers/init.py", line 54, in deserialize
printable_module_name='layer')
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/utils/generic_utils.py", line 140, in deserialize_keras_object
list(custom_objects.items())))
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/engine/topology.py", line 2450, in from_config
process_layer(layer_data)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/engine/topology.py", line 2419, in process_layer
custom_objects=custom_objects)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/layers/init.py", line 54, in deserialize
printable_module_name='layer')
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/utils/generic_utils.py", line 142, in deserialize_keras_object
return cls.from_config(config['config'])
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/engine/topology.py", line 1242, in from_config
return cls(**config)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/layers/advanced_activations.py", line 38, in init
self.alpha = K.cast_to_floatx(alpha)
File "/home/ubuntu/.local/lib/python3.5/site-packages/keras/backend/common.py", line 108, in cast_to_floatx
return np.asarray(x, dtype=_FLOATX)
File "/home/ubuntu/.local/lib/python3.5/site-packages/numpy/core/numeric.py", line 531, in asarray
return array(a, dtype, copy=False, order=order)
TypeError: float() argument must be a string or a number, not 'dict'

I got this...
Any idea, please?

I am getting the following error when I use yad2k.py on yolov2:

Traceback (most recent call last):
File "yad2k.py", line 272, in
_main(parser.parse_args())
File "yad2k.py", line 256, in _main
model.save('{}'.format(output_path))
File "D:\Users...\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2429, in save
save_model(self, filepath, overwrite)
File "D:\Users...\Anaconda3\lib\site-packages\keras\models.py", line 101, in save_model
'config': model.get_config()
File "D:\Users...\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2246, in get_config
layer_config = layer.get_config()
File "D:\Users...\Anaconda3\lib\site-packages\keras\layers\core.py", line 668, in get_config
function = func_dump(self.function)
File "D:\Users...\Anaconda3\lib\site-packages\keras\utils\generic_utils.py", line 177, in func_dump
code = marshal.dumps(func.code).decode('raw_unicode_escape')
UnicodeDecodeError: 'rawunicodeescape' codec can't decode bytes in position 195-196: truncated \uXXXX
Exception ignored in: <bound method BaseSession.del of <tensorflow.python.client.session.Session object at 0x00000198C1BEACF8>>
Traceback (most recent call last):
File "D:\Users...\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 582, in del
UnboundLocalError: local variable 'status' referenced before assignment

I do not get the same error when I run yad2k.py on yolo-tiny.

I changed the code to only save the weights instead via model.save_weights('{}'.format(output_path)) and did not receive any errors, but when I tried to convert the model to json via model.to_json() I received virtually the same error. After doing some digging, people have suggested 2 possible issues:

1. Invalid layer names
2. keras has some issues saving models that are created with lambda layers, on some systems

I am guessing you do not see this issue, since it is the example case in the readme, but would you mind checking, and if not posting the versions of the required packages that you are using?

Hi @allanzelener.
Thank you again for your amazing implementation. I have got one more question. Could you please tell me, does your loss function do batch processing? If yes, Could you please tell me, how do you prepare your ground truth labels data for batch processing? Let me tell you what I mean. Actually what I want to do is to create a tensor named true_boxes with shape [batch, num_true_boxes, 5] for training. As you can guess, Number of bbox in every picture is different. One image may have 4 bboxes, another may have 2 or another 3. So It seems that the num_true_boxes in every image has different value. So how did you solve this issue?

Thanks!

Can I run the this model on keras on top of theano backend?

Gist here

Next is the cocoeval part so I can eventually turn all this into a proper PR if you think it makes sense. This is still a hack (I just took your code and extended the Layer class with it, so it's not clean keras) but I think it makes the project a little bit more readable and closer to the official API. The sess.run part will probably be tricky though...

Always open to criticism of course. Looking forward to your comments! :)

For below two lines, I guess you would like to use output_path, instead of output_root. Then it will use the same file name as keras model file.
https://github.com/allanzelener/YAD2K/blob/master/yad2k.py#L265
https://github.com/allanzelener/YAD2K/blob/master/yad2k.py#L266

@allanzelener In your implementation, you seem to have taken the output from the Keras model and do the post processing steps in tensorflow. I'm wondering what prevent you from using the 100% keras in the implementation, and why you need tensorflow in the postprocessing steps?

In the darkflow implementation, the post-processing steps are in pure python and completely decoupled from the model evaluation steps (for example, the post-processing steps are all in darkflow/net/yolov2/test.py).

I tried to take the Keras model in your implementation and feed into the post-processing steps in the darkflow implementation, and the class of the object is detected correctly, but the bounding box seems slightly off. I'm wondering whether you have any ideas/suggestions about this problem. Here is the detail of my experiment:

I first convert the darknet model into Keras model:

./yad2k.py cfg/tiny-yolo-voc.cfg tiny-yolo-voc.weights model_data/tiny-yolo-voc.h5

Then load the model into Keras and evaluate on an image:

image = Image.open('./images/eagle.jpg')
resized_image = image.resize((416,416), Image.BICUBIC)
image_data = np.array(resized_image, dtype='float32')
image_data /= 255.
image_data = np.expand_dims(image_data, 0)  # Add batch dimension.

model = load_model('./model_data/tiny-yolo-voc.h5')
net_out = (model.predict(image_data))[0]

I then feed the output from the network into the darknet post-processing steps:

plt.figure(figsize = (20,6))
plt.imshow(postprocess(net_out,plt.imread(image_path)))

And here is the result, we can see that the box is slightly off.

Here are the postprocess code:

class BoundBox:
    def __init__(self, classes):
        self.x, self.y = float(), float()
        self.w, self.h = float(), float()
        self.c = float()
        self.class_num = classes
        self.probs = np.zeros((classes,))

def overlap(x1,w1,x2,w2):
    l1 = x1 - w1 / 2.;
    l2 = x2 - w2 / 2.;
    left = max(l1, l2)
    r1 = x1 + w1 / 2.;
    r2 = x2 + w2 / 2.;
    right = min(r1, r2)
    return right - left;

def box_intersection(a, b):
    w = overlap(a.x, a.w, b.x, b.w);
    h = overlap(a.y, a.h, b.y, b.h);
    if w < 0 or h < 0: return 0;
    area = w * h;
    return area;

def box_union(a, b):
    i = box_intersection(a, b);
    u = a.w * a.h + b.w * b.h - i;
    return u;

def box_iou(a, b):
    return box_intersection(a, b) / box_union(a, b);

def prob_compare(box):
    return box.probs[box.class_num]

def prob_compare2(boxa, boxb):
    if (boxa.pi < boxb.pi):
        return 1
    elif(boxa.pi == boxb.pi):
        return 0
    else:
        return -1

def expit(x):
	return 1. / (1. + np.exp(-x))

def _softmax(x):
    e_x = np.exp(x - np.max(x))
    out = e_x / e_x.sum()
    return out

def postprocess(net_out, im, save = True):
	"""
	Takes net output, draw net_out, save to disk
	"""
	# meta
	H, W, _ = [13, 13, 125]
	threshold = 0.2
	C, B = 20, 5
	anchors = [1.08, 1.19, 3.42, 4.41, 6.63, 11.38, 9.42, 5.11, 16.62, 10.52]
	net_out = net_out.reshape([H, W, B, -1])

	boxes = list()
	for row in range(H):
		for col in range(W):
			for b in range(B):
				bx = BoundBox(C)
				bx.x, bx.y, bx.w, bx.h, bx.c = net_out[row, col, b, :5]
				bx.c = expit(bx.c)
				bx.x = (col + expit(bx.x)) / W
				bx.y = (row + expit(bx.y)) / H
				bx.w = math.exp(bx.w) * anchors[2 * b + 0] / W
				bx.h = math.exp(bx.h) * anchors[2 * b + 1] / H
				classes = net_out[row, col, b, 5:]
				bx.probs = _softmax(classes) * bx.c
				bx.probs *= bx.probs > threshold
				boxes.append(bx)

	# non max suppress boxes
	for c in range(C):
		for i in range(len(boxes)):
			boxes[i].class_num = c
		boxes = sorted(boxes, key = prob_compare)
		for i in range(len(boxes)):
			boxi = boxes[i]
			if boxi.probs[c] == 0: continue
			for j in range(i + 1, len(boxes)):
				boxj = boxes[j]
				if box_iou(boxi, boxj) >= .4:
					boxes[j].probs[c] = 0.


	colors = [(254.0, 254.0, 254), (239.88888888888889, 211.66666666666669, 127), 
              (225.77777777777777, 169.33333333333334, 0), (211.66666666666669, 127.0, 254),
              (197.55555555555557, 84.66666666666667, 127), (183.44444444444443, 42.33333333333332, 0),
              (169.33333333333334, 0.0, 254), (155.22222222222223, -42.33333333333335, 127),
              (141.11111111111111, -84.66666666666664, 0), (127.0, 254.0, 254), 
              (112.88888888888889, 211.66666666666669, 127), (98.77777777777777, 169.33333333333334, 0),
              (84.66666666666667, 127.0, 254), (70.55555555555556, 84.66666666666667, 127),
              (56.44444444444444, 42.33333333333332, 0), (42.33333333333332, 0.0, 254), 
              (28.222222222222236, -42.33333333333335, 127), (14.111111111111118, -84.66666666666664, 0),
              (0.0, 254.0, 254), (-14.111111111111118, 211.66666666666669, 127)]
	labels = ['aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 
              'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 
              'dog', 'horse', 'motorbike', 'person', 'pottedplant',
              'sheep', 'sofa', 'train', 'tvmonitor']
	if type(im) is not np.ndarray:
		imgcv = cv2.imread(im)
	else: imgcv = im
	h, w, _ = imgcv.shape
	for b in boxes:
		max_indx = np.argmax(b.probs)
		max_prob = b.probs[max_indx]
		label = 'object' * int(C < 2)
		label += labels[max_indx] * int(C>1)
		if max_prob > threshold:
			left  = int ((b.x - b.w/2.) * w)
			right = int ((b.x + b.w/2.) * w)
			top   = int ((b.y - b.h/2.) * h)
			bot   = int ((b.y + b.h/2.) * h)
			if left  < 0    :  left = 0
			if right > w - 1: right = w - 1
			if top   < 0    :   top = 0
			if bot   > h - 1:   bot = h - 1
			thick = int((h+w)/300)
			cv2.rectangle(imgcv, 
				(left, top), (right, bot), 
				colors[max_indx], thick)
			mess = '{}'.format(label)
			cv2.putText(imgcv, mess, (left, top - 12), 
				0, 1e-3 * h, colors[max_indx],thick//3)

	return imgcv

Hi, does it work with the newest version of keras ? (By the way,you have done a great job)

I tested the prediction time without visualization for your model using the test_yolo code by subtracting time before and after 'sess.run' :
https://github.com/allanzelener/YAD2K/blob/master/test_yolo.py#L142
and it comes out to be around 1350 ms. Am I going wrong in this caluculation somewhere or is this the time it takes?
Because according to the paper, the model should give around 50 fps.
Thankyou.

Firstly, thanks for your great work.
I have already converted the model to keras version (model_data/yolo.h5).
Then I would like to run "./test_yolo.py model_data/yolo.h" but it said that No such file or directory: 'model_data/yolo_anchors.txt'.
I search your repository but still can't find the txt file.
Anything I miss please point me out. Thanks!

Error Description:

Calling yolo_body twice causes an error:

File "/home/user/YAD2K/yad2k/models/keras_yolo.py", line 52, in yolo_body

conv13 = darknet.get_layer('leaky_re_lu_13').output

AttributeError: 'NoneType' object has no attribute 'output'

Example Program:

from keras.layers import Input
from yad2k.models.keras_yolo import yolo_body

anchors =[((1,2), (3,4)), ((5,6), (7,8))]
class_names = ['cat', 'dog', 'moose']

image_input = Input(shape=(416, 416, 3))

yolo_model = yolo_body(image_input, len(anchors), len(class_names))
yolo_model = yolo_body(image_input, len(anchors), len(class_names))

Thoughts:

I added darknet.summary() before the line it errors on, and first conv layer has the name 'conv2d_24', so the names are not resetting for some reason...

The second setup command (./yad2k.py cfg/yolo.cfg yolo.weights model_data/yolo.h5) yields this error:

Traceback (most recent call last):
  File "./yad2k.py", line 274, in <module>
    _main(parser.parse_args())
  File "./yad2k.py", line 155, in _main
    buffer=weights_file.read(weights_size * 4))
TypeError: buffer is too small for requested array

Where can I download the converted H5 file?

I would like to save the features of each detected bounding box for later use. How can this be done? How can I select the specific layer from which I want the features to be extracted?
Thank you

Hi everyone,

I'm new to deep learning and I want to train my own datasets on top of a pre-trained model here in YOLO. If I may ask 3 questions:

1. What image labeler can I use to specify the ground truth ROI of my dataset?

2. How can I supply the annotated data into this YOLO keras frame work?

3. On which layer in YOLO should I build my dataset on (direction in the code will be helpful)?

Thank you so much!

Hi,

I was wondering how to convert the .h5 file of Keras to the .weights file used by Darknet. If I undertsand, the .h5 file created with YAD2K is the entire model. I find on Keras documentation that you can save only the weight of this model, but the data of this .h5 is not the same as the data in the .weight file.

What transformations I have to do to convert this file into .weight ?

Thank you for your attention and your help!

Unfortunately, the version you specified is no the right one (2.0.3).
Because of the recent keras updates the imports are broken!

Does this work for yolo9000? I cant make it work right now

on line number 136 in keras_yolo inside the function yolo_head
you say that
# Note: YOLO iterates over height index before width index.

You calculate pred_xy(used for IOU calculations in yolo_loss) by calling yolo_head.
In yolo_head you calculate pred_xy by
adding to the predicted offsets box_xy and the index values conv_index.

which is calculated by :
conv_index = K.transpose(K.stack([conv_width_index,conv_height_index]))

shouldn't this be
conv_index = K.transpose(K.stack([conv_height_index,conv_width_index]))

like for example
conv[0][2][3]
should be array([[2,3]]) instead of
array([[3,2]]) <-------THIS IS IMPLEMENTED CURRENTLY