r4ghu / ios-coreml-yolo Goto Github PK

View Code? Open in Web Editor NEW

162.0 10.0 25.0 15.41 MB

Almost Real-time Object Detection using Apple's CoreML and YOLO v1 -

Home Page: https://goo.gl/8eVso5

Swift 1.81% Objective-C 0.20% Jupyter Notebook 97.28% Python 0.71%

coreml ios object-detection yolov1 object-detection-pipelines

ios-coreml-yolo's Introduction

iOS-CoreML-Yolo

This is the implementation of Object Detection using Tiny YOLO v1 model on Apple's CoreML Framework.

The app fetches image from your camera and perform object detection @ (average) 17.8 FPS.

Requirements

Xcode 9 and above
iOS 11 and above
For training: Python 2.7 (Keras 1.2.2, TensorFlow 1.1, CoreMLTools 0.7)

Usage

To use this app, open iOS-CoreML-MNIST.xcodeproj in Xcode 9 and run it on a device with iOS 11. (You can also use simulator)

Model conversion

In this project, I am not training YOLO from scratch but converting the already existing model to CoreML model. If you want to create model on your own.

Create Anaconda environment. Open your terminal and type the following commands.

$ conda create -n coreml python=2.7
$ source activate coreml
(coreml) $ conda install pandas matplotlib jupyter notebook scipy scikit-learn opencv
(coreml) $ pip install tensorflow==1.1
(coreml) $ pip install keras==1.2.2
(coreml) $ pip install h5py
(coreml) $ pip install coremltools

Download the weights from the following link and move it into ./nnet directory.
Enter the environment and run the following commands in terminal with ./nnet as master directory.

(coreml) $ sudo python convert.py

I also included a jupyter notebook for better understanding the above code. You need to use it with root permissions for mainly converting the keras model to CoreML model. Initialise the jupyter notebook instance with the following command:

(coreml) $ jupyter notebook --allow-root

CoreML Model Download [New]

The converted CoreML model can be downloaded here:

Google Drive Link: Download

Tutorial

If you are interested in creating the Tiny YOLO v1 model on your own, a step-by-step tutorial is available at - Link

Results

These are the results of the app when tested on iPhone 7.

Author

Sri Raghu Malireddi / @r4ghu

ios-coreml-yolo's People

Contributors

Stargazers

Watchers

ios-coreml-yolo's Issues

Explanation for permutation op

In the model yoloP1P2P3, there is the following layer

model.add(Permute((2,3,1)))

Though it gives correct results, I want to ask about the specific order of (2,3,1). In my understanding, we are switching from Height*Width*Depth convention to Depth*Height*Width , and in that case, shouldn't the tuple be (3,1,2)?

Where is my understanding incorrect?

Can't access Google drive from China because of the GFW.

This is not an issue, but may I ask to get the TinyYOLOv1.mlmodel, I am in China, the god damn it GFW avoid me from access Google, so can you give me another link please? That would be many many thanks if I can have a accessable download link of that TinyYOLOv1.mlmodel model!!!

doesn't detect anything

Orientation, Camera are weird on my iPad

it's always conversely rotation with any iPad position in my hands

also weird zoom, objects on the camera are closer than should be

p.s. I also don't get any detected objects (drawn rectangles), mb it's because of the wrong frames orientation and yolo just doesn't detect anything

...

fixed zoom with setting
session.sessionPreset = AVCaptureSessionPresetPhoto, iPad just isn't 16:9

the convert pipeline is broken

seem the coremltools is not imported, after imported below error happens

Traceback (most recent call last):
  File "convert.py", line 133, in <module>
    coreml_model_p1p2p3 = coremltools.converters.keras.convert(model_p1p2p3,
AttributeError: 'module' object has no attribute 'convert'

Can you create a custom model rather than using pre defined model

Hi,

You are using yolo model which is already trained. Can you able create a custom model to detect image which is used in training. ?

Thank you.

How to create our own images weights rather than pre-trained weights

Hi r4ghu,

I want to create a custom object detection model using Yolo. In your tutorial i found that the pre trained weights. But can you tell me how to create our own images weights.

Thanks in Advance.

Explanation for loading weights

In the load_weights function, there is this line, which ignores the first few values of the loaded weights:

data = data[4:]

Can you please explain why we are doing so? And why is this even working?