alianglan / ios-projects-with-ml-models Goto Github PK

This challenge is that using machine learning model created from TensorFlow on iOS with Core ML or ML Kit(TensorFlow Lite).

‼️ PR for English advice always makes me happy ‼️

한국어 README

• Core ML
• ML Kit
• etc. (Tensorflow Lite, Tensorflow Mobile)

The overall flow is very similar for most ML frameworks. Each framework has its own compatible model format. We need to take the model created in TensorFlow and convert it into the appropriate format, for each mobile ML framework.

Once the compatible model is prepared, you can run the inference using the ML framework. Note that you must perform pre/postprocessing manually.

If you want more explanation, check this slide(Korean).

• Using built-in model with Core ML

• Using built-in on-device model with ML Kit

• Using custom model for Vision with Core ML and ML Kit

• Object Detection with Core ML and ML Kit
• Using built-in cloud model on ML Kit
  • Landmark recognition
• Using custom model for NLP with Core ML and ML Kit
• Using custom model for Audio with Core ML and ML Kit
  • Audio recognition
  • Speech recognition
  • TTS

Example project using MobileNet model.

1. MobileNet-CoreML
2. MobileNet-MLKit

MobileNet with Core ML	MobileNet with ML Kit

1. PoseEstimation-CoreML
2. PoseEstimation-MLKit
3. dont-be-turtle-ios
4. FingertipEstimation-CoreML

• KeypointAnnotation
  : Annotation tool for own custom estimation dataset

• Measure.swift
• PoseView.swift
• HeatmapView.swift

PoseEstimation-CoreML	PoseEstimation-MLKit

dont-be-turtle-ios

FingertipEstimation-CoreML	KeypointAnnotation

1. WordRecognition-CoreML-MLKit(preparing...)
   : Detect character, find a word what I point and then recognize the word using Core ML and ML Kit.
2. WordRecognition-MLKit(preparing...)
   : Just recognize words by using MLKit's text recognition function.

WordRecognition-CoreML-MLKit	WordRecognition-MLKit
	(DEMO preparing...)

1. SimpleClassification-CreateML-CoreML

Create ML	Core ML

This function is implemented(Measure.swift) in PoseEstimation-CoreML, but need to modulization.

Show output for each input? Drawing detail of result? Test for debugging?

• Pose Estimation: draw dot each point and joint, print confidence each point.
• ...

Analyze outputs from a bunch of inputs

• average of inference time and fps
• accumulate execution time, fps...?
• rendering time
• total execution time
• ...

• tucan9389

• Core ML | Apple Developer Documentation
• Machine Learning - Apple Developer
• WWDC17 - Core ML 발표자료
  • WWDC17 703 Session - Introducing Core ML
  • WWDC17 710 Session - Core ML in depth
  • WWDC17 506 Session - Vision Framework: Building on Core ML
• WWDC18 - Core ML 2 발표자료
  • WWDC18 708 Session - What’s New in Core ML, Part 1
  • WWDC18 709 Session - What’s New in Core ML, Part 2
  • WWDC18 717 Session - Vision with Core ML
• ML Kit - Firebase
• Apple's Core ML 2 vs. Google's ML Kit: What's the difference?
• iOS에서 머신러닝 슬라이드 자료
• MoTLabs Blog