This challenge is that using machine learning model created from tensorflow on iOS with Core ML or ML Kit(TensorFlow Lite)
‼️ PR for English advice always makes me happy‼️
- Core ML
- MLKit
- etc. (Tensorflow Lite, Tensorflow Mobile)
Almost machine learning framework have been used similar flow. Convert my model created from TensorFlow to a model which is compatible with mobile machine learning framework. Each framework have compatible model format, and
Once prepared a compatible model, you can run inference by using machine learning framework(like Core ML or ML Kit..). You need preprocess/postprocess before/after inference on your project.
If you want more explanation, check this slide(Korean).
-
using built-in model with Core ML
-
using built-in on-device model with ML Kit
-
using custom model for Vision with Core ML and ML Kit
-
using built-in cloud model on ML Kit
- landmark recognition
-
using custom model for NLP with Core ML and ML Kit
-
using custom model for Audio with Core ML and ML Kit
- audio recognition
- speech recognition
- TTS
Example project using MobileNet model.
MobileNet with Core ML | MobileNet with ML Kit |
---|---|
![]() |
![]() |
- KeypointAnnotation
: Annotation tool for own custom estimation dataset
Measure.swift
PoseView.swift
HeatmapView.swift
PoseEstimation-CoreML | dont-be-turtle-ios |
---|---|
![]() |
![]() |
FingertipEstimation-CoreML | KeypointAnnotation |
---|---|
![]() |
![]() |
- WordRecognition-CoreML-MLKit(preparing...)
: Detect character, find a word what I point and then recognize the word using Core ML and ML Kit. - WordRecognition-MLKit(preparing...)
: Just recognize words by using MLKit's text recognition function.
WordRecognition-CoreML-MLKit | WordRecognition-MLKit |
---|---|
![]() |
- |
Create ML | Core ML |
---|---|
![]() |
This function is implemented(
Measure.swift
) in PoseEstimation-CoreML, but need to modulization.
Show output for each input? Drawing detail of result? Test for debugging?
- Pose Estimation: draw dot each point and joint, print confidence each point.
- ...
Analyze outputs from a bunch of inputs
- average of inference time and fps
- accumulate execution time, fps...?
- rendering time
- total execution time
- ...
- Core ML | Apple Developer Documentation
- Machine Learning - Apple Developer
- WWDC17 - Core ML 발표자료
- WWDC18 - Core ML 2 발표자료
- ML Kit - Firebase
- Apple's Core ML 2 vs. Google's ML Kit: What's the difference?
- iOS에서 머신러닝 슬라이드 자료
- MoTLabs Blog