Comments (4)
This could happen for many reasons. The most common one is that your inputs are not normalized correctly. Change the input formatting to the following code in the 'isolate_inference.dart'
final imageMatrix = List.generate(
imageInput.height,
(y) => List.generate(
imageInput.width,
(x) {
final pixel = imageInput.getPixel(x, y);
return [(pixel.r-127.5)/127.5, (pixel.g-127.5)/127.5, (pixel.b-127.5)/127.5];
},
),
);
If not working, change the divisor (127.5) to 255 and try. You will have to figure out the most appropriate normalization method for your model.
from flutter-tflite.
Your model's output is float32 and not an integer as in the example model. You need to change the output type to double to handle that. And the maxSocre needs to be changed to double as well. I have edited those changes for you. Replace the content of the 'isolate_inference.dart' to following and it should work as intended.
/*
* Copyright 2023 The TensorFlow Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import 'dart:io';
import 'dart:isolate';
import 'package:camera/camera.dart';
import 'package:image/image.dart' as image_lib;
import 'package:image_classification_mobilenet/image_utils.dart';
import 'package:tflite_flutter/tflite_flutter.dart';
class IsolateInference {
static const String _debugName = "TFLITE_INFERENCE";
final ReceivePort _receivePort = ReceivePort();
late Isolate _isolate;
late SendPort _sendPort;
SendPort get sendPort => _sendPort;
Future<void> start() async {
_isolate = await Isolate.spawn<SendPort>(entryPoint, _receivePort.sendPort,
debugName: _debugName);
_sendPort = await _receivePort.first;
}
Future<void> close() async {
_isolate.kill();
_receivePort.close();
}
static void entryPoint(SendPort sendPort) async {
final port = ReceivePort();
sendPort.send(port.sendPort);
await for (final InferenceModel isolateModel in port) {
image_lib.Image? img;
if (isolateModel.isCameraFrame()) {
img = ImageUtils.convertCameraImage(isolateModel.cameraImage!);
} else {
img = isolateModel.image;
}
// resize original image to match model shape.
image_lib.Image imageInput = image_lib.copyResize(
img!,
width: isolateModel.inputShape[1],
height: isolateModel.inputShape[2],
);
if (Platform.isAndroid && isolateModel.isCameraFrame()) {
imageInput = image_lib.copyRotate(imageInput, angle: 90);
}
final imageMatrix = List.generate(
imageInput.height,
(y) => List.generate(
imageInput.width,
(x) {
final pixel = imageInput.getPixel(x, y);
return [pixel.r, pixel.g, pixel.b];
},
),
);
// Set tensor input [1, 224, 224, 3]
final input = [imageMatrix];
// Set tensor output [1, 1001]
final output = [List<double>.filled(isolateModel.outputShape[1], 0)];
// // Run inference
Interpreter interpreter =
Interpreter.fromAddress(isolateModel.interpreterAddress);
interpreter.run(input, output);
// Get first output tensor
final result = output.first;
double maxScore = result.reduce((a, b) => a + b);
// Set classification map {label: points}
var classification = <String, double>{};
for (var i = 0; i < result.length; i++) {
if (result[i] != 0) {
// Set label: points
classification[isolateModel.labels[i]] =
result[i].toDouble() / maxScore.toDouble();
}
}
isolateModel.responsePort.send(classification);
}
}
}
class InferenceModel {
CameraImage? cameraImage;
image_lib.Image? image;
int interpreterAddress;
List<String> labels;
List<int> inputShape;
List<int> outputShape;
late SendPort responsePort;
InferenceModel(this.cameraImage, this.image, this.interpreterAddress,
this.labels, this.inputShape, this.outputShape);
// check if it is camera frame or still image
bool isCameraFrame() {
return cameraImage != null;
}
}
from flutter-tflite.
@chathurach it works but now the result is highly inaccurate. I've tested my model and I'm quite sure that my model accuracy is not supposed to be that low. In the screenshot below, the result should be 'V'.
This is the image test I used.
from flutter-tflite.
It works! I learn something new, thank you.
from flutter-tflite.
Related Issues (20)
- Cannot get yamnet features in flutter
- E/flutter (30995): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Output object shape mismatch, interpreter returned output of shape: [1, 84, 8400] while shape of output provided as argument in run is: [8400, 4] HOT 2
- The inference speed using the old version is 5-8 times faster than the latest version HOT 7
- null check operator used on a null value HOT 1
- IOS release version does't work HOT 2
- Using GPU for tflite
- Model far less accurate when testing with example code
- Why flutter-tflite doesn't support flutter web HOT 3
- Undefined name 'TfLiteGpuInferenceUsage' and 'TfLiteGpuInferencePrioFrity'. HOT 1
- support for the CUDA platform of NVIDIA graphics cards HOT 2
- Is it possible to run more then one TFLite models at the same time on a Flutter App HOT 1
- NNAPI delegate HOT 1
- Model missing in live object detection
- How can I load a tflite model using Isolate? HOT 1
- Super slow inference speed for a certain model
- Build issue with old version
- Need help for super-resolution for onnx like tflite implemention (Not issue)
- Failed to lookup symbol 'TfLiteInterpreterOptionsCreate': dlsym(RTLD_DEFAULT, TfLiteInterpreterOptionsCreate): symbol not found
- Output object shape mismatch, interpreter returned output of shape: [1, 1]
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flutter-tflite.