Giter VIP home page Giter VIP logo

Comments (10)

gregorscholz avatar gregorscholz commented on May 21, 2024 1

I will hopefully push soon an example for object detection where the model takes one input tensor and has 4 output tensors.
Hopefully i understand you correctly and this helps you.

// Set input tensor [1, 300, 300, 3]
final input = [imageMatrix];

// Set output tensor
// Locations: [1, 10, 4]
// Classes: [1, 10],
// Scores: [1, 10],
// Number of detections: [1]
final output = {
  0: [List<List<num>>.filled(10, List<num>.filled(4, 0))],
  1: [List<num>.filled(10, 0)],
  2: [List<num>.filled(10, 0)],
  3: [0.0],
};

 _interpreter.runForMultipleInputs([input], output);

from flutter-tflite.

LachhabMeriem avatar LachhabMeriem commented on May 21, 2024 1

E/flutter (30995): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Output object shape mismatch, interpreter returned output of shape: [1, 84, 8400] while shape of output provided as argument in run is: [8400, 4]

from flutter-tflite.

alexw92 avatar alexw92 commented on May 21, 2024

Yes thanks this is what I ended up doing. At first I didnt realize that runForMultipleInputs is directly callable from outside too and was using run only. By directly using runForMultipleInputs this works perfectly for OD. Just the naming could be improved. Thank you!

from flutter-tflite.

gregorscholz avatar gregorscholz commented on May 21, 2024

Same for me. Thats why i opened this ticket here.

from flutter-tflite.

Rikerz08 avatar Rikerz08 commented on May 21, 2024

Hi @gregorscholz and @alexw92! I'm fairly new to ML and flutter development, do you know how to solve this error? I have a MobileNetSSD 320x320 model.

Exception has occurred.
ArgumentError (Invalid argument(s): Output object shape mismatch, interpreter returned output of shape: [1, 10] while shape of output provided as argument in run is: [1, 10, 4])

from flutter-tflite.

alexw92 avatar alexw92 commented on May 21, 2024

@Rikerz08 Hello there, you need to specify outputs which fit the dimensions of the outputs ur model uses. You can find out the output dims of your model by using Netron or something similiar

from flutter-tflite.

Rikerz08 avatar Rikerz08 commented on May 21, 2024

@alexw92 Thanks for this! However, I'm confused as to why I still need to determine my model's output shape. Isn't it [1,10] as what is stated in the error? Please correct me if I'm wrong.

Additionally, how do I specify and change the expected output size in order to fit my model's shape in this code?

import 'dart:developer';
import 'dart:io';
import 'package:flutter/services.dart';
import 'package:image/image.dart' as img;
import 'package:tflite_flutter/tflite_flutter.dart';

class ObjectDetection {
  static const String _modelPath = 'assets/models/Hemalens.tflite';
  static const String _labelPath = 'assets/models/labelmap.txt';

  Interpreter? _interpreter;
  List<String>? _labels;

  ObjectDetection() {
    _loadModel();
    _loadLabels();
    log('Done.');
  }

  Future<void> _loadModel() async {
    log('Loading interpreter options...');
    final interpreterOptions = InterpreterOptions();

    // Use XNNPACK Delegate
    if (Platform.isAndroid) {
      interpreterOptions.addDelegate(XNNPackDelegate());
    }

    // Use Metal Delegate
    if (Platform.isIOS) {
      interpreterOptions.addDelegate(GpuDelegate());
    }

    log('Loading interpreter...');
    _interpreter =
        await Interpreter.fromAsset(_modelPath, options: interpreterOptions);
  }

  Future<void> _loadLabels() async {
    log('Loading labels...');
    final labelsRaw = await rootBundle.loadString(_labelPath);
    _labels = labelsRaw.split('\n');
  }

  Uint8List analyseImage(String imagePath) {
    log('Analysing image...');
    // Reading image bytes from file
    final imageData = File(imagePath).readAsBytesSync();

    // Decoding image
    final image = img.decodeImage(imageData);

    // Resizing image fpr model, [300, 300]
    final imageInput = img.copyResize(
      image!,
      width: 300,
      height: 300,
    );

    // Creating matrix representation, [300, 300, 3]
    final imageMatrix = List.generate(
      imageInput.height,
      (y) => List.generate(
        imageInput.width,
        (x) {
          final pixel = imageInput.getPixel(x, y);
          return [pixel.r, pixel.g, pixel.b];
        },
      ),
    );

    final output = _runInference(imageMatrix);

    log('Processing outputs...');
    // Location
    final locationsRaw = output.first.first as List<List<double>>;
    final locations = locationsRaw.map((list) {
      return list.map((value) => (value * 300).toInt()).toList();
    }).toList();
    log('Locations: $locations');

    // Classes
    final classesRaw = output.elementAt(1).first as List<double>;
    final classes = classesRaw.map((value) => value.toInt()).toList();
    log('Classes: $classes');

    // Scores
    final scores = output.elementAt(2).first as List<double>;
    log('Scores: $scores');

    // Number of detections
    final numberOfDetectionsRaw = output.last.first as double;
    final numberOfDetections = numberOfDetectionsRaw.toInt();
    log('Number of detections: $numberOfDetections');

    log('Classifying detected objects...');
    final List<String> classication = [];
    for (var i = 0; i < numberOfDetections; i++) {
      classication.add(_labels![classes[i]]);
    }

    log('Outlining objects...');
    for (var i = 0; i < numberOfDetections; i++) {
      if (scores[i] > 0.6) {
        // Rectangle drawing
        img.drawRect(
          imageInput,
          x1: locations[i][1],
          y1: locations[i][0],
          x2: locations[i][3],
          y2: locations[i][2],
          color: img.ColorRgb8(255, 0, 0),
          thickness: 3,
        );

        // Label drawing
        img.drawString(
          imageInput,
          '${classication[i]} ${scores[i]}',
          font: img.arial14,
          x: locations[i][1] + 1,
          y: locations[i][0] + 1,
          color: img.ColorRgb8(255, 0, 0),
        );
      }
    }

    log('Done.');

    return img.encodeJpg(imageInput);
  }

  List<List<Object>> _runInference(
    List<List<List<num>>> imageMatrix,
  ) {
    log('Running inference...');

    // Set input tensor [1, 300, 300, 3]
    final input = [imageMatrix];

    // Set output tensor
    // Locations: [1, 10, 4]
    // Classes: [1, 10],
    // Scores: [1, 10],
    // Number of detections: [1]
    final output = {
      0: [List<List<num>>.filled(10, List<num>.filled(4, 0))],
      1: [List<num>.filled(10, 0)],
      2: [List<num>.filled(10, 0)],
      3: [0.0],
    };

    _interpreter!.runForMultipleInputs([input], output);
    return output.values.toList();
  }
}

from flutter-tflite.

alexw92 avatar alexw92 commented on May 21, 2024

@Rikerz08 The error you see is probably a mismatch of one of the 4 outputs assuming your model also has 4 outputs. Seeing from the code and the error I would guess that the order of the outputs is different in your model. What happens if you change this

final output = {
      0: [List<List<num>>.filled(10, List<num>.filled(4, 0))],
      1: [List<num>.filled(10, 0)],
      2: [List<num>.filled(10, 0)],
      3: [0.0],
    };

to this

final output = {
      0: [List<num>.filled(10, 0)],
      1: [List<List<num>>.filled(10, List<num>.filled(4, 0))],
      2: [List<num>.filled(10, 0)],
      3: [0.0],
    };

for instance?

from flutter-tflite.

Rikerz08 avatar Rikerz08 commented on May 21, 2024

@alexw92

We tried changing it to this:

final output = { 0: [List<num>.filled(10, 0)], 1: [List<List<num>>.filled(10, List<num>.filled(4, 0))], 2: [0.0], 3: [List<num>.filled(10, 0)], };

The shape error was fixed however it had another error which is this one:

_TypeError (type 'List<double>' is not a subtype of type 'List<List<double>>' in type cast)

from flutter-tflite.

alexw92 avatar alexw92 commented on May 21, 2024

@Rikerz08 ah so thats some progress. Just try out all the permutations and in the end you will have the right order. Or add a print statement in the runForMultipleInputs code to check the dimensions of each output manually. Or you use Netron to look up the outputs of your model.

from flutter-tflite.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.