Is there a chance that this lib will work with Camera 0.9 as we go? about tflite_flutter_helper HOT 27 CLOSED

just forked and updated the camera as looks promising no issues with existing code.

Proposed changed to the code base, let him accept and publish the plugin

from tflite_flutter_helper.

+1

from tflite_flutter_helper.

Not just wondering, but kindly asking for it. 0.8.* crashes on several devices, so a dependency upgrade would be fabulous.

from tflite_flutter_helper.

Sorry for late update on this, I will do it by Monday.

from tflite_flutter_helper.

Bump @am15h is this scheduled soon ?

from tflite_flutter_helper.

Thanks, @espbee for testing it. I will try to dig deeper into what exactly could be going wrong here.

from tflite_flutter_helper.

For me, 0.3.1 works with Android. I will check on iOS as well...

from tflite_flutter_helper.

@espbee I have just build iOS version of my app with tflite_flutter 0.9.0 / tflite_flutter_helper 0.3.1 and it works without issues... I am also using runForMultpleInputs. I was upgrading from tflite_flutter_helper 0.3.0.

from tflite_flutter_helper.

@espbee It definitely has to be related to bad input data. Probably you are using some function from the camera API that has changed its behaviour.

For example: It might have started capturing the image in different orientation and the preprocessing you were doing turns the data into something else which is not acceptable by the interpreter. (This thing happend to me once (older version captured a portait image in landscape mode and I had a line of code to rotate it back to portrait, the camera api fixed this in a new version and my code started giving the same error as yours, I removed the extra code to rotate and the error got fixed) Your problem might be different though.

My suggestion on how you should debug it:

1. See if it fails on both android and ios.
2. Make two clones of your project one with older version and another with the latest. Run them parallely.
3. Now at each point of your app right from capturing the frame through the camera API till feeding it to the interpreter, you need to print the data and see what's different with 0.3 as compared to 0.2.
4. As you must be taking the input from camera you can use the emulator/simulator which will virtualize the camera and you will be able to get the same raw input.
5. Now you can compare the image data. Maybe in bytes format.
6. Once you figure out the exact line of code where the data becomes different you can dig into whats going wrong more easily.

I think this whole process would take around 1-2 hour and you will be able make some conclusions.

from tflite_flutter_helper.

@am15h indeed, i get bad state: failed precondition when i wrap the setTo() logic in a try catch. i'll use your advice and proceed. thanks for the help.

from tflite_flutter_helper.

I remember that I ran into simmilar issue when migrating from tflite_flutter 8.0 to 9.0 due to null safety. Unfortunatelly I do not remember the details - I was also getting "bad state: failed precondition". I believe it was somehow connected with TF binary. I will have a look later today when I get home how is it set up now on my mac currently. I use my mac solely for compiling the flutter app there and rarely turn it on otherwise, so I do not have good overview.

from tflite_flutter_helper.

@BernardoGui I'm running the same Flutter as you. The camera frames (same version) seem legit. I think I've isolated my problem though-- I'm going to open a new issue. When I run with 0.3.1 the ImageProcesser doesn't run through the tasks-- I don't get any points or rects when I print stuff in image_process.dart and the image that went in there winds up distorted. I'll write it out in a fresh issue.

EDIT: solved. for me it was a matter of specifying a Float32 type when I made the TensorImage from the image lib image. In the older lib it wasn't necessary. Thanks to all who spent time here.

from tflite_flutter_helper.

just getting caught up on this thread. I can confirm tflite_flutter_helper 0.3.1 works for me on iOS without any issue. it now uses camera "0.9.4+5" I upgraded from 0.2.1 and it all just worked.

Can anyone try v0.3.1, and verify if things work fine? @sircambridge @dimitribocquet @espbee

from tflite_flutter_helper.

@Puspharaj any sense of whether this will happen soon ?

from tflite_flutter_helper.

We need this ASAP.

from tflite_flutter_helper.

thank you so much. looking forward to the camera 0.9 support.

from tflite_flutter_helper.

Can anyone try v0.3.1, and verify if things work fine? @sircambridge @dimitribocquet @espbee

from tflite_flutter_helper.

@am15h -- it is not working for me-- I'm wondering if anyone else has tried this and had more success? @sircambridge @mdejeans @dimitribocquet @Puspharaj

It's not actually executing the predictions. Code the same as it was when it worked (and still works) using tflite_flutter_plugin 8.0 and helper 2.0. In runForMultpleInputs I can getInputTensors() but inputTensors.elementAt(i).setTo(inputs[i]); seems to fail. In the end, classifier.predict(image) is always null.

I'm also getting more errors in didChangeAppLifecycleState .

Part of my issue to obegin with was the same as issue #163 in the tflite_flutter_plugin where the TFLiteC.framwork is not recognized when dropped into ~/.pub-cache/hosted/pub.dartlang.org/tflite_flutter-/ios/ .

I placed the framework into PROJECT/ios/.symlinks/plugins/tflite_flutter/ios which let the app build and run, but then I wind up with no predictions, as described.

Let me know if you could use any more info from me along these lines--

EDIT: the #163 cache folder issue seems sorted out but the lack of predictions / null return remains so far.

from tflite_flutter_helper.

@bazinac indeed, please do if you can-- i didn't get to android, stopped with ios. seemed to be pulling camera frames incorrectly. could certainly be something in my own code, though it worked perfectly till the update-- . curious what you find in ios.

from tflite_flutter_helper.

Upgrade from tflite_flutter 0.9.0 / tflite_flutter_helper 0.2.1 to tflite_flutter 0.9.0 / tflite_flutter_helper 0.3.1 does not work for me (tested only on Android)

from tflite_flutter_helper.

@bazinac thanks for this info, wonder if it's just something silly on my end.

@MoodPatterns if you figure out your problem and want to write back it'd be appreciated. I'm looking around...

from tflite_flutter_helper.

@am15h if you have a moment-- can I ask if there's anything basic that I may have missed from the helper lib 2.0 till now? Something obvious that I would have had to have changed at some point? I'm attempting to solve it but so far am not successful. The specific issue is that running prediction on the frames pulled from the camera (which seem to be legit frames) returns null, rather than predictions or an empty list. The call to setTo() seems to fail silently. If I shift back to 8.0 + 2.0, zero problems.

Anyway thanks again for this library in general.

from tflite_flutter_helper.

@am15h Not solved but I'm going to close this since I think the issue has to do with tflite_flutter 8.0 to 9.0. Finally checked back with prior helper libs and they all break in the same way. So the issue doesn't really belong here I don't think.

In case you see this, and have a sense (by chance) of what this sounds like--

I'm prepping for a custom Yolo model like this:

  imageProcessor ??= ImageProcessorBuilder()
      .add(ResizeWithCropOrPadOp(padSize!, padSize!))
      .add(ResizeOp(INPUT_SIZE.toInt(), INPUT_SIZE.toInt(), ResizeMethod.NEAREST_NEIGHBOUR))
      .add(NormalizeOp(1, 255))
      .build();

  inputImage = imageProcessor!.process(inputImage);

All of this runs, and returns the modified TensorImage inputImage that was made from a camera frame that converted, in turn, to an im.Image . Although the modified, ostensibly prepped TensorImage dimensions are correct, its buffer length is about one quarter the length as it is when running 0.8.0 + 0.2.0 -- around 500K compared to about 2 million. It's tough to pinpoint the trouble more closely. imageProcesser seems to iterate correctly through its list of tasks.

Anyway I'm going to dive into the other lib update and see what I can find. Thanks for the help here, and the update, even if i can't yet enjoy it.

from tflite_flutter_helper.

@bazinac i'm still in there trying to figure it out-- if you can write back later it'll be extremely helpful. thanks.

from tflite_flutter_helper.

from tflite_flutter_helper.

Not working for me. Maybe flutter and camera version is also something we should consider when tracking the issue down (e.g., on flutter 2.5 0.9.0 was not working for me). Anyways, I am on 2.8.1 (stable) with camera 0.9.4+5 and tried to upgrade from 0.9.0/0.2.1, which was working, to 0.9.0/0.3.1.

from tflite_flutter_helper.

It's perfect for me on iOS running 0.9.0 and 0.3.1.

The only hitch was that I had to declare a Tensor Image as Float 32 in the newer lib, wheras 0.2.0 seemed to not care if I did that or not. But once I corrected as much it all just worked without issue.

from tflite_flutter_helper.