Changing Attempts Timing/Delay about double-take HOT 9 CLOSED

I installed this in the morning and have been testing it in the morning. I set it to 100ms delay + increasing the attempts to 40 tries (I know probably taxing my system). I see that my detection is around 4-5 frames per second for 9-10 seconds which pretty much covers all/most frames in the event.

Thanks for the change.

from double-take.

Hey there, thanks for checking out the project and the detailed issue. Let me try to explain some of it as well and make sure we are on the same page. The processing of the Frigate images / MQTT image is a little confusing and the MQTT one is done separately then the event driven one.

Frigate publishes a message to frigate/events:

Double Take then begins to process this event using images from Frigate's API. These images are /api/${camera}/latest.jpg and /api/events/${id}/snapshot.jpg. The snapshot image in the double-take:latest is set to 0 currently due to a bug on the Frigate API side. If you're running Frigate 0.9.0 then you should be fine to increase this. The big difference with the snapshot image here is that it's based off the event ID. This is different than the MQTT image described below.

These two images are processed in parallel with the configured detectors. You're right there is no delay between processing events, but the time to process an image should add delay between each loop. The default amount of retries is 10 per image. Each loop should pull in the current frame from those two images and then pass it for detection.

Are you getting any errors in the 20 attempts in 1-2 seconds? The detector you're using is responding fast enough to process 20 images? I see that the slower model helps with the delays, so your machine must process them quicker than mine which is why I haven't seen the quicker API requests. I can add in a delay to help with this though.

Frigate publishes a message to frigate/+/person/snapshot:

Frigate publishes a JPEG encoded image to this topic when certain conditions are met on the Frigate side. There is no context around this image other than the camera name and JPEG frame. Double Take uses this image to process it through a single recognition loop. It's only processed one time unless Frigate publishes a new image to that topic.

You're right, if you try to use this image for presence detection then you could have reports you are in a room after you left it because Frigate published the best snapshot when the event was done.

I run into this issue right now as well with my setup, but I'm not sure the best way to overcome it. I orginally didn't process the frigate/+/person/snapshot because the snapshot.jpg image was better since it's tied to the ongoing Frigate event. I only added support for the MQTT image when I discovered Frigate had a bug with the snapshot.jpg image, but now that it's fixed in 0.9.0, maybe I just make the processing of the MQTT image a config setting on the Double Take side.

Sorry for the long response, let me know if this makes sense.

from double-take.

Your response is spot on.

Processing Attempts
Nope no errors. Average detection for the Compreface with 1080p images is 100-150ms using the MobileNet model.

Having a tunable delay would probably be helpful for most people and the easiest to implement. I might just compile your code with my delay hardcoded to test it out. Basically I need to put a delay that will take into account the average processing time of my detector (in my case 100-150ms) + the time a new frame might be recorded (5fps would be every 200ms).

A better way probably to calculate the delay depending on the fps of the source. So say double-take finds that a detection is 100ms and it knows that a frame will only come every 200ms, it would delay by 100ms dynamically.

The best way to solve this is to put a face detection model co-compiled with the object detection model that frigate uses, to have a new object detection event (Face detection vs Person detection). I might do this with a custom model.

MQTT Snapshots
I ended up just turning off MQTT snapshots from the Frigate side. So having a way to turn this off in Double-take would be helpful.

from double-take.

I pushed a couple updates to the beta build to help address these issues. Let me know what you think.

frigate:
  url: https://frigate-url.com
  attempts:
    mqtt: false # false: do not process mqtt snapshot topic images | true by default
    delay: .25 # add a delay expressed in seconds between each detection loop

from double-take.

What about making a Feature-Request to Frigate to send .jpg-image with every /MQTT/event/ ?

(Also for some other reasons:
This way one could benefit from the event-context (also for other purposes) and get images over single channel without pulling them.)

from double-take.

Frigate already sends a "best" snapshot with the MQTT/event/.

The problem is Frigate triggers an event based on a "Person" object detection because their models only do object detection. It would be better if they implement a face detection model to have a Face event that could be sent to Double-Take.

from double-take.

Frigate already sends a "best" snapshot with the MQTT/event/.

but it seems the .jpg is missing in the mqtt/event payload...
blakeblackshear/frigate#1560

fully agree:

The problem is Frigate triggers an event based on a "Person" object detection because their models only do object detection. It would be better if they implement a face detection model to have a Face event that could be sent to Double-Take.

or a way to feed custom models from motion-trigger?

from double-take.

New API event/endpoint: clips!
blakeblackshear/frigate#1502 (reply in thread)

There is a new endpoint not yet in the http docs: /api/events/<event_id>/clip.mp4. This will use ffmpeg to dynamically create an mp4 in the tmp directory from the recording segments and then tell nginx to return that file as the result. It is shockingly fas

from double-take.

Closing since this is now included in the 1.0.0 release.

from double-take.