Bbox Pruning Too Aggressive? about dali HOT 5 CLOSED

@5had3z Thank you for reporting this issue. To make sure I understand the problem, could you also provide some toy example data and code snippet so that I can reproduce and debug on my end? You can also play with pipeline seed to provide reproducible results.
Thank you in advance.

from dali.

More bounding boxes are removed than expected around the edges of an image with a random crop.

Can you also explain what you mean by this? What is the "expected" case? This operator removes any bounding box with a centroid outside of the cropping region. That is, if the center of the box is outside of the random crop, it's filtered out. Is this what you mean?

From the documentation:

The third output contains the bounding boxes, after filtering out the ones with a centroid outside
of the cropping window, and with the coordinates mapped to the new coordinate space.

from dali.

From the documentation:

The third output contains the bounding boxes, after filtering out the ones with a centroid outside
of the cropping window, and with the coordinates mapped to the new coordinate space.

I should've rtfm'd a bit harder, I mainly focused on using the anchor, shape and output_bbox_indices output correctly. Since this is currently expected behaviour, I think a new parameter should be added to prune boxes based on area of bbox within the ROI, which I would argue would a prefered metric.

With the current behaviour if my centroid is in the ROI by one pixel in both axis, I have ~25% of the original box and this is kept. But if my bbox centroid is outside of the ROI by 1 pixel in 1 axis, I have just lost a bbox that is still ~50% within the image. Using a %area still within the ROI would keep more boxes that are reasonably within the ROI and would enable the option to make it 0.0 for no pruning.

I'll copy-pasta the existing code and add this new behaviour to a custom plugin for myself for now. If there is interest I can then make the changes on the main fn.random_bbox_crop.

from dali.

Comparison with my new cropping algorithm. This is shown with threshold=0.0 so if any sliver of the box is within roi it is kept, but this can be tuned. Not much needed to be changed, new arg and FilterByCentroid replaced with FilterBboxes that takes a lambda that returns a "should_keep" bool based on crop and box.

-Example 1
Centroid

Threshold

-Example 2
Centroid

Threshold

-Example 3
Centroid

Threshold

from dali.

That's awesome. We will gladly review and incorporate your extension to the operator:

Not much needed to be changed, new arg and FilterByCentroid replaced with FilterBboxes that takes a lambda that returns a "should_keep" bool based on crop and box.

How do you suggest we switch between the current filter by centroid and your new criteria?
filter_by="centroid" vs filter_by="area" argument?

Feel free to open a PR and with your suggested changes and we can continue the discussion in the review.

Thanks!

from dali.