Giter VIP home page Giter VIP logo

Comments (1)

bsmock avatar bsmock commented on July 18, 2024

Great observation, I see the discrepancy you are referring to.

The key question is: what do we do when "no-object" is the most probable class prediction? Do we suppress/filter out such outputs, or do we treat them as predictions for the second most probable class, which is the most probable class that isn't "no-object"?

I'll start out by saying that personally it doesn't seem to me that there is an obvious answer to which convention is the "right" one. So the question of which convention to choose might end up being an empirical/practical one.

The COCO evaluation convention appears to follow the second convention: keep all predictions and assign each prediction the most probable object class label that was produced. Note that this will always be less than or equal to 0.5 if "no-object" is actually the most probable class predicted. We use the code created by the original DETR authors to follow this convention for computing COCO metrics.

In our inference code, we follow the other convention: always assign the most probable class to the prediction, even if it is "no-object". As a result, we can filter out predictions that are labeled "no-object", rather than treat them as a low-probability object class prediction. This is intentional, but we did not empirically study the difference it has in practice, if any, with the other convention.

Do you have a preference or an argument for always favoring one convention over the other?

Best,
Brandon

from table-transformer.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.