settings of multi-frame and number of classes (20) about trackformer HOT 3 OPEN

(1): Computing the focal loss for a single class introduces some noise which we found to be less if we increase the number of classes. The number 20 is a bit arbitrary here.
(2): We mention the multi-frame attention in the implementation details. However, since it is not a key element of our contribution and how track queries work we did not provide an in-depth discussion. In particular, the identity preservation (IDF1 and ID switches) benefit a lot from this.

from trackformer.

Hi @timmeinhardt ,

Thanks for your great works!

I found that the number of classes is added by 1 during the definition of classifier.

At the post process, the last appended class is only removed at this line. But the computation of loss does not remove this appended class.

I am curious why you do such settings. Does this have a influence on the tracking performance.

Another question is that the model is optimized with focal loss, which means that sigmoid is used to activate the predicted logits. However, in the post process, softmax is adopted to acitvate the logits and get the prediction scores. Is this a bug or designed by your experimental findings?

from trackformer.

The additional class is added for background prediction in the original DETR formulation. This means also including the class in the loss. However, when running with focal loss, i.e., in the deformable DETR formulation, we do not add the additional class. See this line where we subtract from the number of classes for focal loss

Your second question is also related to the difference between DETR and Deformable DETR. When running the latter with focal loss we do not apply a softmax in the post processing. See this module:

from trackformer.