Hello, thanks to your great leaderboard! It really helps to know the latest work in FSL! However I found one unreliable from the leaderboard:

LGM-Net | ICML | 2019 | 6CONV | Inductive

I think this result is unreliable because it has a problematic label leak issue from their codes (likesiwell/LGM-Net#5) and the authors of this paper haven't make any response on this issue. Now that this result ranks No. 1 in the inductive settings. I think it may be misleading to people who are not familiar with the paper and not very fair to other researchers. Before the authors of that paper make any clarifications, could you consider remove this result from the leaderboard? I think it would be great to the FSL community. Thanks!

Thank you for your project.
I'd like to know how do you distinguish a few-shot learning method from an inductive one or a transductive one? Is there a clear distinction between these two settings in few-shot learning?

What are the details of your experiment with Cub dataset training on the DEEPEMD-FCN network? In the 5way5shot setup, I ran with only 82% accuracy.

Hi there,
I am the first author of ICI. Very thanks to add our results in your list. The reported accuracy of SSFSL in your repo uses 80 unlabeled data for each class, which is more than the standard setting and resulting in better performance. This may be unfair for other methods. I recommend to use the results of 30/50 unlabeled data, which is most widely setting used in other SSFSL methods. Please see our paper for more details.
Besides, now we have an improved version of ICI with better performance. You may add it in your list. Please note that it is now only a arxiv preprint.

Hey
First of all - great work, really. Your direct competitor is paperswithcode, which is a Facebook company, so you sure did something great!

As for my question - I have seen some differences in your leader board and the provided papers (and my own trials...), and would like to know what is the source of the accuracy reported here.

It should be noticed that while paperswithcode tries to get all of their information from the paper itself, there are many errors in their leader board(s), also they cite non peer-reviewed papers, which makes it more difficult to accept.

There is a problem in the accuracy of Inductive setting Simple CNAPS model on miniImageNet data.Its precision is way over the top.For details, see https://github.com/plai-group/simple-cnaps.

I think that all of us could benefit from a "my own implementation performance" column
i.e. if one tries it out, with any implementation, it is crucial information that can save a lot of time and effort.

I'd limit the addition of entries for this column (section) to an open source code provided with training details (including data) that can be trained to achieve similar performance given the same data
But even without the open source, I think that this feature is nice to have