A few questions regarding the paper about maml HOT 4 CLOSED

1. No, if you go through the math, only 2nd-order terms arise, even when using multiple gradient steps. This is because the gradient steps after the first step are not w.r.t. theta.
2. In the few-shot supervised setting, a task is a set of N classes. Hence, if a dataset has 1200 classes, then there are 1200 choose N tasks total.
3. In general, there is no known measure of similarity. The test tasks and the training tasks should be sampled from the same distribution.
4. See the stop_grad flag in this repo. We stop the gradient through the gradient term, cutting the dependency of grad_theta on theta.
5. Roughly, yes. The directions come from the gradient, so the algorithm is trying to find an initialization such that the gradient points in correct directions. In practice, there is not just one optimum, but a whole space of optimum, particularly with large & deep, overparameterized neural networks. In this way, Figure 1 can be misleading; the optimization is more flexible than it looks.

from maml.

Hi Chelsea,

Thank you very much for your detailed response. I think I understood your explanations but I have one more question regarding the supervised classification setting. Suppose you want to do N-way K-shot classification. In your experiments, is it the case that the batch size of the inner gradient is N x K? (I ask this question because when you call the main.py you define batch size = 1 but later you define the tensor as batch size x number of classes and from what you told me that you define as a task in the classification setting, I think N x K is the correct answer.)
Moreover, during test time, do you still use the same batch size, i.e. N x K? Last, during test time, you simply use gradient descent, right?

Thank you in advance.

Aris Papadopoulos

from maml.

Yes, that is correct

from maml.

Both the questions and answers are very good, which help me understand the paper better. Many thanks!

from maml.