cranial-xix / cagrad Goto Github PK

In your SGD method cagrad you divide by scale, but do not multiply by scale at

I notice in your other two methods that you rescale before returning the gradient. Is this an error in the SGD method?

Can you provide MNIST experiment, which relate to model architecture? Cause, I used LeNet architecture but can't obtain performance as your result in Figure 4 in your paper.

The code given here say that $| g_0|$ is the mean of the gramian matrix. Would it be possible to know what code was run for the plots of the paper ?

The link of the NYU-V2 cannot be found?

Hi, Thank you for sharing this resource and congratulations on the great work!
I have some issue regarding the logs generated by the MT10/MT50 benchmarks for multi-task RL. The paper suggest that the agent is evaluated every 10,000 training steps, but the log files only show evaluation results every 30,000 training steps, after implementing the MT10 benchmark following the given instructions for the setup. Is this expected? In that case is there a way to retrieve evaluation results every 10,000 steps, as described in the paper? If not, have I missed something in the setup or is there any configuration I need to change in order to rectify this issue?
Thanks in advance for the consideration!

Hi, congratulation on your great work.
Currently, I'm trying to reproduce the result for the semi supervised learning setting. So that, could you please share the code to this public repo. Thanks

In case the solution gives $|g_w|=0$, then we get division by $0$. In this case, we get that the lagrangian $\lambda$ is $0$, therefore the solution is the same as the unconstrained one. Going back to the original problem we are therefore solving $\max_{d\in\mathbb R^n} \min_{i\in [n]}\langle g_i, d\rangle$. Now it is not too hard to show that whenever there is $0< w\in \mathbb{R}^m$ such that $A^T w=0$ ( $A$ is the jacobian matrix), then for any $d\neq 0$, then there is $i$ such that $\langle g_i, d\rangle < 0$. This means that the $\min$ is $-\infty$ unless $d=0$, therefore the outer $\max$ is reached for $d=0$.