The case `g_w ~= 0` is ill-specified. about cagrad HOT 3 OPEN

Sorry that I do not see the problem here. Note that when $g_w = 0$ it has to be that $g_0 = 0$, as $g_w^\top g_0 \geq (1 - c) ||g_0||^2 \geq 0$. So it just means that CAGrad finds a stationary point of $L_0$, which is also a Pareto optimal point.

from cagrad.

This holds only when $c\leq 1$, I guess you don't consider so much the case $c>1$, could you explain a bit why ?

If $c>1$, it is possible that $g_w=0$ and $g_0\neq 0$ in that case when dividing by $| g_w|$ the computation of CAGrad in your code will blow up, This is because $\lambda=0$ and setting $d=g_0+g_w/\lambda$ is a problem. Actually in the original paper, I think we are supposed to split the cases when this happens, luckily if we know that $\lambda=0$, then the problem has the same solution as the unconstrained one, therefore from what I said above, you can just pick $d=0$.

from cagrad.

Thanks for asking, I now understand what you mean. Yes, $\lambda = 0$ should be a separate case as the optimal $d$ form is different. Thanks for catching it.

from cagrad.