by571 / qr-dqn Goto Github PK

There is something wrong with the loss function. The variable self.quantile used in loss is a one-dimensional tensor, which would be subtracted by a 3-dimensional tensor. So the self.quantile would be broadcasted automatically. But the automatical broadcasting way is wrong, which would cause a bug that Q function violates monotonicity in its output. Someone who what to use the code have to modify this part manully.