TRPO implementation about rllab HOT 2 CLOSED

The implementation of natural policy gradient algorithm here (and the description in the paper) differs from the traditional natural policy gradient, in that the step size is calculated differently by doing a second-order approximation to the KL divergence constraint. It also makes use of the conjugate gradient algorithm. Hence we named our implementation "truncated" natural policy gradient (TNPG).

Then, TNPG and TRPO deviates only in that TRPO performs a line search to make sure that there is sufficient decrease in the surrogate loss function. TNPG uses almost the same parameters, except that max_backtracks=1, which essentially does not perform backtracking (and thus no line search).

from rllab.

Thanks! That will teach me to read the paper before the code. :)

from rllab.