ikostrikov / jaxrl Goto Github PK

JAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.

License: MIT License

Shell 0.01% Python 11.39% Jupyter Notebook 88.32% Dockerfile 0.28%

deep-learning deep-reinforcement-learning continuous-control reinforcement-learning soft-actor-critic sac deep-deterministic-policy-gradient jax flax gym

jaxrl's People

Contributors

Stargazers

Watchers

jaxrl's Issues

Remove TF dependency

TF is a heavy library that might need a lot of dependencies and time to install. Removing the dependency on tensorflow_probability is not essential but might be good too.

Assert always evaluates to True

In these lines, because the assert statement is broken into multiple lines and hence covered with parentheses, the assert evaluates the tuple itself. So it will always evaluate to True.

jaxrl/jaxrl/networks/common.py

Lines 80 to 81 in 3674862

 assert (loss_fn is not None or grads is not None, 

 'Either a loss function or grads must be specified.')

jaxrl/jaxrl/networks/common.py

Lines 89 to 90 in 3674862

 assert (has_aux, 

 'When grads are provided, expects no aux outputs.')

This can be fixed by changing it to

assert loss_fn is not None or grads is not None, \
                'Either a loss function or grads must be specified.'

assert has_aux, \
                    'When grads are provided, expects no aux outputs.'

Discount factor in main.py might be misleading

Right now it indicates whether the env is done due to a timelimit, while in the update we use 0.99.

[question] Can you add GAIL to this?

Great work btw

Garden variety installation error

Hey @ikostrikov really cool project!

I was trying to install it but hit some installation difficulties. It should be an easy fix if I look around more, but maybe it's worth introducing a proper dependency manager like poetry. I filed PR #20 to close this issue. Let me know if you'd be interested in it and I can add more docs :)

➜  jaxrl git:(main) ✗ . /home/costa/.cache/pypoetry/virtualenvs/jaxrl-m03KgMHV-py3.9/bin/activate
(jaxrl-m03KgMHV-py3.9) ➜  jaxrl git:(main) ✗ pip install -e .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Installing collected packages: jaxrl
  Running setup.py develop for jaxrl
    ERROR: Command errored out with exit status 1:
     command: /home/costa/.cache/pypoetry/virtualenvs/jaxrl-m03KgMHV-py3.9/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"'; __file__='"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/costa/.cache/pypoetry/virtualenvs/jaxrl-m03KgMHV-py3.9/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"'; __file__='"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Add reward plotting script

A question about temperature module

Hi,

Thanks a lot for the great implementation!

I have a question regarding the temperature, I saw that in
https://github.com/ikostrikov/jax-rl/blob/main/jax_rl/agents/sac/temperature.py#L16

You are computing the temperature (entropy coefficient) by this instead of directly taking an exp on the log-temperature. Is there a particular reason that you implement it this way as compared to other implementation (for example, the pytorch DrQ, soft learning etc.)

Thanks!

ikostrikov / jaxrl Goto Github PK

jaxrl's People

Contributors

Stargazers

Watchers

Forkers

jaxrl's Issues

Remove TF dependency

Assert always evaluates to True

Discount factor in main.py might be misleading

[question] Can you add GAIL to this?

Garden variety installation error

Add reward plotting script

A question about temperature module

Comment on installation of MuJoCo and GPU version of JAX

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	assert (loss_fn is not None or grads is not None,
	'Either a loss function or grads must be specified.')

	assert (has_aux,
	'When grads are provided, expects no aux outputs.')