uber-research / go-explore Goto Github PK

Hi,

i was trying to expand go-explore for other games (gym and retro), but ran into the problem of facepixels. As far as I understand you use it to identify player coordinates.

I tried to get the pixel value for super-mario-bros from a gray scale picture, but you commented that the pixel value must be unique (not used elsewhere in the scene), which is not possible for mario and most other games (gym, atari, retro).

Did I understood this wrong and is there any way to make other games work with face pixels (without using x and y coordinates, which are often not avaiable, also would require a rewrite)? How to determine which value to use?

Hi,
I get the below error when I try to run the code in my 4GB Ram 2 Core CPU mac. Is there a parameter I should change so I can run the code?

canermac-3:go-explore apple$ sh ./phase1_pitfall_domain.sh
Code hash: 9ee1ab8d43c267b12e33c9c08aaf7b54dd01609de0371fe29ee6d6b52e9b6a1b
Experiment running in ./results//0011_0797eea89ba54609b9157d9f034451a5/
Time (seconds):   0%|                                                                                                           | 0/1800000.0 [00:00<?, ?it/s]
loky.process_executor._RemoteTraceback: 
'''pute steps:   0%|                                                                                                           | 0/1000000000 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/Users/apple/miniconda3/envs/projectlife/lib/python3.6/site-packages/loky/process_executor.py", line 391, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
  File "/Users/apple/miniconda3/envs/projectlife/lib/python3.6/multiprocessing/queues.py", line 113, in get
    return _ForkingPickler.loads(res)
  File "/Users/apple/.Trash/go-explore/goexplore_py/pitfall_env.py", line 82, in __getattr__
    return getattr(self.env, e)
  File "/Users/apple/.Trash/go-explore/goexplore_py/pitfall_env.py", line 82, in __getattr__
    return getattr(self.env, e)
  File "/Users/apple/.Trash/go-explore/goexplore_py/pitfall_env.py", line 82, in __getattr__
    return getattr(self.env, e)
  [Previous line repeated 993 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "goexplore_py/main.py", line 409, in <module>
    reset_cell_on_update=args.reset_cell_on_update)
  File "goexplore_py/main.py", line 288, in run
    _run(base_path=base_path, **kwargs)
  File "goexplore_py/main.py", line 154, in _run
    expl.run_cycle()
  File "/Users/apple/.Trash/go-explore/goexplore_py/goexplore.py", line 298, in run_cycle
    trajectories = [e.data for e in POOL.map(self.process_cell, chosen_cells)]
  File "/Users/apple/.Trash/go-explore/goexplore_py/goexplore.py", line 298, in <listcomp>
    trajectories = [e.data for e in POOL.map(self.process_cell, chosen_cells)]
  File "/Users/apple/miniconda3/envs/projectlife/lib/python3.6/site-packages/loky/process_executor.py", line 794, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/Users/apple/miniconda3/envs/projectlife/lib/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/Users/apple/miniconda3/envs/projectlife/lib/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/Users/apple/miniconda3/envs/projectlife/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
(projectlife) canermac-3:go-explore apple$ 

Installed all the requirements, getting this error:

The MPI_Comm_test_inter() function was called before MPI_INIT was invoked.

The current license for this code, which prohibits commercial use, is unfamiliar. Is it based on any well-known licenses?

Uber's recent practices seem to emphasize publishing via the Apache 2.0 license (What Every Engineer Should Know About Open Source Software Licenses and IP | Uber Engineering Blog).
Using it here would likely result in this code gaining much more engagement with the community.

Why aren't you using Apache 2.0 here? Can you switch to it?

Is Uber making any patent claims associated with this code?

Installed everything in requirements.txt according to the specification. I have an anaconda environment open on Python v3.6. Getting the following error:

tensorflow.python.framework.errors_impl.NotFoundError: librca.so.0: cannot open shared object file: No such file or directory

Hi!
I implemented the go-explore idea on a toy problem and it seems to work very well.
I modified the algorithm with a counter and a threshold.
Is this implementation inline with your thoughts?

https://github.com/christofer-f/Hierarchical-Actor-Critic-HAC-PyTorch

//Christofer

Dear Ecoffet,
As an MSc student, I am currently working on implementing the explore method in the MDPO algorithm, as described in your paper titled "Mirror Descent Policy Optimization" (https://arxiv.org/pdf/2005.09814.pdf). I have been trying to locate the documentation file for this method, but unfortunately, I have been unable to find it.
I would greatly appreciate it if you could provide me with any instructions or guidance on how to implement the explore method in the MDPO algorithm.
Thank you in advance for your assistance. I am eager to learn and apply this method to further enhance the MDPO algorithm.
Gratefully,
Haneen

I am running the code from https://github.com/uber-research/atari-reset, but since you have not enabled issues there I write here instead.

When I try to run your robustification code with the default parameters from https://github.com/uber-research/go-explore i get the following error:

*** The MPI_Comm_test_inter() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[hampusa:2940] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

I am trying to run it on a single machine, and I have tried if setting --nenvs=1 would help but there is no difference.

As title, not sure what mask means in the GRUPolicy. Any hint about the what mask looks like is appreciated. Thanks!

Hi, I am trying to run Phase 2 (e.g., phase2_atari_test.sh) but I realized that there is no 'atari_reset' directory in the folder 'robustified'.
Am I missing something or is the repository missing some folders?
FYI, here is the command that runs a python file that is in phase2_atari_test.sh:

python atari_reset/check_atari.py --num_timesteps=100000000000 --noops --sticky --num_per_noop=1000 --load_path=$source --game=$game --save_path=$results

Thank you!

I have installed the requirements using pip install -r requirements.txt and git clone https://github.com/uber-research/atari-reset atari_reset. I then run ./phase1_downscaled.sh breakout . 500. I get the following error:

Traceback (most recent call last):
  File "goexplore_py/main.py", line 14, in <module>
    from goexplore_py.randselectors import *
ModuleNotFoundError: No module named 'goexplore_py'

What did I do wrong?

What parts of phase 2 require tensorflow?

Edit: Never mind I see its for the CNN in Atari_reset. It seem to still use tf1.*, but it doesn't look difficult to convert to tf2.0

Hi,
Does the number of max_compute_steps in phase1_montezuma_no_domain.sh refer to the number of game frames seen in the exploration phase or just the number of training frames?
Thank you!