Comments (2)
Hey Badr,
Thanks for your interest in TorchBeast.
I have to acknowledge we never validated monobeast against the Atari suite. It cannot (easily) be run with the right batch size so it's tricky to use the known-good hyperparameters. I believe its design is sound in principle (CPU actors, one GPU learner, PyTorch shared memory tensors for inter-process communication) but there's lots of things that can go wrong with RL.
Re: The NaNs: I don't know what mean_expected_return
is, I'm guessing you're seeing mean_episode_return
? If so, we made the (perhaps questionable) decision to log NaN if there is no episode ending in the batch in the first place. This happens "naturally" via division by zero (the "mean" over no episodes). This implementation logs per-rollout numbers on the learner side and not all rollouts contain finished episodes. Since we also want to log speed, frame counts etc we felt we had to log something for mean_episode_return
so we decided for the somewhat "natural" NaN there. Many plotting libraries ignore these values, which is what you'd want to do at that point. This approach has the obvious downside of scaring researchers, who are typically trained to associate NaNs with "something went very wrong".
from torchbeast.
I see, and yes I apologize for the typo, I indeed meant mean_episode_return
.
Thank you for your prompt response!
from torchbeast.
Related Issues (20)
- "Done" default of 1 results in 0 reward episodes HOT 1
- Cannot reproduce the performance of "SpaceInvaders" game? HOT 1
- error install nest on Ubuntu 16 HOT 1
- How exactly monobeast and polybeast are different in performance perspective? HOT 1
- PolyBeast build fails with Python 3.8 HOT 8
- Using more than 2 GPUs with Polybeast HOT 5
- Minimum parameter configuration for not bad training results HOT 1
- Error on installing PolyBeast HOT 1
- How does IMPALA join the exploration mechanism? HOT 2
- Why doesn't the test code use LSTM? HOT 2
- Continuous Action Apace
- torchbeast works poorly on atari.
- request for support for continuous and multi-discrete action space HOT 2
- default instructions for monobeast in Pong HOT 1
- I encountered some problems when I ran the command pip install ".[polybeast]"
- Can't install polybeast in WSL2
- is it possible to use this architecture with multiple cpu?
- Reproducibility Issue for torchbeast
- Excessive memory use in monobeast.py
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torchbeast.