shelhamer / ale-record Goto Github PK

We should nicely package this a module:

• README
• ale_record dir that holds the actual model, since this is importable by Python
• whatever else is nice

Having made demonstrations now that we record only complete episodes, I have some regrets:

1. the frame limit interrupts an episode, discarding the partial demo at that point
2. an interrupted episode cannot be resumed

I argued for (2) in that it might make demonstrations somehow unnatural, but I'm now more bothered by lost timesteps (and my time) than I am by potential noise in the demonstrations.

At some point, it might be nice to make sure we never lose a timestep that we have demonstrations and leave it to be discarded downstream.

On attempting to record a number of episodes I was unhappy to discover that saving took >10 minutes for ~30 minutes of demonstration while leaving my machine inoperable all the while. We need to speed this up.

• give up on compression infeasible, one episode of space invaders is ~1gb
• compress on-the-fly/chunk -- see #6
• append (only save new episodes)
• multiprocessing for compressing/saving

It's an almost absolute certainty that the format of recordings is going to change at some point in the future, so we may as well make our lives a little easier from the get-go and record versioning metadata in the output. This could be as simple as an integer that increments every time the format changes.

This also raises the question of whether or not we would like to maintain backwards compatibility. We could also just check out older versions whenever we need to read older recordings.

Loading everything all-at-once is memory-intensive and excessive if only part of the data is needed. For instance only snapshots and a span of state-actions are needed for restoring from demos.

We should figure out a more granular approach than Demonstration.load().