Parallelise the tournament about axelrod HOT 12 CLOSED

I've no experience of this, but I fancy learning!

Any preferences on how this should be done and what library to use? From the (little) research I've done, it looks like multiprocessing might the easiest option.

from axelrod.

That would be awesome, I'm just struggling for time and really think that docs should be my priority... So it's awesome if you can pick this up.

The multiprocessing library is what I've always used (and again it's in the base library: 👍 ). I might be able to help along the way though so throw some questions my way (although as I said above: not sure I've every actually done this the right way - it has always felt hacky).

from axelrod.

Right - I'll make a start. Branch 68 in my fork will contain anything I've done so far so anyone can take a look and contribute. I'll create a PR here when there's something worth merging.

from axelrod.

Just taken a look.

Is the idea to have this as an option that gets passed to the tournament_manager?

Eventually this could be then passed to the run_tournament script through something like:

python run_tournament.py # Run on one core (status quo)
python run_tournament.py -p # Run with all available cores
python run_tournament.py -p 2 # Run with two cores

from axelrod.

Yep, that's exactly the plan!

To start, I'm just going to run the individual tournaments in parallel (basic_strategies, cheating_strategies etc.) I'll look into the rounds and repetitions once I get that working.

from axelrod.

I was wondering about this myself. What do you reckon it make sense to
parallelise? The tournament or repetitions of the tournament?

I don't think in the grand scheme of things that it matters (as long as all
workers are busy) but the second approach would be easier to implement...

On Wed, Mar 11, 2015 at 10:01 AM Owen Campbell [email protected]
wrote:

Yep, that's exactly the plan!

To start, I'm just going to run the individual tournaments in parallel
(basic_strategies, cheating_strategies etc.) I'll look into the rounds and
repetitions once I get that working.

—
Reply to this email directly or view it on GitHub
https://github.com/drvinceknight/Axelrod/issues/68#issuecomment-78234611
.

from axelrod.

No idea! I intend to hit the easiest option first and see what happens.

from axelrod.

Cool :)

On Wed, Mar 11, 2015 at 10:05 AM Owen Campbell [email protected]
wrote:

No idea! I intend to hit the easiest option first and see what happens.

—
Reply to this email directly or view it on GitHub
https://github.com/drvinceknight/Axelrod/issues/68#issuecomment-78235260
.

from axelrod.

Progress so far......

Running the individual tournaments in parallel is simple, but yields little benefit since the longest running tournament is significantly longer than the quickest.

Running the repetitions in parallel is simple so long as the deterministic cache is rebuilt on each repetition (since attempting to share any kind of global object between processes is best avoided). Losing the cache destroys the gains made by running in parallel!

Here's what I plan to try next:
Play the first round robin, in order to build the cache. Play the remaining repetitions in parallel with the cache then available to all from the first.

from axelrod.

That sounds eminently sensible to me. Thanks!

On Tue, Mar 17, 2015 at 9:01 AM Owen Campbell [email protected]
wrote:

Progress so far......

Running the individual tournaments in parallel is simple, but yields
little benefit since the longest running tournament is significantly longer
than the quickest.

Running the repetitions in parallel is simple so long as the deterministic
cache is rebuilt on each repetition (since attempting to share any kind of
global object between processes is best avoided). Losing the cache destroys
the gains made by running in parallel!

Here's what I plan to try next:
Play the first round robin, in order to build the cache. Play the
remaining repetitions in parallel with the cache then available to all from
the first.

—
Reply to this email directly or view it on GitHub
https://github.com/drvinceknight/Axelrod/issues/68#issuecomment-82434670
.

from axelrod.

Running the second and subsequent repetitions is showing some promise. It's crude at the moment, but there's working code on branch 68 in my fork.

(There are new tests required for the refactoring of the Tournament class and I've hard coded the max number of parallel processes for now).

from axelrod.

Will try and take a look although things are pretty busy during the day
here.

Why hard code it?

import multiprocessing

multiprocessing.cpu_count()

On Wed, Mar 18, 2015 at 5:04 AM Owen Campbell [email protected]
wrote:

Running the second and subsequent repetitions is showing some promise.
It's crude at the moment, but there's working code on branch 68 in my fork.

(There are new tests required for the refactoring of the Tournament class
and I've hard coded the max number of parallel processes for now).

—
Reply to this email directly or view it on GitHub
https://github.com/drvinceknight/Axelrod/issues/68#issuecomment-82936775
.

from axelrod.