Multi-processing with MPI about kinc HOT 5 CLOSED

The MPI virtual functions will be OpenCL neutral, so you can use OpenCL or not. You would test if OpenCL is available with if ( EOpenCLDevice::getInstance().getStatus() == Ok ) { /* Do OpenCL stuff / } else { / DO serial stuff */ }

from kinc.

That is exactly what I am working on right now. It is still in the research/test assumptions stage so I can make no promises but I know what I would like to do;

Basically ACE will handle MPI behind the scenes so a developer using ACE does not have to worry about it. The general functions I will be shooting for are these;

bool buildWorkPackage(QByteArray* package);
Build a package of work to be done on a node, return false if there is no more work to be done else return true.
void processWorkPackage(const QByteArray& package);
Process the work a node has done.
QByteArray workPackage(cosnt QByteArray& package);
As a work node do the work given by package and save the results as the return byte array.

So with that the manager(first) node will use buildWorkPackage() and processWorkPackage(). Each worker node will run workPackage().

This is all off the top of my head and the function names may change but that is the general idea I am working on right now.

from kinc.

Okay, yeah I think it's a good idea to incorporate it within ACE. I'll be ready to test whatever API you come up with on the clustering analytics whenever it comes out, assuming the idea works.

from kinc.

Here's another question I was wondering about: when each node receives it's piece of work, will it be free to process it using serial code or OpenCL? As opposed to forcing every node to use one or the other uniformly?

I ask because if it can be heterogeneous, we could utilize more nodes on a cluster like Palmetto, which has both CPU nodes and GPU nodes.

from kinc.

The GMM analytic now has MPI support, and each slave node is able to run OpenCL or serial based on availability. Continuing this thread in the ACE issue.

from kinc.