MPIFarm

Is a project of render farm working with Open MPI.

Requirements

Open MPI is only available on Linux.
You will need to install C++ compiler and cmake (and OpenSSH server if you plan to create a cluster).

# Debian/Ubuntu
sudo apt install g++ cmake (optional openssh-server)
# Arch Linux
sudo pacman -Sy gcc cmake (optional ssh)

You will also need to install Open MPI and OpenCV components like this :

# Debian/Ubuntu
sudo apt install libopenmpi-dev openmpi-bin libopencv-dev
# Arch Linux
sudo pacman -Sy openmpi opencv vtk

Compiling

To compile, just type:

./cmakecompile [optionnal: --release]

Configure

On master/local computer

Generate SSH Key Pair

ssh-keygen -t rsa -b 4096

it might be created in ~/.ssh/id_rsa

Copy Public Key to the Remote Machine

ssh-copy-id username@remote-machine

it might be copied in ~/.ssh/authorized_keys

On remote machine

Modify SSH Configuration (if necessary)

sudo nvim /etc/ssh/sshd_config

You can also use nano instead of nvim :p

PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no

# Debian/Ubuntu
sudo systemctl restart ssh
# Arch Linux
sudo systemctl restart sshd

Start SSH Agent for the User (optionnal ?)

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_rsa

Enjoy !

Now, you should be able to connect in ssh to the distant machine without password.

Define set of images

You can define set of images using jsonGenerator.

Compile it

./jsonCompileGenerator.sh

Then execute it

./jsonGenerator.o

And then choose your settings.

Execute

Using start.sh

./start single|mega|custom your_image_set.json [optionnal: number_of_processes_to_spawn]

Manually

To start the program:

/usr/bin/mpirun -np 2 build/MPIFarm P170B328_ServieresV_L3_small.json
# or
mpirun -hostfile hostConfigs/h_hostfile.cfg -np 2 build/MPIFarm P170B328_ServieresV_L3_small.json

If you have an Invalid MIT-MAGIC-COOKIE-1 key6533927 error :

ssh -X user@remote_machine
# and/or
xauth generate :0 . trusted # or :1, check echo $DISPLAY

If it is still here but cause no problem, just hide it with:

mpirun -hostfile hostConfigs/h_hostfile.cfg -np 2 build/MPIFarm P170B328_ServieresV_L3_small.json 2>/dev/null

To compile and execute some tests, use

mpicxx -o test.o test.cpp
mpiexec -hostfile hostfile -np 5 ./test.o

magictintin / ktu_s5_mpifarm Goto Github PK

ktu_s5_mpifarm's Introduction