Giter VIP home page Giter VIP logo

beowulf-cluster's Introduction

Beowulf-Cluster

Made by: Paul Gleason, Matt Compton

  1. Project Description
  2. Goals
  3. Prerequisites
  4. Script Setup
  5. Manual Setup

Project Description

This project is to hold our scripts and configs to configure multilocal computing using Chapel.

Goals

  • Learn more about Chapel and parallel computing
  • Make scripts to setup client and controller install scripts

Progress

  • client-install.sh
  • controller-install.sh
  • Have connectivity between nodes and controller
  • Figure out multinode configuration
  • Test benchmark
  • Make weather prediction application
  • Have fully running weather prediction

Prerequisites

Networking

  • 6 Static IP's

Devices

  • 6 Updated Debian machine

OS Config

  • sudo user

Environment

Controller

  • OS: Debian 11
  • Number of Nodes: 1
  • Hosts file to contain nodes

Nodes

  • OS: Debian 11
  • Number of Nodes: 5
  • Host file to contain other nodes and controller

Script Setup

controller-install.sh

  1. Be in the a users home directory.
  2. wget https://raw.githubusercontent.com/ChampPG/Beowulf-Cluster/main/controller-install.sh
  3. chmod +x controller-install.sh
  4. ./controller-install.sh
    • If you already have ssh-keys generated This won't pop up
    • Configure SSH-Keys how you would like
    • Enter

image

  1. Now script will update debian and install Chapel Prerequisites.

client-install.sh

  1. Be in the a users home directory.
  2. wget https://raw.githubusercontent.com/ChampPG/Beowulf-Cluster/main/client-install.sh
  3. chmod +x client-install.sh
  4. ./client-install.sh
    • If you already have ssh-keys generated This won't pop up
    • Configure SSH-Keys how you would like
    • Enter

image

Manual Setup

Controller

  1. sudo hostnamectl set-hostname controller - Rename machine
  2. Configure /etc/hosts

image

  1. ssh-keygen - Generate SSH keys
  2. Install Prerequisites
    sudo apt-get update
    sudo apt-get install gcc g++ m4 perl python3 python3-dev bash make mawk git pkg-config cmake
    sudo apt-get install llvm-dev llvm clang libclang-dev libclang-cpp-dev libedit-dev
    
  3. wget https://github.com/chapel-lang/chapel/releases/download/1.29.0/chapel-1.29.0.tar.gz - wget Chapel-1.29.0.tar.gz
  4. tar xzf chapel-1.29.0.tar.gz - untar ball Chapel-1.29.0
  5. cd chapel-1.29.0 - Enter Chapel dir
  6. source util/setchplenv.bash - Set source
  7. sudo make - Compiler Chapel | Will take some time...
  8. ./util/printchplenv - Show current Chapel environment
  9. export CHPL_HOME=/home/<user>/chapel-1.29.0.tar.gz - Set CHPL_HOME
  10. export CHPL_COMM=gasnet - Set CHPL_COMM
  11. cd $CHPL_HOME - Enter $CHPL_HOME
  12. make - Recompile
  13. chpl -o hello $CHPL_HOME/examples/hello6-taskpar-dist.chpl - Make hello program
  14. export GASNET_SPAWNFN=S - Set GASNET_SPAWNFN
  15. export GASNET_SSH_SERVERS="node1 node2 ..." - Set GASNET_SSH_SERVER

Client

  1. sudo hostnamectl set-hostname node<XX> - Rename machine
  2. Configure /etc/hosts

image

  1. ssh-keygen - Generate SSH keys
  2. Install Prerequisites
    sudo apt-get update
    sudo apt-get install gcc g++ m4 perl python3 python3-dev bash make mawk git pkg-config cmake
    sudo apt-get install llvm-dev llvm clang libclang-dev libclang-cpp-dev libedit-dev
    
  3. wget https://github.com/chapel-lang/chapel/releases/download/1.29.0/chapel-1.29.0.tar.gz - wget Chapel-1.29.0.tar.gz
  4. tar xzf chapel-1.29.0.tar.gz - untar ball Chapel-1.29.0
  5. cd chapel-1.29.0 - Enter Chapel dir
  6. source util/setchplenv.bash - Set source
  7. sudo make - Compiler Chapel | Will take some time...
  8. ./util/printchplenv - Show current Chapel environment
  9. export CHPL_HOME=/home/<user>/chapel-1.29.0.tar.gz - Set CHPL_HOME
  10. export CHPL_COMM=gasnet - Set CHPL_COMM
  11. cd $CHPL_HOME - Enter $CHPL_HOME
  12. make - Recompile
  13. chpl -o hello $CHPL_HOME/examples/hello6-taskpar-dist.chpl - Make hello program
  14. export GASNET_SPAWNFN=S - Set GASNET_SPAWNFN
  15. export GASNET_SSH_SERVERS="controller node1 node2 ..." - Set GASNET_SSH_SERVER | Test without doing this step first as well. For this step make sure not to put the current node in the list

This project has moved:

GitLab Repository

beowulf-cluster's People

Contributors

champpg avatar somethinggeneric avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.