Giter VIP home page Giter VIP logo

cyclecloud-slurm's Introduction

Slurm

This project sets up an auto-scaling Slurm cluster

Pre-Requisites

This sample requires the following:

  1. CycleCloud must be installed and running.

    a. If this is not the case, see the CycleCloud QuickStart Guide for assistance.

  2. The CycleCloud CLI must be installed and configured for use.

  3. You must have access to log in to CycleCloud.

  4. You must have access to upload data and launch instances in your chosen Azure subscription

  5. You must have access to a configured CycleCloud "Locker" for Project Storage (Cluster-Init and Chef).

  6. You need to build the Slurm RPMs and put them in the blobs/ directory. There is a script in the cluster-init scratch directory to build the RPMs on a cloud node. Alternatively, you can run the included ./docker-rpmbuild.sh on a development machine with docker installed. This docker-rpmbuild.sh script will generate the rpm files for CentOS 7.x compatible nodes and place them in the blobs/ directory.

  7. Optional: To use the cyclecloud project upload <locker> command, you must have a Pogo configuration file set up with write-access to your locker.

    a. You may use your preferred tool to interact with your storage "Locker" instead.

  8. Centos 7 with Cyclecloud 6.7.0 has been tested and confirmed working.

Usage

A. Deploying the Project

The first step is to configure the project for use with your storage locker:

  1. Open a terminal session with the CycleCloud CLI enabled.

  2. Switch to the Slurm sample directory.

  3. Upload the project (including any local changes) to your target locker, run the cyclecloud project upload command from the project directory. The expected output looks like this:::

$ cyclecloud project upload
Sync completed!

IMPORTANT

For the upload to succeed, you must have a valid Pogo configuration for your target Locker.

B. Importing the Cluster Template

To import the cluster:

  1. Open a terminal session with the CycleCloud CLI enabled.

  2. Switch to the Slurm directory.

  3. Run cyclecloud import_template Slurm -f templates/slurm_template.txt. The expected output looks like this:::

    $ cyclecloud import_template Slurm -f templates/slurm_template.txt Importing template Slurm....

    Slurm : template

    Keypair: $keypair Cluster nodes: master: off Total nodes: 1

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.