Giter VIP home page Giter VIP logo

marathon's Introduction

Marathon Build Status

Marathon is a Mesos framework for long-running services. Given that you have Mesos running as the kernel for your datacenter, Marathon is the init or upstart daemon.

Marathon provides a REST API for starting, stopping, and scaling services. Marathon is written in Scala and can run in highly-available mode by running multiple Marathon instances. The state of running tasks gets stored in the Mesos state abstraction.

Try Marathon now on Elastic Mesos.

Go to the interactive Marathon tutorial that can be personalized for your cluster.

Marathon is a meta framework: you can start other Mesos frameworks such as Chronos or Storm. It can launch anything that can be launched in a standard shell. In fact, you can even start other Marathon instances via Marathon.

Features

  • HA -- run any number of Marathon schedulers, but only one gets elected as leader; if you access a non-leader, you get an HTTP redirect to the current leader
  • Basic Auth and SSL
  • REST API
  • Web UI
  • Metrics -- via Coda Hale's metrics library
  • Service Constraints -- e.g., only one instance of a service per rack, node, etc.
  • Service Discovery and Monitoring
  • Event Subscription -- e.g., if you need to notify an external service about task updates or state changes, you can supply an HTTP endpoint to receive notifications. See the wiki page for details.

Overview

The graphic shown below depicts how Marathon runs on top of Mesos together with the Chronos framework. In this case, Marathon is the first framework to be launched and it runs alongside Mesos. In other words, the Marathon scheduler processes were started outside of Mesos using init, upstart, or a similar tool. Marathon launches two instances of the Chronos scheduler as a Marathon task. If either of the two Chronos tasks dies -- due to underlying slave crashes, power loss in the cluster, etc. -- Marathon will re-start an Chronos instance on another slave. This approach ensures that two Chronos processes are always running.

Since Chronos itself is a framework and receives Mesos resource offers, it can start tasks on Mesos. In the use case shown below, Chronos is currently running two tasks. One dumps a production MySQL database to S3, while another sends an email newsletter to all customers via Rake. Meanwhile, Marathon also runs the services required for the web app, in general.

architecture

The next graphic shows a more application-centric view of Marathon running three tasks: Search, Jetty, and Rails.

Marathon1

As the website gains traction and the user base grows, we decide to scale-out the search and Rails-based services.

Marathon2

Imagine that one of the datacenter workers trips over a power cord and a server gets unplugged. No problem for Marathon, it moves the affected search service and Rails instance to a node that has spare capacity. The engineer may be temporarily embarrased, but Marathon saves him from having to explain a difficult situation!

Marathon3

Setting Up And Running Marathon

Requirements

Installation

  1. Install Mesos. One easy way is via your system's package manager. Current builds for major Linux distributions and Mac OS X are available from Mesosphere on their downloads page.

    If building from source, see the Getting Started page or the Mesosphere tutorial for details. Running make install will install Mesos in /usr/local in the same way as these packages do.

  2. Check out Marathon and use Maven to build a JAR:

     mvn package
    
  3. Run ./bin/build-distribution to package Marathon as an executable JAR (optional).

Running in Production Mode

To launch Marathon in production mode, you need to have both Zookeeper and Mesos running. The following command launches Marathon on Mesos in production mode. Point your web browser to localhost:8080 and you should see the Marathon UI.

./bin/start --master zk://zk1.foo.bar/mesos,zk2.foo.bar/mesos --zk_hosts zk1.foo.bar:2181,zk2.foo.bar:2181

Note the different format of the --master and --zk_hosts options. Marathon uses --master to find the Mesos masters, and --zk_hosts to find Zookeepers for storing state. They are separate options because Mesos masters can be discovered in other ways as well.

Running in Development Mode

Mesos local mode allows you to run Marathon without launching a full Mesos cluster. It is meant for experimentation and not recommended for production use. Note that you still need to run Zookeeper for storing state. The following command launches Marathon on Mesos in local mode. Point your web browser to http://localhost:8080, and you should see the Marathon UI.

./bin/start --master local --zk_hosts localhost:2181

Working on assets

When editing assets like CSS and JavaScript locally, they are loaded from the packaged JAR by default and are not editable. To load them from a directory for easy editing, set the assets_path flag when running Marathon:

./bin/start --master local --zk_hosts localhost:2181 --assets_path src/main/resources/assets/

Configuration Options

  • MESOS_NATIVE_LIBRARY: bin/start searches the common installation paths, /usr/lib and /usr/local/lib, for the Mesos native library. If the library lives elsewhere in your configuration, set the environment variable MESOS_NATIVE_LIBRARY to its full path.

    For example:

    MESOS_NATIVE_LIBRARY=/Users/bob/libmesos.dylib ./bin/start --master local --zk_hosts localhost:2181
    

Run ./bin/start --help for a full list of configuration options.

Example API Usage

Using HTTPie:

http POST localhost:8080/v1/apps/start id=sleep cmd='sleep 600' instances=1 mem=128 cpus=1
http POST localhost:8080/v1/apps/scale id=sleep instances=2
http POST localhost:8080/v1/apps/stop id=sleep

Using HTTPie with constraints:

http POST localhost:8080/v1/apps/start id=constraints cmd='hostname && sleep 600' \
    instances=100 mem=64 cpus=0.1 constraints:='[["hostname", "UNIQUE", ""]]'

Running Chronos with the Marathon Client:

marathon start -i chronos -u https://s3.amazonaws.com/mesosphere-binaries-public/chronos/chronos.tgz \
    -C "./chronos/bin/demo ./chronos/config/nomail.yml \
    ./chronos/target/chronos-1.0-SNAPSHOT.jar" -c 1.0 -m 1024 -H http://foo.bar:8080

Marathon Clients

Help

If you have questions, please post on the Marathon Framework Group email list. You can find Mesos support in the #mesos channel on freenode (IRC). The team at Mesosphere is also happy to answer any questions.

Contributors

Marathon was written by the team at Mesosphere that also developed Chronos, with many contributions from the community.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.