• Roadmap
  • Features
  • Depends on upcoming Mesos features
  • Developer Tools
  • User tools
  • Certification
• Getting Started
• Users Guide
  • How to install on Marathon
  • User Interface
• Developers Guide
  • Quickstart
  • How to run on Linux
    • Requirements
  • How to run on Mac
    • Requirements
  • System test
    • How to run system tests on Linux
      • Requirements
    • How to run on Mac
      • Requirements
  • How to release
• Sponsors
• License

• Deployment
• Durable cluster topology (via ZooKeeper)
• Web UI on scheduler port 8080
• Support deploying multiple Elasticsearch clusters to single Mesos cluster
• Fault tolerance
• High availability (master, indexer, replica)
• Upgrading configuration
• Scale cluster horizontally
• Scale cluster vertically
• Upgrade
• Rollback
• Snapshot and restore

• Faster task recovery with Mesos dynamic reservations (https://issues.apache.org/jira/browse/MESOS-1554)
• Persistent storage

• Local environment (Docker-machine)
• Rapid code + test (Mini Mesos)
• Build automation (Gradle)

• One click DCOS install
• One JSON post to marathon install

• DCOS Certified

We recommend that users install via marathon or via the DCOS command line (coming soon!).

This framework requires:

• A running Mesos cluster
• The use of Marathon is strongly recommended to provide resiliency against scheduler failover.

Create a Marathon file like the one below and fill in the IP addresses and other configuration.

{
  "id": "elasticsearch-mesos-scheduler",
  "container": {
    "docker": {
      "image": "mesos/elasticsearch-scheduler",
      "network": "HOST"
    }
  },
  "args": ["-n", "3", "-zk", "zk://ZOOKEEPER_IP_ADDRESS:2181/mesos", "-ram", "2048"],
  "cpus": 0.2,
  "mem": 512.0,
  "env": {
    "JAVA_OPTS": "-Xms128m -Xmx256m"
  },
  "instances": 1
}

Then post to marathon to instantiate the scheduler: curl -k -XPOST -d @marathon.json -H "Content-Type: application/json" http://MARATHON_IP_ADDRESS:8080/v2/apps

Note: the JAVA_OPTS line is required. If this is not set, then the Java heap space will be incorrectly set.

The web based user interface is available on port 8080 of the scheduler by default. It displays real time information about the tasks running in the cluster and a basic configuration overview of the cluster.

The user interface uses REST API of the Elasticsearch Mesos Framework. You can find the API documentation here: docs.elasticsearchmesosui.apiary.io.

• Issue #206: The GUI will not represent the true state of the cluster after a scheduler or executor reconciliation event. E.g. If the scheduler is killed and restarted, the GUI will show zero executors, even though there are executors present.
• Issue #188: Database data IS NOT persisted to disk. Data storage is wholly reliant on cluster redundancy. This means that the framework is not yet recommended for production use.

For developers, we have provided a range of tools for testing and running the project. Check out the mini-mesos project for an in-memory Mesos cluster for integration testing.

You can run Mesos-Elasticsearch using Mini Mesos, a containerized Mesos cluster for testing frameworks.

• Docker

$ ./gradlew build system-test:main

• Docker Machine

$ docker-machine create -d virtualbox --virtualbox-memory 4096 --virtualbox-cpu-count 2 mesos-es
$ eval $(docker-machine env mesos-es)
$ ./gradlew build buildDockerImage system-test:main

The project contains a system-test module which tests if the framework interacts correctly with Mesos, using Mini Mesos. We currently test Zookeeper discovery and the Scheduler's API by calling endpoints and verifying the results. As the framework grows we will add more system tests.

• Docker

$ ./gradlew build system-test:systemTest

• Docker Machine

$ docker-machine create -d virtualbox --virtualbox-memory 4096 --virtualbox-cpu-count 2 mesos-es
$ eval $(docker-machine env mesos-es)
$ ./gradlew build buildDockerImage system-test:systemTest

Create the following Gradle property file in ~/.gradle/gradle.properties and refer to your Github and Docker Hub user/pass.

systemProp.org.ajoberstar.grgit.auth.interactive.allow=false
systemProp.org.ajoberstar.grgit.auth.ssh.private=~/.ssh/id_rsa
systemProp.org.ajoberstar.grgit.auth.username=user
systemProp.org.ajoberstar.grgit.auth.password=password
dockerHubUsername=user
dockerHubPassword=******
dockerHubEmail=email

Now perform a release and specify the release type: major, minor or patch and your username.

$ ./gradlew build system-test:systemTest

If build and tests succeed release

$ ./gradlew release -PreleaseType={major,minor,patch} -PuserName={user}

This project is sponsored by Cisco Cloud Services

Apache License 2.0