Giter VIP home page Giter VIP logo

elasticsearch's Introduction

Elasticsearch

An opinionated Docker implementation for Elasticsearch focused on providing a reliable yet flexible solution based on the upstream official image.

Features

There are a number of features that differentiate this implementation from the official or other solutions.

  • Configuration determined at runtime by a powerful templating engine.
  • AWS availability zone awareness auto detection.
  • Extra configuration snippets easy to add.
  • Heap sized based on container memory and percentages.
  • Enables AWS S3 plugin for snapshots.
  • Enables Prometheus plugin for metrics.

Getting Started

This implementation is designed to be used with a Helm chart. However, there is a docker compose file included for testing this implementation locally. Bringing up a test cluster and discovering the client endpoint is easily done with just 2 commands.

docker-compose up

It will likely take a few minutes for the containers to start and you can verify when it is available using curl.

curl $(docker-compose port client 9200)

Prerequisites

The only requirements for bringing up this implementation locally is Docker.

Docker 17.12.0+

However, be aware that Elasticsearch has significant memory requirements especially since it is Java based and it is necessary to allocate the heap on start to prevent memory fragmentation. With 4 Elasticsearch containers (Discovery, Master, Data and Client), Kibana and Cerebro at least 4388Mi of RAM is required in heap allocation, preferably 6Gi if there is capacity.

Installing

It is expected that the Helm chart will be used to install this container. For local development, there are a number of ways to interact with Elasticsearch.

The client endpoint is the main way applications interact with Elasticsearch.

docker-compose port client 9200

Management of Elasticsearch is available using Kibana.

docker-compose port kibana 5601

An alternative management interface is available using Cerebro.

docker-compose port cerebro 9000

Environmental Variables

Variable Default Comment
ES_CLUSTER_NAME example
ES_NETWORK_HOST 0.0.0.0
ES_NODE_NAME node
ES_NODE_MASTER false
ES_NODE_DATA false
ES_XPACK_SECURITY_ENABLED false
ES_XPACK_MONITORING_ENABLED true
ES_XPACK_MONITORING_COLLECTION_ENABLED true
ES_EMAIL_AWS_SES_ENABLED false
ES_EMAIL_AWS_SES_HOST email-smtp.us-east-1.amazonaws.com
ES_EMAIL_AWS_SES_USER Unset
ES_EMAIL_AWS_SES_PASSWORD Unset
ES_HTTP_CORS_ENABLED false
ES_HTTP_CORS_ALLOW_ORIGIN *
ES_GATEWAY_RECOVER_AFTER_DATA_NODES 1
ES_GATEWAY_EXPECTED_DATA_NODES 1
ES_GATEWAY_RECOVER_AFTER_TIME 1m
ES_DISCOVERY_ZEN_MINIMUM_MASTER_NODES 1
ES_DISCOVERY_ZEN_PING_UNICAST_HOSTS localhost
ES_DISCOVERY_TYPE Unset

Extra Settings

The existence of /usr/share/elasticsearch/config/elasticsearch_custom.yml will automatically be appended to the elasticsearch.yml settings at runtime.

Memory Allocation

Running a JVM in a container requires an understand of the relationship of the cgroups memory limit and the heap size. There is an overhead required in running a JVM as well which is approximate 348Mi.

A simple formula to use to determine the RAM percentage is as follows:

(Memory Limit - 384Mi / Memory Limit) * 100

Deployment

For use on live systems, see the documentation for the Helm chart.

Components

  • Elasticsearch - Open Source, Distributed, RESTful Search Engine
  • Elasticsearch-Docker - Official Elasticsearch Docker image from Elastic.
  • Gomplate - A flexible commandline tool for template rendering. Supports lots of local and remote datasources.
  • Kibana - Kibana analytics and search dashboard for Elasticsearch
  • Cerebro - Cerebro is an open source Elasticsearch web admin tool built using Scala, Play Framework, AngularJS and Bootstrap
  • Prometheus Exporter Plugin for Elasticsearch - Prometheus exporter plugin for Elasticsearch

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

Authors

  • Michael Lorant - Initial work - Nine

See also the list of contributors who participated in this project.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments

  • Elastic
  • Nine

elasticsearch's People

Contributors

mikelorant avatar renovate-bot avatar matthope avatar renovate[bot] avatar

Watchers

 avatar James Cloos avatar Ben Tucker avatar Joel avatar  avatar Marc de Beer avatar Syed Imam avatar  avatar Kostas Georgiou avatar  avatar

elasticsearch's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

docker-compose
docker-compose.yaml
  • docker.elastic.co/kibana/kibana 6.8.21
dockerfile
Dockerfile
  • hairyhenderson/gomplate v3.11.5
  • docker.elastic.co/elasticsearch/elasticsearch 6.8.21

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.