Giter VIP home page Giter VIP logo

elasticsearch-changes-plugin's Introduction

Elasticsearch Changes Plugin

The Elasticsearch Changes plugin is a plugin for Elasticsearch that tracks changes to indices and makes this information available
through a REST service. It provides information on node basis. So if you want the complete change information you have to query all nodes in your cluster.

This plugin is outdated! Please go to to https://github.com/jurgc11/es-change-feed-plugin for an updated version.

Installation

Run bin/plugin -install derryx/elasticsearch-changes-plugin

Configuration

You can set the length of the history buffer with the setting changes.history.size. The default is 100.

Usage

Point your browser to http://localhost:9200/_changes/ to see changes for all indices.
If you want to see changes only for one or a list of indices you can use http://localhost:9200/foo,bar,baz/_changes/.

You will get a response like this:

{
  "twitter" : {
    "lastChange" : 1330372696400,
    "changes" : [ {
      "type" : "CREATE",
      "id" : "qf310jYSSK2OsM0fWEXyIA",
      "timestamp" : 1330372680742,
      "version" : 1
    }, {
      "type" : "INDEX",
      "id" : "1",
      "timestamp" : 1330372689220,
      "version" : 2
    }, {
      "type" : "DELETE",
      "id" : "1",
      "timestamp" : 1330372696400,
      "version" : 3
    } ]
  },
  "bubu" : {
    "lastChange" : 1330369466133,
    "changes" : [ ]
  }
}

The timestamp is a UNIX timestamp in milliseconds (epoch midnight, January 1, 1970 UTC).

See changes after a given timestamp

If you want to see only changes after a given timestamp you can add the parameter since to the request (for example http://localhost:9200/_changes/?since=1330282941692 ). The timestamp again is a UNIX timestamp (in milliseconds).

Wait for a index change to happen

You can make your request wait for a change to happen with the parameter wait. Additionally you can give a timeout parameter (in milliseconds).
(example: http://localhost:9200/_changes/?wait=true&timeout=60000 ). The default timeout is 15 minutes.

Version mapping

  • Change Plugin 0.1.x → Elasticsearch 0.18.x
  • Change Plugin 0.2.x → Elasticsearch 0.19.x
  • Change Plugin 0.3.x → Elasticsearch 1.0.x
  • Change Plugin 0.4.x → Elasticsearch 1.7.x

Other information

See project’s GitHub pages for details.

elasticsearch-changes-plugin's People

Contributors

bobby-garmin avatar derryx avatar flavioperezcarreto avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

elasticsearch-changes-plugin's Issues

can't install this plugin on ES 1.3.2

While trying to install this plugin on a vm running elasticsearch 1.3.2 I get the following:

spaans@estest:/usr/share/elasticsearch$ sudo bin/plugin -install derryx/elasticsearch-changes-plugin
-> Installing derryx/elasticsearch-changes-plugin...
Trying https://github.com/derryx/elasticsearch-changes-plugin/archive/master.zip...
Downloading .................DONE
Installed derryx/elasticsearch-changes-plugin into /usr/share/elasticsearch/plugins/changes-plugin
Usage:
    -u, --url     [plugin location]   : Set exact URL to download the plugin from
    -i, --install [plugin name]       : Downloads and installs listed plugins [*]
    -t, --timeout [duration]          : Timeout setting: 30s, 1m, 1h... (infinite by default)
    -r, --remove  [plugin name]       : Removes listed plugins
    -l, --list                        : List installed plugins
    -v, --verbose                     : Prints verbose messages
    -s, --silent                      : Run in silent mode
    -h, --help                        : Prints this help message

 [*] Plugin name could be:
     elasticsearch/plugin/version for official elasticsearch plugins (download from download.elasticsearch.org)
     groupId/artifactId/version   for community plugins (download from maven central or oss sonatype)
     username/repository          for site plugins (download from github master)

Message:
   Error while installing plugin, reason: IllegalArgumentException: Plugin installation assumed to be site plugin, but contains source code, aborting installation.

(Side note: is this a supported configuration at all?)

ES 6.8.0 java.lang.IllegalArgumentException: if you don't need a prefix then use a regular logger

The exception in the title occur at plugin load time, preventing ES from starting up.

Appears to be caused by this change: elastic/elasticsearch@e3763b4

Changing all calls to Logger.getLogger() from something like:

private final Logger log = Loggers.getLogger(ChangesFeedPlugin.class);

to:

private final Logger log = Loggers.getLogger(ChangesFeedPlugin.class, "Changes Feed");

Seems to fix the problem.

Flag to distinguish changes from another cluster

Hiya

I'm following this plugin with interest.

One thing that'd be nice to see is a flag somewhere that allows us to distinguish which changes should show up in the changes stream.

For instance, lets say we're trying two way replication between two clusters, thus allowing writes on both clusters. If a change happens on ClusterA, and you replicate it to ClusterB, you want that change to be flagged as having originated from ClusterA so that you don't get a cycle

The details of solving this I'll leave to you :)

thanks

Clint

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.