derryx / elasticsearch-changes-plugin Goto Github PK

Hiya

I'm following this plugin with interest.

One thing that'd be nice to see is a flag somewhere that allows us to distinguish which changes should show up in the changes stream.

For instance, lets say we're trying two way replication between two clusters, thus allowing writes on both clusters. If a change happens on ClusterA, and you replicate it to ClusterB, you want that change to be flagged as having originated from ClusterA so that you don't get a cycle

The details of solving this I'll leave to you :)

thanks

Clint

Any plans on adding support for the latest beta versions? If yes, that will be great help.

While trying to install this plugin on a vm running elasticsearch 1.3.2 I get the following:

spaans@estest:/usr/share/elasticsearch$ sudo bin/plugin -install derryx/elasticsearch-changes-plugin
-> Installing derryx/elasticsearch-changes-plugin...
Trying https://github.com/derryx/elasticsearch-changes-plugin/archive/master.zip...
Downloading .................DONE
Installed derryx/elasticsearch-changes-plugin into /usr/share/elasticsearch/plugins/changes-plugin
Usage:
    -u, --url     [plugin location]   : Set exact URL to download the plugin from
    -i, --install [plugin name]       : Downloads and installs listed plugins [*]
    -t, --timeout [duration]          : Timeout setting: 30s, 1m, 1h... (infinite by default)
    -r, --remove  [plugin name]       : Removes listed plugins
    -l, --list                        : List installed plugins
    -v, --verbose                     : Prints verbose messages
    -s, --silent                      : Run in silent mode
    -h, --help                        : Prints this help message

 [*] Plugin name could be:
     elasticsearch/plugin/version for official elasticsearch plugins (download from download.elasticsearch.org)
     groupId/artifactId/version   for community plugins (download from maven central or oss sonatype)
     username/repository          for site plugins (download from github master)

Message:
   Error while installing plugin, reason: IllegalArgumentException: Plugin installation assumed to be site plugin, but contains source code, aborting installation.

(Side note: is this a supported configuration at all?)

When a shard switches from replica to primary state or vice versa we do not get all changes or show duplicates.

The exception in the title occur at plugin load time, preventing ES from starting up.

Appears to be caused by this change: elastic/elasticsearch@e3763b4

Changing all calls to Logger.getLogger() from something like:

private final Logger log = Loggers.getLogger(ChangesFeedPlugin.class);

to:

private final Logger log = Loggers.getLogger(ChangesFeedPlugin.class, "Changes Feed");

Seems to fix the problem.

Allow to connect to the plugin and have the connection waiting for a change on an index and then come back with the first incoming change.
So the client gets an instant information about a change on indices.

This feature should be inspired by http://guide.couchdb.org/draft/notifications.html#long

We should collect changes from other nodes as well so you can ask one node for changes on all nodes.

Don't use a circular buffer to track changes but use the translog. That saves memory.