obierlaire / elastic-doc-manager Goto Github PK

This package is a document manager for mongo-connector.

The easiest way to install elastic-doc-manager is with pip:

pip install elastic-doc-manager

You can also install the development version of elastic-doc-manager manually:

git clone https://github.com/mongodb-labs/elastic-doc-manager
cd elastic-doc-manager
python setup.py install

You may have to run python setup.py install with sudo, depending on where you're installing mongo-connector and what privileges you have.

For information on running mongo-connector with elastic, please see https://github.com/mongodb-labs/mongo-connector/wiki/Usage%20with%20ElasticSearch

1. Copy of the Elastic 1.x Document Manager Github repository

The tests are not included in the package from PyPI and can only be acquired by cloning this repository on Github:

git clone https://github.com/mongodb-labs/elastic-doc-manager

2. Mongo Orchestration

Mongo Connector runs MongoDB on its own using another tool called Mongo Orchestration. This package should install automatically if you run python setup.py test, but the Mongo Orchestration server still needs to be started manually before running the tests:

mongo-orchestration --bind 127.0.0.1 --config orchestration.config start

will start the server. To stop it:

mongo-orchestration --bind 127.0.0.1 --config orchestration.config stop

The location of the MongoDB server should be set in orchestration.config. For more information on how to use Mongo Orchestration, or how to use it with different arguments, please look at the Mongo-Orchestration README.

3. Environment variables

There are a few influential environment variables that affect the tests. These are:

• DB_USER is the username to use if running the tests with authentication enabled.
• DB_PASSWORD is the password for the above.
• MONGO_PORT is the starting port for running MongoDB. Future nodes will be started on sequentially increasing ports.
• ES_HOST is the hostname on which Elasticsearch is running.
• ES_PORT is the port on which Elasticsearch is running.
• MO_ADDRESS is the address to use for Mongo Orchestration (i.e. hostname:port)

All the tests live in the tests directory.

While the tests take care of setting up and tearing down MongoDB clusters on their own, make sure to start Elastic before doing a full test run!

You can run all the tests with one command (this works in all supported Python versions):

python setup.py test

In addition, you can be more selective with which tests you run (in Python > 2.6 only)! For example, if you only wanted to run the elastic doc manager tests:

python -m unittest tests.test_elastic_doc_manager

Some of the tests are meant to generate lots of ERROR-level log messages, especially the rollback tests. mongo-connector logs exceptions it encounters while iterating the cursor in the oplog, so we see these in the console output while MongoDB clusters are being torn apart in the tests. As long as all the tests pass with an OK message, all is well.