Giter VIP home page Giter VIP logo

mongo-migrate's Introduction

mongo-migrate

NPM: mongodb-migrate

=============

Built starting with a framework from: https://github.com/visionmedia/node-migrate

Installation

$ npm install mongodb-migrate

Usage

Usage: node mongodb-migrate [options] [command]

Options:
	-runmm, --runMongoMigrate		Run the migration from the command line
	-c, --chdir <path>				Change the working directory (if you wish to store your migrations outside of this folder
	-dbc, --dbConfig            	Valid JSON string containing db settings (overrides -c, -cfg, & -dbn), like this:
										-dbc='{ "host": "localhost", "db": "mydbname", "port": 27017, "username": "myuser", "password": "mypwd"}'
	-cfg, --config <filename>		DB config file name
	-dbn, --dbPropName <string>	Property name for the database connection in the config file. The configuration file should
									contain something like
										{
											appDb : { //appDb would be the dbPropName
												host: 'localhost',
												db: 'mydbname',
												//port: '27017' //include a port if necessary
											}
										}

Commands:
	down [revision]		migrate down (stop at optional revision name/number)
	up [revision]		migrate up (stop at optional revision name/number)
	create [title]		create a new migration file with optional [title]

Command-line usage

NPM will install mongodb-migrate into a node_modules folder within the directory it is run. To use mongodb-migrate from the command line, you must always specify the relative path from your current directory to the mongodb-migrate directory for node to be able to find it. Such as: node ./node_modules/mongodb-migrate -runmm create (shown in examples below), or on *nix machines node ./node_modules/mongodb-migrate -runmm create.

Creating Migrations

To create a migration execute with node ./node_modules/mongodb-migrate -runmm create and optionally a title. mongodb-migrate will create a node module within ./migrations/ which contains the following two exports:

var mongodb = require('mongodb');

exports.up = function (db, next) {
	next();
};

exports.down = function (db, next) {
	next();
};

All you have to do is populate these, invoking next() when complete, and you are ready to migrate! If you detect an error during the exports.up or exports.down pass next(err) and the migration will attempt to revert the opposite direction. If you're migrating up and error, it'll try to do that migration down.

For example:

	$ node ./node_modules/mongodb-migrate -runmm create add-pets
	$ node ./node_modules/mongodb-migrate -runmm create add-owners

The first call creates ./migrations/0005-add-pets.js, which we can populate:

exports.up = function (db, next) {
	var pets = db.Collection('pets');
	pets.insert({name: 'tobi'}, next);
};

exports.down = function (db, next) {
	var pets = db.Collection('pets');
	pets.findAndModify({name: 'tobi'}, [], {}, { remove: true }, next);
};

The second creates ./migrations/0010-add-owners.js, which we can populate:

	exports.up = function(db, next){
		var owners = db.Collection('owners');
		owners.insert({name: 'taylor'}, next);
    };

	exports.down = function(db, next){
		var owners = db.Collection('owners');
		owners.findAndModify({name: 'taylor'}, [], {}, { remove: true }, next);
	};

Note, for mongodb 2.x you need to use db.collection('<collection-name>') instead of mongodb.Collection(db, '<collection-name>').

Running Migrations

When first running the migrations, all will be executed in sequence.

	node ./node_modules/mongodb-migrate -runmm
	up : migrations/0005-add-pets.js
	up : migrations/0010-add-owners.js
	migration : complete

Subsequent attempts will simply output "complete", as they have already been executed on the given database. mongodb-migrate knows this because it stores migrations already run against the database in the migrations collection.

	$ node mongodb-migrate -runmm
	migration : complete

If we were to create another migration using node ./node_modules/mongodb-migrate -runmm create coolest-owner, and then execute migrations again, we would execute only those not previously executed:

	$ node ./node_modules/mongodb-migrate -runmm
	up : migrations/0015-coolest-owner

If we were to then migrate using node ./node_modules/mongodb-migrate -runmm down 5. This means to run from current revision, which in this case would be 0015-coolecst-owner, down to revision number 5. Note that you can use either the revision number, or then full revision name 0005-add-pets

	$ node ./node_modules/mongodb-migrate -runmm down 5
	down : migrations/0015-coolest-owner
	down : migrations/0010-add-owners

Configuration

JSON String

This option allows you to pass in the database configuration on the command line, eliminating the need to store settings in a config file. The argument should be wrapped in single quotes, and all keys and string values must be in double quotes. Using this option overrides any of the other config options described below. The "port", "username", and "password" properties are optional.

$ node ./node_modules/mongodb-migrate -runmm -dbc '{ "host":"localhost","db":"mydbname","port":27017,"username":"myuser","password":"mypwd"}' up
migration : complete

Working Directory

The options for connecting to the database are read in from a file. You can configure where the file is read in from and where the migration directory root is by the -c <path> option.

	$ node ./node_modules/mongodb-migrate -runmm -c ../.. up
	migration : complete

This would set the working directory two levels above the mongodb-migrate directory, such as if you included it into another project and it was nested in the node_modules folder.

Config filename

The default configuration filename is default-config.json. If you wish to use a different filename, use the -cfg <filename> option:

	$ node ./node_modules/mongodb-migrate -runmm -cfg my-config.json up
	migration : complete

Config file property name

Inside the configuration file, mongodb-migrate expects the database connection information to be nested inside an object. The default object name is mongoAppDb. If you wish to change this you can use the -dbn <string> option:

	$ node ./node_modules/mongodb-migrate -runmm -dbn dbSettings up
	migration : complete

This would tell mongodb-migrate your config file looks something like:

	{
		"dbSettings": {
			"host": "localhost",
			"db": "myDatabaseName",
			//"port": 27017 //Specifying a port is optional
		}
	}

To connect to a replica set, use the replicaSet property:

	{
		"dbSettings": {
			"replicaSet" : ["localhost:27017","localhost:27018","localhost:27019"],
			"db": "myDatabaseName",
		}
	}

or use connectionString property:

	{
		"dbSettings": {
			"connectionString": "mongodb://user:[email protected]:27018,mongo2.host.com:27018,mongo-arbiter.host.com:27018/?w=majority&amp;wtimeoutMS=10000&amp;journal=true"
		}
	}

connectionString has priority over the other properties

All of these settings can be combined as desired, except for the up/down obviously ;)

Licence

(The MIT License)

Copyright © 2017 Austin Floyd

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

mongo-migrate's People

Contributors

afloyd avatar alanjames1987 avatar avgur avatar bjoernwenzel-tommapps avatar bryant1410 avatar danguja avatar demetriusnunes avatar dxns avatar eduardorost avatar emirose avatar jbrechtel avatar kmiyashiro avatar lvela avatar oshybystyi avatar raiyankamal avatar siboulet avatar timjohns avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

mongo-migrate's Issues

Remove -runmm flag

Apparently, the package does not perform actions without the -runmm argument? Having to add the argument seems very superfluous to me, and I couldn't think of a situation where functionality of CLI tools would require this for basic operations.

Cannot seem to log from up or down function

Hi,

I'm trying your nice module and it seems that I cannot see my console.log() statements printed.

Any idea on what could be the problem? Is there another way to log information on the console?

Can't launch migrations inside Docker

I'm using the following commands to launch the migrations

npm install -g migrate-mongo
npm i mongodb
node migrate-mongo status
node migrate-mongo up

When I run it locally from insider my Dockerfile, which is based on docker:18.05.0-ce and has npm installed with RUN apk add nodejs nodejs-npm make gcc g++ python libgcc libstdc++ everything works fine.

When I launch it on bitbucket pipelines I have the following error

+ [email protected]
added 23 packages in 1.673s
> [email protected] install /opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build
node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.1/bcrypt_lib-v3.0.1-node-v57-linux-x64-musl.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v57 ABI, musl) (falling back to source compile with node-gyp) 
gyp WARN EACCES user "undefined" does not have permission to access the dev dir "/root/.node-gyp/8.9.3"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/.node-gyp"
make: Entering directory '/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/build'
make: Leaving directory '/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/build'
make: *** No rule to make target '../.node-gyp/8.9.3/include/node/common.gypi', needed by 'Makefile'.  Stop.
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/opt/atlassian/pipelines/agent/build/app/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Linux 4.14.63-coreos
gyp ERR! command "/usr/bin/node" "/opt/atlassian/pipelines/agent/build/app/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "build" "--fallback-to-build" "--module=/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/lib/binding/bcrypt_lib.node" "--module_name=bcrypt_lib" "--module_path=/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/lib/binding" "--napi_version=1" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v57"
gyp ERR! cwd /opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt
gyp ERR! node -v v8.9.3
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/node /opt/atlassian/pipelines/agent/build/app/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/lib/binding/bcrypt_lib.node --module_name=bcrypt_lib --module_path=/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/lib/binding --napi_version=1 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v57' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at emitTwo (events.js:126:13)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:925:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
node-pre-gyp ERR! System Linux 4.14.63-coreos
node-pre-gyp ERR! command "/usr/bin/node" "/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt
node-pre-gyp ERR! node -v v8.9.3
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok 
Failed to execute '/usr/bin/node /opt/atlassian/pipelines/agent/build/app/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/lib/binding/bcrypt_lib.node --module_name=bcrypt_lib --module_path=/opt/atlassian/pipelines/agent/build/app/node_modules/bcrypt/lib/binding --napi_version=1 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v57' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Has anyone else faced a similar problem?

Authentication?

Is it possible to specify authentication info in the db config? I need to supply credentials and don't see a clear way on how to set this up.

Migration is only updating first record in a collection

I'm trying to do a relatively simple operation and it is only updating the first record in the collection. This is my first time using mongo-migrate so perhaps I'm doing something wrong....

var mongodb = require('mongodb');

exports.up = function(db, next){
    db.collection('users').find().forEach(function(user) {
        console.log('migrating user: ' + user.emails[0].address);
        user.profile.normalizedEmail = user.emails[0].address.toLowerCase();
        db.collection('users').save(user);
    });

    next();
};

Running node mongo-migrate create setup does not work

Tried to create first patch file by doing:

<main_project> $ npm install mongo-migrate
$ cd node_modules/mongo-migrate
$ npm install (for some reason devDependencies not pulled in (mongodb)
$ cd ..
$ node mogo-migrate create setup
$

Nothing happens and no output is recorded.

Outdated Examples

It seems the examples are outdated, or perhaps its because I'm using the latest 1.2.0 with Mongo 2.x, but I have found the following to work:

exports.up = function(db, next){
    var owners = db.collection('owners');
    owners.insert({name: 'taylor'}, next);
};

exports.down = function(db, next){
    var owners = db.collection('owners');
    pets.findAndModify({name: 'taylor'}, [], {}, { remove: true }, next);
};

It's no longer necessary to require mongodb in migrations.

Error on muliple next() calls

It would be great if the migrator would error when a migration calls next() multiple times. Presently calling next() multiple times from within a migration will result in that migration being recorded in the migrations collection multiple times.

change of value is NOT working

change of value is NOT working

async function changeValueMongo (db, next) {
res = await db.collection('mongos').find({}).forEach(async function (mongo) {
if (mongo.plan.mongoOperation && mongo.plan.mongoOperation.trip) {
mongo.plan.mongoOperation.trip.forEach(function (mongoData) {
if (mongoData.mongoMethod === 'mongo') {
mongoData.mongoMethod = 'mongodb'
}
})
}
await db.collection('mongos').save(mongo)
});
}

During migrate the value is NOT changing in DB

Exception during rollback/down function

I'm new in mongoDb, mongo-migrate and so on.

There is something I don't like, when I run

node node_modules/mongo-migrate -cfg config/custom/mongodb.json -dbn development-migrate -runmm down 5

If there is an exception throwed during the down(), we got the exception (and nothing else, not the line where the exception was generated, bad too) but it looks like the rollback was executed because I can't rollback again! I mean, if there is an exception, everything done during the down function should be rollback to the previous state automatically. And the migration SHOULD NOT be considered done!

Maybe I'm wrong but it looks like that right now, because I need to run the up() function to use the down() function again. Pretty bad, hopefully I'm just doing test and the data are absolutely useless so it's not a problem, yet.

add binary to package.json

So that you can use this as a global cli and so that you can add it to the project package.json scripts

node can not find mongo-migrate after installing

My node doesn't seem to be able to find your module for some reason. I followed the steps in README but doesn't work

npm install mongo-migrate
node mongo-migrate --> Error: Cannot find module

requiring mongodb in migrations

Hi,

I'm just curious, in each new migration you have line var mongodb = require('mongodb'); but it is not used in up or down exports. Why is it there?

npm entry outdated?

Hi :-)

First of all thank you for your work on this project! I came here through npm and was quite surprised to see quite a big difference between what version is listed there (0.1.0) and the changes to the package.json that happened over the last 12 months since 0.1.0 had been pushed there.

Was this just an oversight that 0.2.0 and in January 1.0.0 were never pushed to npm or is some huge refactoring going on right now? :-)

migrate on heroku (and mongoLab)

after successfully migrating changes in db on local host , i couldn't duplicate migration on heroku where my app is deployed.

I run the script throw heroku console:
heroku run "node ./node_modules/mongodb-migrate -runmm -cfg migration-config.json up"

what should be in the configuration file under host, db and port?

Dry run

I couldn't find a way to do a dry run before migrating, is this a feature that's planned for the future?

Error: collection name must be a String

$node ./node_modules/mongodb-migrate -runmm

Connected correctly to server
  up : migrations/0005-add-posts.js
Error inside migration:  migrations/0005-add-posts.js
Error:  [Error: collection name must be a String]
var mongodb = require('mongodb');

exports.up = function(db, next){
  var posts = mongodb.Collection(db, 'posts');
  posts.insert({title: 'post awef'});
  next();
};
exports.down = function(db, next){
  next();
};

Hi,
I can't figure out what's wrong with my code

thanks.

Add support to mongo 3.0

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':migrate'.
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:296)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:86)
    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)
    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:54)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:148)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:105)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:85)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:81)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:39)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:29)
    at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:50)
    at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
    at org.gradle.launcher.Main.doAction(Main.java:33)
    at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
    at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
Caused by: org.gradle.process.internal.ExecException: Process 'command 'node'' finished with non-zero exit value 1
    at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:365)
    at org.gradle.process.internal.DefaultExecAction.execute(DefaultExecAction.java:31)
    at org.gradle.api.internal.file.DefaultFileOperations.exec(DefaultFileOperations.java:164)
    at org.gradle.api.internal.project.AbstractProject.exec(AbstractProject.java:753)
    at org.gradle.groovy.scripts.DefaultScript.exec(DefaultScript.java:186)
    at org.gradle.api.internal.ProcessOperations$exec.callCurrent(Unknown Source)
    at build_7tasdoup20a927ah8moq8bq80n.runIn(/var/lib/jenkins/workspace/game-server-dev/build.gradle:82)
    at build_7tasdoup20a927ah8moq8bq80n.run(/var/lib/jenkins/workspace/game-server-dev/build.gradle:91)
    at build_7tasdoup20a927ah8moq8bq80n$_run_closure22.doCall(/var/lib/jenkins/workspace/game-server-dev/build.gradle:259)
    at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:539)
    at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:520)
    at org.gradle.api.internal.tasks.TaskMutator$1.execute(TaskMutator.java:77)
    at org.gradle.api.internal.tasks.TaskMutator$1.execute(TaskMutator.java:73)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
    ... 44 more

Ignore subdirectories when looking for migrations

mongo-migrate reads content of the "migrations" directory when looking for migrations file. It compares anything found in the migrations directory looking for files that match *.js: https://github.com/afloyd/mongo-migrate/blob/master/index.js#L139

This include comparing subdirectories names.

For example creating a "data" directory to hold json or xml files needed for migrations yields the following warning when running mongo-migrate:

"data" ignored. Does not match migration naming schema

To avoid this warning the data directory needs to be placed outside the migrations directory.

Command line parsing buggy

After an installation through npm, even basic functionality is not working for me:

$ node mongo-migrate -runmm create first-post

path.js:360
        throw new TypeError('Arguments to path.join must be strings');
              ^
TypeError: Arguments to path.join must be strings
    at path.js:360:15
    at Array.filter (native)
    at exports.join (path.js:358:36)
    at create (/home/patrik/tmp/node_modules/mongo-migrate/index.js:234:17)
    at commands.create (/home/patrik/tmp/node_modules/mongo-migrate/index.js:223:4)
    at runMongoMigrate (/home/patrik/tmp/node_modules/mongo-migrate/index.js:299:10)
    at Object.<anonymous> (/home/patrik/tmp/node_modules/mongo-migrate/index.js:351:2)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)

... I've checked with the node-migration package, and the problem exists for me with that package as well.

Does this migration library support replication?

If I have a primary and secondary node, can I just pass both of the host names? For example,

{
    "mongoAppDb"        : {
        "host" : "mongo-host1, mongo-host2",
        "db"   : "db_name",
        "port" : "27017",
        "username" : "username",
        "password" : "password"
    }
}

Will the driver be smart enough to find the primary node when executing the migrations?

Support for ssl connections

Our development mongo instance mandates that we only connect over ssl. And we would like to run migrations against it. But currently it does not seem to support ssl connections.

There are a number of ssl related options that the driver supports but as part of this ticket I propose adding support for only:

ssl (boolean: false)
sslValidate (boolean: true)

How does this sound?

I implement this if it sounds reasonable.

version 0.0.1 from npm registry contains some errors

All those errors are already fixed in current master

  • invalid config file search algorithm
  • impossible to use 'down' mode when importing mongo-migrate as a module

It is preferable to publish version 0.0.2 to npm registry

Cannot read property 'raw' of undefined

TypeError: Cannot read property 'raw' of undefined
    at null.<anonymous> (C:\wamp\www\Ayolan-Translation\node_modules\mongodb\lib\mongodb\connection\server.js:443:77)
    at EventEmitter.emit (events.js:95:17)
    at null.<anonymous> (C:\wamp\www\Ayolan-Translation\node_modules\mongodb\lib\mongodb\connection\connection_pool.js:191:13)
    at EventEmitter.emit (events.js:98:17)
    at Socket.<anonymous> (C:\wamp\www\Ayolan-Translation\node_modules\mongodb\lib\mongodb\connection\connection.js:457:22)
    at Socket.EventEmitter.emit (events.js:95:17)
    at Socket.<anonymous> (_stream_readable.js:746:14)
    at Socket.EventEmitter.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:408:10)
    at emitReadable (_stream_readable.js:404:5)

I ran:

node node_modules/mongo-migrate -cfg config/custom/mongodb.json -dbn development -runmm

My application can connect to the mongo db without any problem.

{
    "development": {
        "host" : "127.0.0.1",
        "port" : "3306",
        "database" : "xxx",
        "db" : "xxx",
        "user" : "",
        "password" : ""
    },

    "production": {
        "host" : "127.0.0.1",
        "port" : "3306",
        "database" : "yyy",
        "db" : "yyy",
        "user" : "",
        "password" : ""
    }
}

Bin do not work because of windows line ending format

Mongo-migrate version located on npm contains cariage returns:

#!/usr/bin/env node^M¬
  2 ^
  3 require('../index.js');^

And this is the reason why bin/mongo-migrate.js doesn't work on linux and mac os (do not know about winsows).

And this is why you suggest to use:

node ./node_modules/mongodb-migrate -runmm create

Insteed of:

mongodb-migrate -runmm create

Could you please fix version on npm (you could use dos2unix for this purpose) and create alias with same bin name as npm-package called: mongodb-migrate.

Collection Undefined on a Simple Migration

I am trying to do a very basic migration by adding a field to all documents. However, I keep getting an error "TypeError: Cannot read property 'update' of undefined". Here is my migration file:

'use strict';

module.exports = {

  up: function (db, next) {
    // TODO write your migration here
    db.Activity.update({}, {$set: {"runtime" : {
                                                    "started" : false,
                                                    "startDate" : null,
                                                    "endDate" : null
                                                }
                                   }
                            }, 
                        false, true);
    next();
  },

  down: function (db, next) {
    // TODO write the statements to rollback your migration (if possible)
    db.Activity.update({}, {$unset: {"runtime" : ""}}, false, true);
    next();
  }

};`

If I run the same queries in MongoDB shell (Robomongo) they work just fine.

Replica sets in mongodb > 2.0

Using latest mongodb native(2.0.49) the function for defining Replica set servers seems to have been changed to ReplSet, and is no longer ReplSetServers

node_modules/mongodb-migrate/lib/db.js:22
   return new mongodb.ReplSetServers(replServers);
          ^
TypeError: undefined is not a function
    at getReplicaSetServers (...node_modules/mongodb-migrate/lib/db.js:22:11)
    at Object.getConnection (...node_modules/mongodb-migrate/lib/db.js:31:11)
    at performMigration (...node_modules/mongodb-migrate/index.js:267:6)
    at commands.up (...node_modules/mongodb-migrate/index.js:203:4)
    at runMongoMigrate (...node_modules/mongodb-migrate/index.js:319:10)
    at Object.<anonymous> (...node_modules/mongodb-migrate/index.js:375:2)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)

verror is not defined

When database connection failed

ReferenceError: verror is not defined
at /app/node_modules/mongodb-migrate/index.js:268:21

ReferenceError: verror is not defined
at /app/node_modules/mongodb-migrate/index.js:275:22

Migration method "run()", pass revision number

Hi, I use mongo-migrate inside of grunt task like this:

let mm = require('mongodb-migrate')

mm.setDbConfig(JSON.stringify(mmConfig));
mm.run("up");

Can I pass revision number to this method like?

mm.setDbConfig(JSON.stringify(mmConfig));
mm.run("up", 10);

Or which other way can I get exect result as using this command from console?

node ./node_modules/mongodb-migrate -runmm down 10

What does migrationEnd mean function runMongoMigrate(direction, migrationEnd, next)?

Not relevant, have found mistake it was on my side.

TypeError: db.Collection is not a function

From your example

exports.up = function (db, next) {
    var pets = db.Collection('pets');
    pets.insert({name: 'tobi'}, next);
};

I get TypeError: db.Collection is not a function
when I run
./node_modules/mongodb-migrate -runmm -dbc {"host":"localhost","db":"mydb","port":27017} up

Full trace:

at Migration.exports.up (migrations/0100-migrate.js:5:17)
    at next (node_modules/mongodb-migrate/lib/set.js:161:24)
    at Set._migrate (node_modules/mongodb-migrate/lib/set.js:212:2)
    at node_modules/mongodb-migrate/lib/set.js:102:8
    at Set.load (node_modules/mongodb-migrate/lib/set.js:59:2)
    at Set.migrate (node_modules/mongodb-migrate/lib/set.js:93:7)
    at Set.up (node_modules/mongodb-migrate/lib/set.js:79:7)
    at node_modules/mongodb-migrate/index.js:312:19
    at handleCallback (node_modules/mongodb/lib/utils.js:120:56)
    at node_modules/mongodb/lib/cursor.js:861:16
    at handleCallback (node_modules/mongodb-core/lib/cursor.js:171:5)
    at setCursorDeadAndNotified (node_modules/mongodb-core/lib/cursor.js:505:3)
    at nextFunction (node_modules/mongodb-core/lib/cursor.js:608:12)
    at Cursor.next [as _next] (node_modules/mongodb-core/lib/cursor.js:701:3)
    at fetchDocs (node_modules/mongodb/lib/cursor.js:857:10)
    at node_modules/mongodb/lib/cursor.js:880:7

Version:

npm view mongodb-migrate version
2.0.2

Adding subdocuments while seeding data

I'm trying to add seed data using mongo-migrate. The problem I'm running into is that I want to add a subdocument as defined here. Subdocuments also have an _id field, but I'm not sure how to make that work. For example, I have a TimelineItem model which has a notes subdocument (which comes from noteSchema), but the only way I can add a new document to the timelineitems collection (and kind of emulating a subdocument) is to say timelineitems.insert({notes: [{//first note}, {//second note}]) but I don't think that captures the structure of the subdocument (like it doesn't have an _id etc.).

How should I make this work?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.