The README suggests a default value for port of 0, but 0 doesn't validate.

In order to replace usage of something like https://www.npmjs.com/package/command-line-args it would be helpful to allow arg to have an alias. For example, I want -h and --help to both allow for help to be called.

Example usage:

        help: {
            arg: ['help', 'h'],
            type: Boolean,
            doc: 'Display this usage message',
            default: false
        },

On a related note, getSchemaString seems to print something like:

{
  "properties": {
    "help": {},
    "env": {},
  }
}

Any idea why it is mostly blank. Ideally, I'd like to see the output mirror https://www.npmjs.com/package/command-line-args#module_command-line-args--CommandLineArgs+getUsage

I'm suddenly struct by the possibility of using convict in something like awsboxen and other command-line tools, to let you specify options either in a config file, in the environment, or on the command-line, and have them validated against the schema automatically.

Do you think this would be in-scope for convict? I'm happy to try hacking on it if so.

Production deployment services like heroku typically use a diverse set of "standard" providers, each with a similar format but their own environment variable. It'd be nice to support these "out of the box". So something like:

mongoUri: {
    doc: "Mongo DB configuration string.",
    format: "uri",
    default: 'mongodb://localhost/fullstack',
    env: ["MONGOLAB_URI", "MONGOHQ_URL"]
  }

hey all -

more of a question than an issue here. i'm using convict for a fairly traditional web app. i'd like to make it a bit easier to share config settings between server and client. for example, say you wanted to share validation rules / constraints - something like username max length. today, you basically serialize some subset of your configs (taking care to avoid exposing sensitive settings), output with your markup, then deserialize and use client-side. there are variations to the theme here obviously but the common bits are a) flagging certain settings as safe to send to the client and b) only exposing the safe subset of settings.

the key thing I could see convict helping w/ is the ability to flag certain settings as "public" or safe to share with the client. to this end, i'm imagining extending the existing schema options w/ something like public: true (by default all settings would be considered private / server-side only) and adding a getter to retrieve all "public" settings for easier serialization.

i'm happy to dive in and put together a PR but figured i would check here first. is this something that sounds useful and a good addition to convict? are people already handling this in a nicer way?

thanks for a helpful lib!
-matt

Convict's package.json is giving me trouble with license validation. It appears to use a deprecated license format; please consider updating it to something more recent:

https://docs.npmjs.com/files/package.json#license

If you add a function and try to get it, it returns an empty object:

function someFunction () {};
conf.set('path', someFunction);
conf.get('path'); // returns {}

Seems like it got disabled somehow https://travis-ci.org/mozilla/node-convict ?

Steps to reproduce:

1. Copy/paste the following magic sauce as ./index.js:

   var convict = require('convict');
   
   var conf = convict({
   session: {
     format: 'duration',
     default: '8 seconds'
   }
   });
   
   // conf.loadFile('./config.json');
   conf.validate();
   
   console.log(conf.get('session'));
   
   setTimeout(function () {
   console.log('timeout called after %s milliseconds', conf.get('session'));
   }, conf.get('session'));

2. Save and run. Output should look like:

   $ node index
   8000
   timeout called after 8000 milliseconds # <--- This will appear after ~8 seconds.

3. Add a ./config.json file and paste in the following:

   {
   "session": "19 seconds"
   }

4. Uncomment the // conf.loadFile('./config.json'); line in the ./index.js file.

5. Save and run. Output should look like:

   $ node index
   19 seconds
   timeout called after 19 seconds milliseconds # <--- This is invoked immediately and says "19 seconds" instead of 19000.

Actual results:

timeout called after 19 seconds milliseconds

Expected results:

timeout called after 19000 milliseconds

I use convict only for validation and use env variables, but I can't create schema without default values.

Re: https://nodesecurity.io/advisories/validator_XSS_Filter_Bypass_via_Encoded_URL

We should update to validator@2.0.0 or later.

Hello there

I think your module is probably one of the best out there dealing with configs!

But what I'm missing is the possibility to define the schema outside, in a separate json file. So instead of a schema object something like this for the convict constructor would be cool:
var conf = convict('/path/to/schema.json');

Because I think a schema file is a "data file" and data normally does not belong to program code ...

@zaach @vladikoff could you publish [email protected] on NPM please?

It has been tagged with 11e4e43

Thanks a lot

See https://npmjs.org/doc/files/package.json.html#dependencies

Ideally you should use "~x.y.z", so your users can benefit from bugfix releases (z) in the dependencies.

Steps to reproduce

1. Create an index.js file and paste the following code:

   var convict = require('convict');
   
   var conf = convict({
     session: {
       doc: 'a doc',
       format: 'duration',
       default: '2 days'
     }
   });
   
   conf.loadFile('./config.json');
   conf.validate();
   
   console.log(conf.get('session'));

2. Create a config.json file in the same directory as index.js created in step 1:

   {
     "session": "4 days"
   }

3. Run node index

Actual results:

$ node index

/Users/pdehaan/dev/tmp/convict-test/node_modules/convict/lib/convict.js:393
        throw new Error(errBuf);
              ^
Error: session: must be a positive integer: value was "2 days"
    at Object.rv.validate (/Users/pdehaan/dev/tmp/convict-test/node_modules/convict/lib/convict.js:393:15)
    at Object.<anonymous> (/Users/pdehaan/dev/tmp/convict-test/index.js:12:6)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:906:3

This seems to be because of the "2 days" in the config.json file. It works fine when I set the "default" value in the index.js but implodes horribly in the JSON file.

Workaround:

Set the duration as milliseconds in the config.json file.

When using strict mode, I get troubles with unknowns parameters. I am thinking that it may be because I am using a nested tree and I am completing it with parameters from a json file.

        throw new Error(errBuf);
              ^
Error: configuration param '0' not declared in the schema
configuration param '1' not declared in the schema
configuration param '2' not declared in the schema
configuration param '3' not declared in the schema
...

This is while I am using ES6 notation.

When I try with a really simple test on ES5, I cannot get farther than:

  Object.keys(instance).reduce(function(previousErrors, name) {
         ^
TypeError: Object.keys called on non-object
    at Function.keys (native)
    at validate ([...]/test/node_modules/convict/lib/convict.js:64:10)

Here the sample in ES5:
config.js

var convict = require('convict');

var schema = {
    env: {
        doc: 'The application environment.',
        format: ['production', 'development', 'test'],
        default: 'development',
        env: 'NODE_ENV'
    },
    first: {
        second: {
            third: {
                doc: 'Rest API backend URL',
                format: 'url'
            }
        }
    }
};

var conf = convict(schema);
var env = conf.get('env');
conf.loadFile(__dirname + '/config/' + env + '.json');
conf.validate({strict:true});

developement.json

{
  "first": {
    "second": {
      "third": "http://foo.com/"
    }
  }
}

Notice that w/ or w/0 the strict mode enabled, it does not work with a nested tree in ES5. In ES6 it does work only with strict mode disabled.

I do not think having forgotten something because it does work while I am not configuring values for nested parameters.

There is currently an Array format, but you can't restrict the values in the array to a certain type.

I was thinking of just wrapping the format property in an array to signify that it should be a list with elements of only that type.

E.g.:

var config = convict({
  foo: {
    default: ["5 seconds", "10 seconds"],
    format: ["duration"]
  }
});

One concern is that it looks similar to the enum format, although enums shouldn't be allowed to have a single element. An alternative would be to add a list: true to the schema.

I didn't see this anywhere in the documentation, but is there a way to have nested config variables in the config schema definition? For example:

var config = require('convict')({
  loggly: {
    token: {
      doc: '',
      format: String,
      env: 'LOGGLY_TOKEN'
    },
    subdomain: {
      doc: '',
      format: String,
      env: 'LOGGLY_SUBDOMAIN'
    }
  }
});

edit: Or is there another way to go about this completely? My only requirement would be to not store the information in a file. I know I can do nested config there, but I want the schema functionality to get the config variables from environment variables.

We may want to use something like shrinkwrap ($ npm shrinkwrap --dev) to lock down the dependency versions. Plus, it'd make it easier to integrate tooling with tools like nsp which checks for known vulnerabilities against the https://nodesecurity.io/advisories database.

Cons: we'd need to rerun npm shrinkwrap --dev each time we change the module versions in package.json and commit the npm-shrinkrwap.json file into the repo.

Current nsp output:

$ npm shrinkwrap --dev
wrote npm-shrinkwrap.json

$ nsp shrinkwrap
No vulnerable modules found

Just started playing with this, and the first thing I saw missing in types was/is filename. Specifically the filename of the cert/key that are used for tsl settings in https server.

We could allow authors to add their own format.

Something like:

convict.addFormat({
  name: 'mytype',
  format: function(val) { ... },
  convert: function(string) {
    // convert a string to a proper value
    // allows formats such as duration to accept human readable string values
  }
});

I'm trying to achieve this:

    const schema = {
      env: {
        doc: 'The application environment.',
        format: ['production', 'development', 'test'],
        default: 'development',
        env: 'NODE_ENV',
        arg: 'env'
      },
      configPath: {
        doc: 'Path to the configuration files.',
        env: 'NP_CONFIG_PATH',
        arg: 'config-path',
        format: 'path',
        default: '${local}/config'
      },
      include: {
        doc: 'Any other paths to include.\n' +
        'placeholders: (case sensitive)\n' +
        '  ${env} the NODE_ENV variable or the node-env argument.\n' +
        '  ${configPath} is the path to the configuration files.\n' +
        '  ${root} is the root path to this NodePress installation.\n' +
        '  ${local} is the path to the local directory of this NodePress installation.\n'
        ,
        env: 'NP_CONFIG',
        arg: 'config',
        format: Array,
        default: ['${configPath}/${env}.json']
      },
      ....

The 'path' custom format looks like this:

    convict.addFormat({
      name: 'path',
      validate() {
      },
      coerce: function(val) {
        console.log(this);
        return val.replace(/\$\{(\w+)}/g, (v,m) => this.get(m));
      }
    });

This doesn't work inside coerce because it gets passed the format object itself instead of the config object.

Is there a workaround?

It looks like the ChangeLog file hasn't been updated since 0.4.1 and the Releases page hasn't been updated since 0.5.1.

It makes it a bit tricky to see what's changed between versions unless you compare 2 different SHAs.

Not sure if there is an easy Git command that you can use to generate a passable looking changelog so you don't need to do it manually, or bring in Grunt or something.

In the validation section of the README.md, it uses validator.check, which disappeared from validator a long time ago. I've created a PR to remove that and show a valid example of a custom format function.

What is the meaning of the rv variable in convict.js code?

Could we rename rv into something more meaningful?

This rv name has always puzzled me and makes me uncomfortable with one contribution I would like to make.

@lloyd, being the author of this line, could you help on this please? Thank you!

Apparently, there's a memory leak on the 2.3.1 version of moment.js (the one defined in convict's package.json). You can check it by including either one of the packages in some project and run a test using something like Lab (which also checks for memory leaks).

I would have sent a pull request, but the fix is trivial (update to the latest version of moment.js). However, I'm glad to do it if needed.

Can you guys fix this?

When using a command line argument, a custom format function, and a null default, convict attempts to run JSON.parse on the argument.

This is happening in convict 1.0.2, but the code seems to be the same in master.

The issue seems to originate in the getFormat function:

function getFormat(schema, path) {
  var o = traverseSchema(schema, path);
  return o ? (typeof o.format === 'string' ? o.format : typeof o.default) : null;
} 

In the return conditional, o.format is a function and o.default is null, resulting in (typeof null) == "object".
This causes the coerce function to try and JSON.parse the content.

This conflicts with the FAQ, stating: "However, you can set a default value to null and if your format doesn't accept null it will throw an error." (#29)

I am currently working around this by setting default to an empty string, as my use case is to check for certain URL formats and I can validate against it.

Example:

var schema = {
  arg: "amqp-endpoint",
  format: function validate(value) {
    // Custom logic here
    if (value == null) throw new Error("amqp endpoint must be specified");
    return true;
  },

  // default is set to null as per FAQ
  default: null
}

node foo.js -- --amqp-endpoint=amqp://localhost
Error: Unexpected token a

Hi,

I'm using bunyan and I would like to use node-convict to handle my configuration.

...
bunyan: {
    doc: 'The bunyan log configuration.',
    format: Object,
    default: {
      name: 'app',
      stream: process.stdout,
      level: 'info'
    }
}
...

The problem I encountered was because of the deep copy on 'get' function (line 303 of lib/convict.js).

If the object returned was the one defined on the configuration file a stream or any object with circular dependency could be used.

Another approach could be having a raw property to indicate that a deep copy wasn't necessary.

If this module shouldn't be used this way please let me know.

So i compared the configuration schema for convict to a normal database schema. A table can only have columns that are declared in the schema and no additional ones.

I think convict should have a strict validation mode, in which only those properties should be allowed to be declared in different config files(staging,dev,prod) which are declared in the schema.

I have added a pull request doing just that. @zaach, Please review.

I'd love to be able to tell if a property has been changed from its default, either through something like:

if (conf.modified('foo')) { /* ... */ }

or

if (conf.get('foo') !== conf.default('foo')) { /* ... */ }

I'm pretty sure tests are not needed in production. :)
They just bloat project dependencies, install time, use bandwidth, etc.

You could either use the package.json files property to whitelist files for production or .npmignore to blacklist unneeded ones.

For more info on .npmignore and an automatic generator see https://github.com/inikulin/dmn

Because of: https://github.com/mozilla/node-convict/blob/master/lib/convict.js#L379-L381

This behavior is counterintuitive, since if you execute conf.load(some object), the result does not necessarily include the values you supplied. There is no other method to allow you to load an object literal or multiple keys in, with precedence over environment vars / arguments. I'm not sure what the desired solution would be, but there should be some way to do this without having to loop and use .set...

I've seen this kind of error really often.

The configuration file being loaded:

// config/default.json
{
  "parse": {
      "account": {
          "email": "Set up your Parse account's email in XYZ.config.",
          "password": "Set up your Parse account's password in XYZ.config."
      }
  }
}

The code loading it

    var conf = convict({
        parse: {
            account: {
                email: {
                    doc: "Parse account email address.",
                    format: "email",
                    env: "PARSE_EMAIL",
                },
                password: {
                    doc: "Parse account password.",
                    format: 'string',
                    env: "PARSE_PASSWORD"
                }
            }
        },
    }).loadFile('config/default.json').validate({strict:true});

The error trace:

Error: configuration param '0' not declared in the schema
        configuration param '1' not declared in the schema
        configuration param '2' not declared in the schema
        configuration param '3' not declared in the schema
        configuration param '4' not declared in the schema
        configuration param '5' not declared in the schema
        configuration param '6' not declared in the schema

These are opaque and completely unhelpful. What's going on?

I want my app to have a single config.json file - but in this file there may be other properties that the enduser can customise (ie I have no idea what they may be called, and may be added outside of the application)

Can I use convict in this manner ? The conf.load() seems to load the json file without errors, but if I try and get a property that is not in the schema, it fails with the message "Error: cannot find configuration param "

thanks

I'm writing a small framework that will make use of plugins. I want plugins to be able to add to the config schema when they are loaded.

However before the plugins load i need to load the config so i can start up the server. Is there any way for me to add to the schema dynamically after it has been loaded? The only way i can see to do it is by storing the schema object, adding to it and then reload the whole schema after each plugin is loaded.

Is there any other way?

In #64 it looks like we added a strict option when validating a convict schema.
It doesn't look like we added any documentation to the README telling people how to use the awesome new feature.

Hey,

Perhaps I'm misreading the docs, but there doesn't appear to be a way to define a configuration property as "required" but without a default value i.e. if the property is not found in the configuration file, then validation will fail.

This would be especially useful when working with a team of developers. As it stands, a developer could delete a config property without understanding the impact; the application might appear to function, falling back to its default value as defined in the schema, but with consequences further down the line.

If this seems useful, I can try to create a pull request.

Would love to see mongodb://.. URI style to validate against 'url' format, as featured in:
http://blog.nodejitsu.com/npmawesome-managing-app-configuration-with-convict/

From what i can see, this might require to use:

    assert(validator.isURL(mongouri, {require_valid_protocol: false}));

instead of using it without the options object.

A custom validator function works fine this way:

     mongo: {
      doc: 'main database',
      bind: {
        format: function check (url) {
          if (!validator.isURL(url,{require_valid_protocol: false})) {
            throw new Error('must be a URL like');
          }
        },
        default: 'mongodb://localhost:27017/'+my_db_name,
        env: "DB_BIND"
      },

Hi.

I'm trying to use your package in a browserify setup, but because conf.loadFile('some.json') calls various fs methods (fs.statSync, fs.readdirSync), it fails in the browser.

→ Is it something that sounds feasible with convict (or maybe with a browserify transform?), or do you suggest going for another package / custom solution?

Thanks for the help, and sorry for posting this as an issue, it's more of a question to which "we do not and will not support that" can be a fine answer.

Re: #119 (comment)

we should really update the changelog one day https://github.com/mozilla/node-convict/blob/master/ChangeLog

We should probably just try and find a module that does a pretty changelog for us. It doesn't look like we're using Grunt or Gulp here, so adding that may be too much overhead. But it'd be nice to add auto-changelog generation on an npm prepublish hook or something.

I've experimented that the prefedence for args parsing is (from low to hight)
Default -> File -> environment args -> command line -> api

Default is overwritten by File, File by Environment args...

I.e, in this scenario

var convict = require('convict');

var conf = convict({
  ip: {
    doc: "The IP address to bind.",
    format: "ipaddress",
    default: "127.0.0.1",
    env: "IP_ADDRESS",
    arg: "ip"
  }
});
conf.loadFile('./config.json');

conf.set('ip', '134.234.32.56'); //or conf.load({ip:'134.234.32.56'});
conf.validate();

console.log(conf.get('ip'))

config.json

{
 "ip": "192.168.1.1"
}

$ IP_ADDRESS=10.95.78.123 node server.js --ip 0.0.0.0
# Output: 134.234.32.56
# Precedence: Default(127.0.0.1) -> File (192.168.1.1) -> Env(10.95.78.123) -> Arg(0.0.0.0) -> API (134.234.32.56)

Is this correct?
Maybe would you add this to the FAQ?

This is a problem since I'd like to load two configuration having the same schema:

var convict = require("convict");
var schema = {
  publicServerAddress:  {
    doc: 'The public-facing server address',
    format: String,
    default: 'localhost:5000'
  }
};
var conf = convict(schema);
var conf2 = convict(schema);

The second convict is throwing this error:

Error: 'publicServerAddress' uses an unknown format type: string

This is because I have a default, I don't get the error when there is no default.
We should make sure that convict doesn't modify the schema object.

Check this out. This is the contents of test.js:

var convict = require('convict');

var config = convict({
  foo: {
    doc: 'This is an optional option.',
    env: 'FOO',
    format: '*',
    default: undefined
  }
});

config.validate();

console.log(config.get('foo'));

This is what I get when I run it:

$ node test.js
{ doc: 'This is an optional option.', env: 'FOO', format: '*' }

Especially the fact that Convict uses Orderly's syntax...

I wracked my brain trying to figure out what it meant. It'd help with a message Cannot add additional property "foo", not defined in schema or something.

node-convict @ 1.1.2

Error thrown

TypeError: Cannot read property 'get' of undefined
    at /home/steven/code/restau/config.js:11:72
    at String.replace (native)
    at Object.convict.addFormat.coerce [as placeholder] (/home/steven/code/restau/config.js:11:16)
    at coerce (/home/steven/code/restau/node_modules/convict/lib/convict.js:273:32)
    at /home/steven/code/restau/node_modules/convict/lib/convict.js:238:15
    at Array.forEach (native)
    at overlay (/home/steven/code/restau/node_modules/convict/lib/convict.js:235:21)
    at Object.rv.load (/home/steven/code/restau/node_modules/convict/lib/convict.js:378:7)
    at Object.<anonymous> (/home/steven/code/restau/config.js:42:6)
    at Module._compile (module.js:398:26)

Cause

• coerce function requires a convict instance in the fourth param
• overlay function call coerce without convict instance because overylay has no reference to one
• load and loadFile functions don't give the convict instance reference to overlay (378 and 388)

How to reproduce it

'use strict';

const convict = require('convict');

convict.addFormat({
  name: "placeholder",
  validate: function(val) {
    /* validate proper path here */
  },
  coerce: function(val, config) {
    return val.replace(/\$\{([\w\.]+)}/g, function(v,m) { return config.get(m); }); /* line 11 */
  }
});

var conf = convict({
  env: {
    format: ['production', 'development'],
    default: 'development',
    env: 'NODE_ENV',
    doc: 'The environment that we\'re running in.'
  },
  configPath: {
    format: 'placeholder',
    default: '/path/to/config',
    doc: 'Path to configuration files.'
  },
  config: {
    format: 'placeholder',
    default: '${configPath}/${env}.json',
    doc: 'Path to configuration file.'
  }
});

conf.get('config');
// return "/path/to/config/development.json"

conf.set('config', '${configPath}/foo.json');
conf.get('config');
// return "/path/to/config/foo.json"

conf.load({ config: '${configPath}/bar.json' }); /* line 42 */
// throw "TypeError: Cannot read property 'get' of undefined"

How to fix it

Transport the convict instance reference from load and loadFile to coerce with adding it in overlay last param.

Pull request

I will make a PR in few minutes to fix that issue.

I don't have time right this minute but could put something together by picking the bones of @zaach's blog post...

https://github.com/mozilla/node-convict/blob/master/lib/convict.js#L71

This line doesn't return previousErrors, causing an unintended throw when previousErrors.push is called here:

https://github.com/mozilla/node-convict/blob/master/lib/convict.js#L80

I'm preparing a PR for a fix, but i actually spent 10 times as long trying to write a test due to some peculiar circumstances. I have a test that will cause the error and succeed when it's fixed; I'm uncertain why one aspect of the test behaves a certain way, so I'm opening this issue so perhaps someone else can run that down further.

The test in question is:

  it('must not break when a failed validation follows an undeclared property', function() {
    (function() {
      var config = convict({
        test2: {
          one: { default: 0 },
          two: { default: 0 }
        }
      });

      // if this key is a number, the error occurs; if it is a string, it does not
      // i don't know why. the deep nesting is also required.
      config.load({'0': true});
      config.load({ test2: { two: 'two' } });
      config.validate();
    }).must.throw(/should be of type Number/);
  });

I don't know why whether the key of a loaded configuration is numeric should matter, but it appears to alter the code flow and may be undesirable. PR to follow shortly.

Using v1.0.2

I am defining an array of regexes like this:

var convict = require('convict');

var config = convict({
  authorizedReferrers: {
    doc: 'Authorized HTTP referrers that can access this web service',
    format: Array,
    default: [
      /^http:\/\/www.domain1.com\/.*/,
      /^http:\/\/www.domain2.com\/.*/
    ],
    env: 'DATA_API_AUTHORIZED_REFERRERS'
  }
});

module.exports = config;

Later, in another file trying to retrieve this array of regexes, I get an array of empty objects instead of regexes:

config.get('authorizedReferrers') // <-- Returns empty an array of empty objects [ {}, {} ]

Works fine if I replace regexes with strings. Am I missing something?