Giter VIP home page Giter VIP logo

envalid's Introduction

Envalid text logo with drop shadow

Envalid is a small library for validating and accessing
environment variables in Node.js programs

Current GitHub Build Status Badge

Envalid is a small library for validating and accessing environment variables in Node.js programs, aiming to:

  • Ensure that your program only runs when all of its environment dependencies are met
  • Give you executable documentation about the environment your program expects to run in
  • Give you an immutable API for your environment variables, so they don't change from under you while the program is running

Why Envalid?

  • Type-safe: written completely in TypeScript, with great support for inference
  • Light: no dependencies besides tslib
  • Modular: customize behavior with custom validators, middleware, and reporters

API

envalid.cleanEnv(environment, validators, options)

cleanEnv() returns a sanitized, immutable environment object, and accepts three positional arguments:

  • environment - An object containing your env vars (eg. process.env)
  • validators - An object that specifies the format of required vars.
  • options - An (optional) object, which supports the following key:
    • reporter - Pass in a function to override the default error handling and console output. See src/reporter.ts for the default implementation.

By default, cleanEnv() will log an error message and exit (in Node) or throw (in browser) if any required env vars are missing or invalid. You can override this behavior by writing your own reporter.

import { cleanEnv, str, email, json } from 'envalid'

const env = cleanEnv(process.env, {
  API_KEY: str(),
  ADMIN_EMAIL: email({ default: '[email protected]' }),
  EMAIL_CONFIG_JSON: json({ desc: 'Additional email parameters' }),
  NODE_ENV: str({ choices: ['development', 'test', 'production', 'staging'] }),
})

// Read an environment variable, which is validated and cleaned during
// and/or filtering that you specified with cleanEnv().
env.ADMIN_EMAIL // -> '[email protected]'

// Envalid checks for NODE_ENV automatically, and provides the following
// shortcut (boolean) properties for checking its value:
env.isProduction // true if NODE_ENV === 'production'
env.isTest // true if NODE_ENV === 'test'
env.isDev // true if NODE_ENV === 'development'

For an example you can play with, clone this repo and see the example/ directory.

git clone https://github.com/af/envalid
cd envalid
yarn prepare
node example/server.js

Validator types

Node's process.env only stores strings, but sometimes you want to retrieve other types (booleans, numbers), or validate that an env var is in a specific format (JSON, URL, email address). To these ends, the following validation functions are available:

  • str() - Passes string values through, will ensure a value is present unless a default value is given. Note that an empty string is considered a valid value - if this is undesirable you can easily create your own validator (see below)
  • bool() - Parses env var strings "1", "0", "true", "false", "t", "f" into booleans
  • num() - Parses an env var (eg. "42", "0.23", "1e5") into a Number
  • email() - Ensures an env var is an email address
  • host() - Ensures an env var is either a domain name or an ip address (v4 or v6)
  • port() - Ensures an env var is a TCP port (1-65535)
  • url() - Ensures an env var is a URL with a protocol and hostname
  • json() - Parses an env var with JSON.parse

Each validation function accepts an (optional) object with the following attributes:

  • choices - An Array that lists the admissible parsed values for the env var.
  • default - A fallback value, which will be present in the output if the env var wasn't specified. Providing a default effectively makes the env var optional. Note that default values are not passed through validation logic, they are default output values.
  • devDefault - A fallback value to use only when NODE_ENV is explicitly set and not 'production'. This is handy for env vars that are required for production environments, but optional for development and testing.
  • desc - A string that describes the env var.
  • example - An example value for the env var.
  • docs - A URL that leads to more detailed documentation about the env var.

Custom validators

Basic usage

You can easily create your own validator functions with envalid.makeValidator(). It takes a function as its only parameter, and should either return a cleaned value, or throw if the input is unacceptable:

import { makeValidator, cleanEnv } from 'envalid'
const twochars = makeValidator((x) => {
  if (/^[A-Za-z]{2}$/.test(x)) return x.toUpperCase()
  else throw new Error('Expected two letters')
})

const env = cleanEnv(process.env, {
  INITIALS: twochars(),
})

TypeScript users

You can use either one of makeValidator, makeExactValidator and makeStructuredValidator depending on your use case.

makeValidator<BaseT>

This validator has the output narrowed down to a subtype of BaseT (e.g. str). Example of a custom integer validator:

const int = makeValidator<number>((input: string) => {
  const coerced = parseInt(input, 10)
  if (Number.isNaN(coerced)) throw new EnvError(`Invalid integer input: "${input}"`)
  return coerced
})
const MAX_RETRIES = int({ choices: [1, 2, 3, 4] })
// Narrows down output type to '1 | 2 | 3 | 4' which is a subtype of 'number'

makeExactValidator<T>

This validator has the output widened to T (e.g. bool). To understand the difference with makeValidator, let's use it in the same scenario:

const int = makeExactValidator<number>((input: string) => {
  const coerced = parseInt(input, 10)
  if (Number.isNaN(coerced)) throw new EnvError(`Invalid integer input: "${input}"`)
  return coerced
})
const MAX_RETRIES = int({ choices: [1, 2, 3, 4] })
// Output type is 'number'

As you can see in this instance, the output type is exactly number, the parameter type of makeExactValidator. Also note that here, int is not parametrizable.

Error Reporting

By default, if any required environment variables are missing or have invalid values, Envalid will log a message and call process.exit(1). You can override this behavior by passing in your own function as options.reporter. For example:

const env = cleanEnv(process.env, myValidators, {
  reporter: ({ errors, env }) => {
    emailSiteAdmins('Invalid env vars: ' + Object.keys(errors))
  },
})

Additionally, Envalid exposes EnvError and EnvMissingError, which can be checked in case specific error handling is desired:

const env = cleanEnv(process.env, myValidators, {
    reporter: ({ errors, env }) => {
        for (const [envVar, err] of Object.entries(errors)) {
            if (err instanceof envalid.EnvError) {
                ...
            } else if (err instanceof envalid.EnvMissingError) {
                ...
            } else {
                ...
            }
        }
    }
})

Custom Middleware (advanced)

In addition to cleanEnv(), as of v7 there is a new customCleanEnv() function, which allows you to completely replace the processing that Envalid applies after applying validations. You can use this custom escape hatch to transform the output however you wish.

envalid.customCleanEnv(environment, validators, applyMiddleware, options)

customCleanEnv() uses the same API as cleanEnv(), but with an additional applyMiddleware argument required in the third position:

  • applyMiddleware - A function that can modify the env object after it's validated and cleaned. Envalid ships (and exports) its own default middleware (see src/middleware.ts), which you can mix and match with your own custom logic to get the behavior you desire.

Utils

testOnly

The testOnly helper function is available for setting a default value for an env var only when NODE_ENV=test. It is recommended to use this function along with devDefault. For example:

const env = cleanEnv(process.env, {
  SOME_VAR: envalid.str({ devDefault: testOnly('myTestValue') }),
})

For more context see this issue.

FAQ

Can I call structuredClone() on Envalid's validated output?

Since by default Envalid's output is wrapped in a Proxy, structuredClone will not work on it. See #177.

Related projects

  • dotenv is a very handy tool for loading env vars from .env files. It was previously used as a dependency of Envalid. To use them together, simply call require('dotenv').config() before you pass process.env to your envalid.cleanEnv().

  • react-native-config can be useful for React Native projects for reading env vars from a .env file

  • fastify-envalid is a wrapper for using Envalid within Fastify

  • nestjs-envalid is a wrapper for using Envalid with NestJS

  • nuxt-envalid is a wrapper for using Envalid with NuxtJS

Motivation

http://www.12factor.net/config

envalid's People

Contributors

af avatar barbogast avatar beaulac avatar dependabot[bot] avatar duc-dangvu avatar dylanvann avatar ibratoev avatar intelagense avatar jazmon avatar jgravois avatar jlsjonas avatar jsamr avatar kachkaev avatar lostfictions avatar louis-vinchon avatar manuelhenke avatar maxprilutskiy avatar n1ru4l avatar neezer avatar novemberborn avatar rafamel avatar sammurphydev avatar saramorillon avatar simenandre avatar simenb avatar simonua avatar sjoerdsmink avatar theneva avatar vunovati avatar witq avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

envalid's Issues

Unknown Property Names Should Not Throw Errors In Strict Mode

I can understand the intention in #43 for preventing typos, but I do not think throwing errors is the correct solution.

For example, this small example fails:

import _ from 'lodash'
import envalid from 'envalid'

const config = envalid.cleanEnv(process.env, {
  NODE_ENV: envalid.str({default: 'development'})
}, {
  strict: true
})
console.log(_.size(config))

Expected: 1

Actual: ReferenceError: [envalid] Env var not found: length

See: Error thrown on this line

See: Lodash's size implementation

Feature request: Safe getter.

While envalid is great for validating and parsing the variables from the environment, It's still possible to be affected by typos:

const env = cleanEnv(process.env, {HANDKERCHIEF: str()});

env.HANKERCHIEF; // Oops, misspelled.

It would be great to have a safeguard against this type of problem built into the library.

The impementation can be as trivial as:

function get(key) {

    if (env.hasOwnProperty[key]) {
        return env[key];
    }

    throw new Error('Invalid config key: ' + key);
}

Also possible are more fancy get functions. I have used something like this in the past:

[email protected]
AUTH_EMAILS_CHARSET=utf-8
AUTH_TOKENEXPSECONDS=3600
env.get('auth.emails.from') // key is transformed to uppercase and . -> _ before check.

Build latest code base and release to NPM

Hi there,

Can we get another release of this library, probably only needs to be a patch version by looking at the commits.

In particular I need #114 to be implemented (I do have a workaround, but as I develop for commercial clients, to maintain code sustainability I would prefer this to be available as part of the library).

Thanks

different defaults for test and development

Hi, bringing up the similar question to #32

The problem I'm trying to solve is to use different URI for mongo in tests because I'm dropping the entire collection after each test.

So what I want to achieve is something like that:

ENV_MONGO_URI: str({ devDefault: 'mongodb://mongo:27017/app', testDefault: 'mongodb://mongo:27017/app-test' })

If I understand correctly I can't use testOnly for that case since in development my uri will be undefined.

Thanks in advance!

Conditional required env?

Is there a way to conditionally mark ENV vars as required based on the values of other ENV vars?

In my case, I have the following 4 vars:

  • USE_TLS
  • TLS_CA_CERT_PATH
  • TLS_CERT_PATH
  • TLS_KEY_PATH

If USE_TLS is true, then I want the other three flagged as required. If it is false, then the other three should be optional.

Are all dependencies needed?

I'm looking into using this on the front-end as opposed to the env-var lib due to the ability to override the container (process.env). But this is almost 20x the size. Bundlephobia shows this breakdown: https://bundlephobia.com/[email protected]

Or maybe a better question would be is if this lib is suited for client-side?

Strict mode: suggest solutions when throwing errors

  • If an env var was accessed that's not in the schema, but is in process.env, we can suggest adding a validator for that var

  • If an accessed env var was not in the validated object, but a similar one was (ie. the error was likely a typo), we can suggest the similar env var name. We don't have to anything fancy like calculating the levenshtein distance; for now could just check if:

    1. The length of name A is within 1 of name B
    2. The first N chars are the same between A and B

Typescript type declarations

Adding Typescript type declarations will make the package easier to integrate in Typescript projects.

I am currently using it with Typescript and I can contribute the typings. @af, are you interested in a PR?

Migrate codebase to TypeScript

I was a little slow jumping on the TS bandwagon but I'm a full convert now. I would like to convert the full codebase to TS sometime in the near future, but haven't had the time to do so yet. Creating this as a tracking issue and advertising the intent, in case someone wants to do this before I get around to it :)

testDefault?

devDefault - A fallback value to use only when NODE_ENV is not 'production'. This is handy for env vars that are required for production environments, but optional for development and testing.

I'm using envalid in a server setup which needs to interact with AWS resources. I have a use case where a variable is required during development, but not during testing. I could set devDefault but that'll just lead to runtime errors. It'd be great if envalid could support a testDefault which only applies when NODE_ENV === 'test'.

New Release?

The latest release on the Github releases tab is 3.0.0, but on npm v3.1.0 has been published.

Is it possible to update and cut a new release?

I am especially interested in the following commit as it was throwing warnings when we used it with the pkg tool:

3a5ce26

Injecting additional values into an existing env

Hi 👋

I have a Node.js project with a few scripts (entrypoints). Some of them require additional parameters like auth tokens etc. and I would like to extend my env object inside some of the actions. This is not the same as #78 because the presence of extra values depends on the code logic rather than the values certain env variables have. An example:

// config.ts
import * as envalid from "envalid";

export const env = envalid.cleanEnv(process.env, {
  SHARED_THING: envalid.str({
    default: "this variable needs to present in all scripts",
  }),
}, { strict: true });
// peculiarTask.ts

import { env } from './config';

const extendedEnv = envalid.cleanEnv(env, {
  EXTRA_THING: envalid.str({
    default: "this variable should only be cleaned inside peculiarTask.ts",
  }),
});

There are a couple of problems at the moment:

  • Using { strict: true } to get extendedEnv clears out all variables that used to exist in env.
  • Even when there is no { strict: true } in the second case, TypeScript autocompletion for extendedEnv.SHARED_THING does not work.

What improvements could we make here? What do others do in similar cases?

Skip process.exit(1) in Browser mode

First of all, thank you for a handy and well crafted tool.

Small issue: the process.exit(1) call throws in a browser because the exit method is not defined.
Maybe skip that call when in browser mode?

Add composition of validators

I have a need of composing two validators, ie the value can satisfy either of validators. My use-case is origin url for cors setup - it can either be an actual url or a string '*' to allow any origin.

Here is my take on this:

// #region `oneOf` definition
type MkValidator<T> = (spec?: Spec<T>) => ValidatorSpec<T>;

// helper function to encapsulate usage of internals
function parse<T>(val: ValidatorSpec<T>, x: string): T {
  const res = val._parse(x);
  if (val.choices) {
    if (val.choices.indexOf(res) === -1) {
      throw new Error(
        `Value "${x}" not in choices: [${val.choices.join(", ")}]`
      );
    } else {
      return res;
    }
  } else {
    return res;
  }
}

// define `oneOf` combinator
const oneOf = <T>(
  val: ValidatorSpec<T>,
  ...vals: ValidatorSpec<T>[]
): MkValidator<T> => {
  const all = [val, ...vals];

  // I went with simpler approach to specify all documentary stuff on resulting validator
  // and reject all these properties on sub validators.
  for (const v of all) {
    if (v.default !== undefined) {
      throw new Error(
        `'default' property is not supported in oneOf sub validators.`
      );
    }
    if (v.devDefault !== undefined) {
      throw new Error(
        `'devDefault' property is not supported in oneOf sub validators.`
      );
    }
    if (v.desc !== undefined) {
      throw new Error(
        `'desc' property is not supported in oneOf sub validators.`
      );
    }
    if (v.docs !== undefined) {
      throw new Error(
        `'docs' property is not supported in oneOf sub validators.`
      );
    }
    if (v.example !== undefined) {
      throw new Error(
        `'example' property is not supported in oneOf sub validators.`
      );
    }
  }

  const types = all.map(v => v.type);
  const type = `oneOf: [${types.join(", ")}]`;

  return makeValidator(x => {
    try {
      return parse(val, x);
    } catch (err) {
      const errs = [err];
      for (const v of vals) {
        try {
          return parse(v, x);
        } catch (e) {
          errs.push(e);
          continue;
        }
      }
      throw new Error(
        `oneOf validator failed: [\n${errs.map(e => e.message).join(",\n")}\n]`
      );
    }
  }, type);
};
// #endregion

And here is the usage:

// creating `origin` validator using `oneOf` combinator
const origin = oneOf(url(), str({ choices: ["*"] }));

export default cleanEnv(
  process.env,
  {
    // example usage of newly created validator
    FRONTEND_ORIGIN: origin({
      devDefault: "*",
      example: "http://localhost:3000"
    })
  }
);

Invalid NODE_ENV message improvement

throw new EnvError(`Value "${value}" not in choices [${spec.choices}]`)

In my company we use envalid but with non-default environment values. Thus, I got my builds failing with the error above.

Currently the function validateVar in envalidWithoutDotenv.js doesn't give a comprehensive error message with potential reasons for this issue and how to fix them. I had to actually go into the source code of envalid to understand the reason why my builds are failing and how to fix it.

My suggestion is to improve the error message and add guides for the user to understand how to fix this issue. Also, I suggest to add a part about NODE_ENV in the documentation that if it's not specified in cleanEnv then it takes the value of defaultNodeEnv from the same file on line 56.

I'd be more than glad to contribute.

Use ^ in package.json for version numbers

I noticed that the dependency versions in package.json are fixed, e.g. "chalk": "2.1.0" instead of "chalk": "^2.1.0". I might be wrong but this may cause duplicates in the front end bundles if the same package is being referred from somewhere else (e.g. as "chalk": "^2.3.0"). Not sure there is any benefit from hard-locking the versions, so how about using ^? This will allow for any library version higher than the given one, but smaller than the next major version.

If there's a need to ensure that the dev dependencies are stable in CI, package-lock.json can be added to the source control. WDYT?

[email protected] introduces breaking change

Good morning @af,

A breaking change was introduced by removing the lib directory and, presumably, replacing it with src. I don't know whether this was an intentional change or an omission in the publish. If this was intentional, 5.0.0 should have been used instead of 4.2.0.

We import validators via
const Validator = require("envalid/lib/validators");
which would now need to be changed to
const Validator = require("envalid/src/validators");

image

Feature request: Allow prefixing env variables.

Would it be possible to specify a prefix when cleaning an environment?

const env = envalid.cleanEnv(process.env, {
    API_KEY:            str(),
    ADMIN_EMAIL:        email({ default: '[email protected]' }),
    EMAIL_CONFIG_JSON:  json({ desc: 'Additional email parameters' })
}, {prefix: 'MY_APP_NAME'});

env.API_KEY; // Returns the value present at real env variable MY_APP_NAME_API_KEY.

My use case is for my dev environment where I run multiple apps and some of the more common env variables clash, less commonly I also have some multi-app deployments.

A current valid solution is to just specify the APP_NAME prefix in all the vars:

const env = envalid.cleanEnv(process.env, {
    MY_APP_NAME_API_KEY:            str(),
    MY_APP_NAME_ADMIN_EMAIL:        email({ default: '[email protected]' }),
    MY_APP_NAME_EMAIL_CONFIG_JSON:  json({ desc: 'Additional email parameters' })
});

env.MY_APP_NAME_EMAIL_CONFIG_JSON;

This works just fine, but it's a little verbose. I could also use a transformer and remove the prefix after all the variables are loaded.

Feature Idea: dependent variables

First of all, thank you for providing such a useful utility <3

In my scenario: I want to validate an optional username and password, but also say that if there is one without the other, the validation should fail. E.g. allow configuration against a resources that allows authenticated and unauthenticated requests. One can opt to add user/pass, or not user/pass, but if the provide user/pass, BOTH have to be there.

This can be solved in a transform function, which is what I’m going with for now, but something like this would be nice:

let config = {
  username: str(dependsOn:['password']),
  password: str(dependsOn: ['username'])
}

This is just to sketch what this could look like. Using an array of depends could open this up to more than two dependent variables. Although having to make the dependency on both sides will explode maintenance complexity quickly.

Alternatively, maybe this can be done with grouping:

let config = {
  group: {
    username: str(dependsOn:['password']),
    password: str(dependsOn: ['username'])
  }
}

where group is an arbitrary label, but the semantics are: if one of the values is missing, validation fails.

Another variant would be explicit grouping:

let config = {
  username: str(),
  password: str()
}

const options = {
  groups: [['username', 'password']]
}
envalid.cleanEnv(process.env, config, options)

Suggestion: Rename 'isDev' to 'isDevelopment'?

In the README:

// Envalid parses NODE_ENV automatically, and provides the following
// shortcut (boolean) properties for checking its value:
env.isProduction    // true if NODE_ENV === 'production'
env.isTest          // true if NODE_ENV === 'test'
env.isDev           // true if NODE_ENV === 'development'

isDev is the only one that doesn't match the value from NODE_ENV.

For consistency I'm suggestion the isDev property to be renamed to isDevelopment. I understand if you are opinionated about this and don't want to change it. It's just a suggestion :)

Reporter received args are not consistent with documentation

Hiya! Thanks for the amazing module!

The documentation states a reporter receives two args, which makes sense to me. In practise however, a reporter receives an object that contains the two named arguments as properties. I went to create a PR but noticed you are consistent and pass an object containing an error object and env object in your test as well. This is probably a doc update, if not, I'm still willing to submit a PR.

In any case, either the module or the documentation should be updated.

Cheers

Default not working anymore

Hey guys! It seems that from the [https://github.com/af/envalid/commit/68f95bd4012b3f3017a67e0f50a2dece1de0ce00](last commit) default is not working anymore... This part of the code haven't exported the variable:

SOME_VAR: envalid.str({ default: '127.0.0.1' })

Generating example .env file

It would be cool if it was possible to generate example .env file from envalid configuration. Would be helpful when someone installs application for the first time and wants to generate .env file (to run app locally).
It could also assign example values to env vars (if they are defined in envalid config).

Set defaults during validation

I like this module, but there's one thing that could be improved.

It would be great to have a defaults key as part of the environment variable specifications that applies a default when the environment variable is undefined. (Clearly this would be exclusive with required: true.)

Sure it's possible to provide a default value when calling envalid.get() but that quickly leads to repetition of the default value if it is referenced in several places.

I thought I could work around this by implementing a parse function which returns a default value if the variable is undefined, until I realized that .get() only works for environment variables that exist, so the parse function never gets called.

"Tag" boolean

What about a "booleanTag" type, which would match value against variable presence only ?

I mean having a simple (VAR != null) (loose null or undefined, roughly means "presence") check that would allow anything, (including 0 or "") as value to be truthy. Acting like a --param tag and simpler booleans.

This shouldn't be mixed with the true boolean type though, IMHO, as they are pretty different

command !!VAR
VAR="" node script.js true
VAR=0 node script.js true
VAR= node script.js true
node script.js false
const myenv = envalid({ 
  ...process.env,
  VAR: undefined
})

// myenv.VAR === false
const myenv = envalid({ 
  ...process.env,
  VAR: null
})

// myenv.VAR === false

Problems with Node 4.3.2

When using the package with node 4.3.2 requiring 'envalid' module produces the following syntax error:

{
  "errorMessage": "Unexpected token {",
  "errorType": "SyntaxError",
  ...
}

I haven't seen any mentions in Readme about this limitation so is this on purpose?

Proposal: enable usage of envalid outside node-js by dropping the dependency on dotenv

Hi,

I like the idea of envalid a lot and would like to use it in a react-native app.

Unfortunately dotenv doesn't work within the app since it tries to import the fs module which isn't available in the app. I was able to solve this by using react-native-config to get the configuration object. Now I would like to use envalid to validate this configuration object. But since envalid imports dotenv that's not possible right now.

One solution would be to entirely remove the dotenv dependency from envalid. So instead of

const env = envalid.cleanEnv(process.env, {...})

users would have to call

const env = envalid.cleanEnv(require('dotenv').config(), {...})

The option dotEnvPath could then be removed.

This change would allow the retrieval of the environment by arbitary libraries (like react-native-config) or even different sources (json-files, databases) without the implicit loading of the environment by dotenv.

Thanks
Ben

Change env: any to env: NodeJS.ProcessEnv

NOTE: This is a continuation to this disucssion.

I propose changing the type of env/environment (1, 2, 3, 4) from any to NodeJS.ProcessEnv.

As I see it, this library exists to do one thing: easily parse external environment variables.

Envalid is a small library for validating and accessing environment variables in Node.js

This library does not aim to be a generic object schema validator.

So I believe the env input object to cleanEnv and associated functions should expect env/environment to act exactly like—or as close as possible to—the actual object that will be passed in: process.env.

  • This change would not affect the tests as they are currently written, since the tests are not using TypeScript.
  • TS developers can work with a more rigidly defined object type.
  • Doesn't affect JS developers—as the code is written right now.

Implementation of this proposal would introduce breaking behavior for TS users who are not already using an argument that satisfies NodeJS.ProcessEnv to cleanEnv and associated functions.

JS users should be unaffected.


I'd be happy to toss up a PR with the proposed change if y'all want to see it in action.

Custom parsers are typed to receive strings, but spec requires default values to match the type

const hex = envalid.makeValidator<Buffer>(value => Buffer.from(value, 'hex'))

envalid.cleanEnv({
  VALUE: 'decafbad',
}, {
  VALUE: hex({ default: 'ff' }), // TypeScript expects a Buffer or undefined
})

The parser is typed as receiving a string value:

parser: (input: string) => any,

However the validator specs infer the validated type and mandate it for the default and devDefault values. That's nice for numbers and booleans but I suspect it'll trip up the json validator. It works for my hex validator above but only because Node.js doesn't throw an error for the unnecessary encoding argument.

Perhaps the default values should always be strings, and be typed as such?

dotenv usage question

Current State

Currently dotenv.config is called inside the cleanEnv method like that:

const env = extend(dotenv.config({ silent: true }), inputEnv)

First, let's check the dotenv.config 2.0 documentation: "Config will read your .env file, parse the contents, and assign it to process.env."

So, what happens is that if you call cleanEnv({ foo: 'bar' }, spec) and there is a .env file available this will change your process.env state. I find this kind of unexpected.

Using this library for a while, I feel that the goal of cleanEnv is to clean and validate the environment that is passed as the first parameter with the spec provided as the second parameter. I don't really see how dotenv.config fits.

How to fix this? Here are my ideas for the next major version.

Proposal

The current cleanEnv method can be renamed to cleanProcessEnv and the first parameter can be removed. Generally, it can have this simple implementation:

dotenv.config(); // This will load the .env file and update `process.env`.
return cleanEnv(process.env, spec); // create a clean env (dotenv.config is called in cleanEnv)

cleanEnv should not call internally dotenv.config().

As an added benefit, this will be compatible with the latest dotenv version.

What do you guys think?

What are the breaking changes with version 4.x?

Hey there,

I see that you've pushed out a new envalid version 4.x. We currently depend on 3.x and wonder what the breaking changes are for that new major version and what we need to do to migrate to that version. So far I can't really find anything and there's also no changelog or release notes on the GitHub release.

Would be super nice if you could clarify, so we can start migrating :)

Thank you so much for all you work so far and have a great day!

Next release

When do you plan to release the next release? Is it going to be 3.0.0?
Consider releasing a preview release 3.0.0-beta.

types showing `string` for json variable

With the latest changes, we seeing types broken for json variable as it gives type as string instead of any.

Here I have REDIS_CLUSTER_URL as json variable, but type shown as string instead.

image

It was broken in 6.0.2

Unable to override NODE_ENV validation

I tried overriding the validation rules for NODE_ENV to support additional types by setting choices as follows:

{
  NODE_ENV: str({default: 'development', choices: ['production', 'development', 'test', 'staging']})
}

but when I actually set NODE_ENV=staging, I get the following error:

{ EnvError: Value "staging" not in choices [development,test,production]

Load envalid declarations from directories/files

The way we have our envalid declarations set up at work is to put them by category in different files in a directory, then load the whole directory, merging all declarations, then passing the result through cleanEnv and making it immutable.

After #44 we can drop our immutable proxy wrapper, but we still need our own code to load all en variables. What do you think about adding that capability directly to envalid?

Example:

../frontpage/server/config
├── assets.js
├── cdn.js
├── messages.js
├── nunjucks.js
├── security.js
├── services.js
├── tracking.js
└── trackjs.js

0 directories, 8 files

Each file is e.g security.js

'use strict';

const { bool } = require('envalid');

module.exports = {
    SECURE_COOKIES: bool({
        desc: 'Enable to set `Secure` flag on all cookies',
        default: true,
        devDefault: false,
    }),
};

You can provide multiple directories, or single files of these, and they are merged.
handling of dupes are undefined (in practice last one wins, but you should avoid overlap). That can be defined, though, if you want that.

extend env result out of `cleanEnv`

Hi! thanks for providing us with this amazing tool.

I'm looking for some guidance when trying to extend the resulting object that comes out of cleanEnv method, particularly because we have staging as an environment and we will like to have the same capabilities that are automatically provided for test, production and development environments. (ex: isTest, isProduction/isProd, isDevelopment/isDev)

Some things we have tried

const env = envalid.cleanEnv(process.env, {
     PORT: num({ default: DEFAULT_PORT }),
     NODE_ENV: str({default: 'development'}),
});

return {...env, isStaging: env.NODE_ENV === 'staging'};

this example will result in loosing all the automatic methods generated like isProduction/isProd, isDevelopment/isDev

const env = envalid.cleanEnv(process.env, {
     PORT: num({ default: DEFAULT_PORT })
});
env.isStaging = env.NODE_ENV === 'staging';

return env;

It will fail because the object is read-only (at least in ts).
I can potentially flip the interface to be able to add a property, but it feels against the lib intent.

Thanks!

Contradictory behaviour when NODE_ENV is undefined and no validation rule set

If you define validation for NODE_ENV and it is undefined then due to [1] isProduction will return true and isDev returns false.

However due to [2] dev defaults will be used. This behaviour is contradictory and confusing.

I'd submit a PR but I'm not 100% on what the fix would be, I think possibly that:

  • actual node env should be calculated early on (including using default if not set)
  • actual node env should then be used through out
  • actual node env should be added to output

[1] https://github.com/af/envalid/blob/master/index.js#L79
[2] https://github.com/af/envalid/blob/master/index.js#L89

More standard validators to cover common use cases

Hi @af and @SimenB! I've been using your lib in a few node apps and it's been a pleasure!

Just noticed myself copying a couple of custom validators between projects and decided to ask what you think about adding more to the standard set. In my case, port and host would be useful (a port is an integer from 1 to 65535 and host is either a domain name or ipv4/ipv6). The implementation is quite straightforward, but is slightly more specific than num / str.

What do you think of adding these two and potentially many others in future?

jest test crashes when we have envalid

Not sure if this issue needs to be posted here as the issue happens only when we run jest tests. Wondering if anyone has this issue before?

cmd: 'cross-env NODE_ENV=test ENV_FILE=config/test/test.env jest "service"'


Error

13 verbose stack Error:  test: `cross-env NODE_ENV=test ENV_FILE=config/test/test.env jest "service"`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at EventEmitter.emit (events.js:214:7)
13 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at ChildProcess.emit (events.js:214:7)
13 verbose stack     at maybeClose (internal/child_process.js:915:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)

Versions:

Darwin 15.6.0
node v8.12.0
npm v6.4.1

Unable to set default value for a JSON env variable

I'm getting this error:

================================
 Invalid environment variables:
    POSTMARK_TEMPLATES_ID: Invalid json: "[object Object]"
    POSTMARK_TEMPLATE_VALUES: Invalid json: "[object Object]"

 Exiting with error code 1
================================

And here's my code:

// config/index.js

const envalid = require('envalid');

const { str, num, json } = envalid;

const env = envalid.cleanEnv(
  process.env,
  {
    APP_MODE: str({ default: 'dev' }),
    POSTMARK_TEMPLATES_ID: json({
      default: {
        VERIFY_EMAIL: 1,
        FORGOT_PASSWORD: 2,
        PASSWORD_RESETED: 3,
        PASSWORD_CHANGED: 4,
        EMAIL_CHANGED: 5,
      },
    }),
    POSTMARK_TEMPLATE_VALUES: json({
      default: {
        product_name: 'ACME Software v1.0',
        product_url: 'https://domain.io',
        support_url: 'https://support.domain.io',
        company_name: 'ACME Inc.',
        company_address: null,
      },
    }),
  },
  {
    dotEnvPath: '.env',
  },
);

// eslint-disable-next-line import/no-dynamic-require
const modeConfig = require(`./${env.APP_MODE}`);

module.exports = Object.freeze({
  ...env,
  ...modeConfig,
});

I'm not sure at all if I'm doing something wrong, but I believe it is a bug.

Asynchronous validation

It would be nice to support asynchronous validation via callbacks. This would unlock some use cases that aren't possible with synchronous validation. Examples:

  • Validate a hostname using a DNS lookup. This is a much stronger validation than can be provided with just regular expressions.
  • Test network connectivity to supporting services in the launch phase. A process could choose to exit if some dependencies are unavailable. Then the retry logic can be taken care of by a supervisor process like runit or upstart.

I have some code that's already doing this, but I'm not sure if this matches your goals for the project. Maybe it would work better as a separate module. Let me know what you think!

Feature idea: `example` and `docs` fields for validators

I find that my descriptions for env vars often contain example values, or links to web pages that contain more context. For example:

SENTRY_DSN: str({ desc: 'DSN for sentry. eg. "my.fake.value" See http://foo.com/bar' })

Maybe it makes sense to create new fields so these semantics can be captured and dealt with more intelligently by the formatter(s). So that previous example could be:

SENTRY_DSN: str({ desc: 'DSN for sentry', example: 'my.fake.value', reference: 'http://foo.com/bar' })

Do you think this would be useful? cc @SimenB

Envalid behaviour for str() validator changed from v2.4.1 to v2.4.2

Hey,

Our test started failing when upgraded to v2.4.2. We check if envalid reports an error when a variable is set to an empty string when the validator is str(). Version 2.4.1 does raise the error, version v2.4.2 doesn't anymore. I'm suspecting this line to be the culprit: 5923489...v2.4.2#diff-168726dbe96b3ce427e7fedce31bb0bcR63

No biggie but it is a behaviour change you wouldn't expect in a minor version change.

I can contribute a test if necessary, but I can't make a decision on what the desired behaviour is.

Cheers!

Suggest making strict mode more configurable

There are some libraries that check if an object is a Promise. Accessing the .then property causes an error 😅

Would be awesome to only use the "remove additional environment variables" option or maybe to allow checking for ".then" method or maybe there is a better way to test the object in the package ?

Optional values

I would like to validate an environment variable, but allow it to optionally be undefined. I know I can do something like this:

MY_AWESOME_VARIABLE: envalid.str({
  default: '',
  choices: ['astonishing', 'awe-inspiring', 'magnificent', 'wonderous'],
  desc: 'My awesome variable'
})

However, it seems more precise to have the value of MY_AWESOME_VARIABLE be undefined (since it's not defined). Is there a way to achieve this with Envalid? Something like this would be great:

MY_AWESOME_VARIABLE: envalid.str({
  required: false,
  choices: ['astonishing', 'awe-inspiring', 'magnificent', 'wonderous'],
  desc: 'My awesome variable'
})

Thanks!

Checking __esModule in strict mode should be fine

This issue is similar to #74

I have a Next.js project with TypeScript enabled. In its root, there are files called env.config.js and next.config.js:

// env.config.js
const envalid = require("envalid")

module.exports = envalid.cleanEnv(
  process.env,
  {
    FOO: envalid.xxx(),
    BAR: envalid.yyy(),
  },
  { strict: true },
)
// next.config.js
const env = require("./env.config")

module.exports = { /* Next.js config object, which depends on env.FOO and env.BAR */ }

This works fine so far.

In addition to TypeScript files loaded by Next.js, the project also includes a custom server.

// server/index.ts
import * as env from "../env.config"

/* ... code using env.FOO and env.BAR */

The problem with envalid starts when I set esModuleInterop: true in tsconfig.json, which allows me to replace import * as env from "../env.config" with import env from "../env.config".

Starting the server produces this error:

ReferenceError: [envalid] Env var not found: __esModule
    at Object.get (/path/to/project/node_modules/envalid/src/strictProxy.js:54:23)

The reason is that esModuleInterop flag implies checking module.__esModule on import (ref).
It'd be great if envalid ignored reads of env.__esModule, just as it does for env.then.

not failing when env variable not present

When we don't define the NODE_ENV, it is still picking the default value of CONNECTION_ID. Was it suppose to errored when CONNECTION_ID not defined in the environment?

export const config = envalid.cleanEnv(process.env, {
    NODE_ENV: str({ default: 'production', choices: ['production', 'staging', 'development', 'test'] }),
    CONNECTION_ID: str({ devDefault: 'CONNECTION_ID' }),
});

Why limiting NODE_ENV to `development,test,production` ?

Currently, I am using up for the serverless deployment of my node.js project.

There are three predefined stage variables from up:
• development
• staging
• production

So I cannot assign a stage variable like test or whatever.

Here is the error I received: EnvError: Value "staging" not in choices [development,test,production].

I think it is quite normal to use staging in NODE_ENV, is it possible to make the NODE_ENV flexible to cater this kind of usage?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.