Giter VIP home page Giter VIP logo

dieproduktmacher / serverless-env-generator Goto Github PK

View Code? Open in Web Editor NEW
31.0 5.0 11.0 1.3 MB

A Serverless 1.x plugin to manage environment variables with YAML and turn them into a .env file on deployment. Supports encryption with KMS, multiple stages and custom profiles.

License: MIT License

JavaScript 100.00%
serverless-plugin environment-variables credentials encryption aws aws-kms

serverless-env-generator's Introduction

Serverless Env Generator Plugin

License NPM Build Status Requirements Status Coverage Status

This plugin automatically creates a .env file during deployment by merging environment variables from one or more YAML files. During runtime these variables can then be loaded into process.env using dotenv.

For a brief introduction, read our blogpost about introducing serverless-env-generator.

Key features:

  • Support for multi-stage configurations and custom profiles
  • Value of environment variables can be encrypted with AWS KMS, allowing teams to manage sensitive information in git.
  • By using KMS, access to secrets can be controlled with IAM. We recommend to create one KMS key per serverless-profile, so you can limit access to credentials to deployment privileges.
  • During deployment a temporary .env file is created and uploaded to Lambda by merging and decrypting values of your environment YAML files.
  • Environment variables can be loaded with dotenv at startup in Lambda without delays from KMS.
  • Supports serverless-local-dev-server and serverless offline for local development.

Notes

Please note that the uploaded .env file contains secrets in cleartext. Therefore we recommend to use Serverless Crypt for critical secrets. This tool aims to strike a balance between storing secrets in plaintext in Lambda environment variables and having to decrypt them at runtime using KMS.

Furthermore the tool does not support environment variables generated by Serverless. We recommend to set these variables directly in each functions configuration in serverless.yml.

When used with serverless-local-dev-server your environment variables are directly loaded into process.env. No .env file is created to make sure that your local development and deployment tasks do not interfere :-)

This package requires node >= 8.0. Due to the reliance on KMS, encryption is only supported for AWS.

The .env.local file in the project root is here only for the tests.

Table of Contents

Requirements

Getting Started

1. Install the plugin and dotenv

npm install dotenv --save
npm install serverless-env-generator --save-dev

2. Create a key on KMS

See: https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

Please make sure to create the KMS key in the same region as your deployment.

For aliases we recommend to use the service name, for administration privileges no user (your AWS account has full permissions by default) and for usage privileges "serverless-admin" to link access permissions to deployment permissions.

3. Add the plugin to your serverless configuration file

serverless.yml configuration example:

provider:
  name: aws
  runtime: nodejs8.10

functions:
  hello:
    handler: handler.hello

# Add serverless-env-generator to your plugins:
plugins:
  - serverless-env-generator

# Plugin config goes into custom:
custom:
  envFiles: #YAML files used to create .env file
    - environment.yml
  envEncryptionKeyId: #KMS Key used for encrypting values
    dev: ${env:AWS_KMS_KEYID} #Key used for development-stage

4. Add the .env file to your .gitignore

As the generated .env file contains the secrets in cleartext, make sure that it will never be checked into git!

.gitignore code example:

.env

5. Add variables to your environment YAML file

Command example:

serverless env --attribute name --value "This is not a secret"
serverless env --attribute secret_name --value "This is a secret" --encrypt

6. Write your function

Note that the .env file is automatically created when you deploy your function, so you can just load those variables with dotenv ๐ŸŽ‰

Code example:

require('dotenv').config() // Load variables from .env file

module.exports.hello = (event, context, callback) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({
      message: process.env.secret_name,
      input: event
    })
  }
  callback(null, response)
}

7. Deploy & test your function

Command example:

serverless deploy
serverless invoke -f $FUNCTION_NAME

Result example:

{
    "body": "{\"input\": {}, \"message\": \"This is a secret\"}",
    "statusCode": 200
}

Commands

You can use these commands to modify your YAML environment files.

If no stage is specified the default one as specified in serverless.yml is used.

Viewing environment variables

Use the following commands to read and decrypt variables from your YAML environment files:

List variables

serverless env
serverless env --stage $STAGE

View one variable

serverless env --attribute $NAME
serverless env --attribute $NAME --stage $STAGE

#shorthand:
sls env -a $NAME
sls env -a $NAME -s $STAGE

Decrypt variables

serverless env --decrypt
serverless env --attribute $NAME --decrypt
serverless env --attribute $NAME --stage $STAGE --decrypt

#shorthand:
sls env -a $NAME --decrypt
sls env -a $NAME -s $STAGE -d

Setting environment variables

Use the following commands to store and encrypt variables in your YAML environment files:

Note that variables are stored to the first file listed in envFiles.

Set a variable

serverless env --attribute $NAME --value $PLAINTEXT
serverless env --attribute $NAME --value $PLAINTEXT --stage $STAGE

#shorthand:
sls env -a $NAME -v $PLAINTEXT
sls env --a $NAME -v $PLAINTEXT --s $STAGE

Set and encrypt a variable

serverless env --attribute $NAME --value $PLAINTEXT --encrypt
serverless env --attribute $NAME --value $PLAINTEXT --stage $STAGE --encrypt

#shorthand:
sls env -a $NAME -v $PLAINTEXT -e
sls env -a $NAME -v $PLAINTEXT -s $STAGE -e

YAML File Structure

Environment variables are stored in stage-agnostic YAML files, which are then merged into a .env file on deployment.

File example:

dev: #stage
    foo: bar #cleartext variable
    bla: crypted:bc89hwnch8hncoaiwjnd... #encrypted variable

production:
    foo: baz
    bla: crypted:ncibinv0iwokncoiao3d...

You can create additional YAML environment files, for example to include variables that are dynamically generated. Just add them to the envFiles in your serverless.yml.

Usage with the serverless-plugin-webpack

In case you are also using the serverless-plugin-webpack there are some caveats:

1. Plugin order in `serverless.yml'

You have to place serverless-env-generator before the serverless-plugin-webpack in the serverless.yml

# serverless.yml
plugins:
  - serverless-env-generator
  - serverless-plugin-webpack

2. Additional dotenv-webpack

You need to have the dotenv-webpack plugin installed:

npm install dotenv-webpack --save-dev

and configured:

// webpack.config.js
const Dotenv = require('dotenv-webpack')
module.exports = {
  // ...
  plugins: [
    // ...
    new Dotenv()
  ]
}

Contribute

Anyone is more than welcome to contribute to the serverless-env-generator plugin. Here just a few things to consider when doing so:

  • this project uses yarn as a package manager
  • make sure to pass all tests (run yarn test)
  • you can add your local serverless-env-generator version to other projects: yarn add --dev file:/../serverless-env-generator

License & Credits

Licensed under the MIT license.

Created and maintained by DieProduktMacher.

Inspired by Serverless Crypt.

serverless-env-generator's People

Contributors

a-leung avatar afulton11 avatar antesberger avatar chrisschneider avatar enase avatar sergeykiselev1995 avatar tlimp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

serverless-env-generator's Issues

TypeError: Cannot convert undefined or null to object

Hi,

I'm currently trying to use serverless-env-generator but must be doing something wrong, as I'm running into this error

TypeError: Cannot convert undefined or null to object
    at Function.assign (<anonymous>)
    at helper.getEnvVars.then.envFiles (/Users/frix00/Sites/betility/betility-graphql/node_modules/serverless-env-generator/src/index.js:120:41)
From previous event:
    at PluginManager.invoke (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:464:22)
    at PluginManager.run (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:496:17)
    at variables.populateService.then (/usr/local/lib/node_modules/serverless/lib/Serverless.js:116:33)
    at runCallback (timers.js:693:18)
    at tryOnImmediate (timers.js:664:5)
    at processImmediate (timers.js:646:5)
    at process.topLevelDomainCallback (domain.js:121:23)
From previous event:
    at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:103:74)
    at serverless.init.then (/usr/local/lib/node_modules/serverless/bin/serverless.js:52:28)
    at /usr/local/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:111:16
    at /usr/local/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:45:10
    at FSReqWrap.oncomplete (fs.js:139:20)
From previous event:
    at initializeErrorReporter.then (/usr/local/lib/node_modules/serverless/bin/serverless.js:52:6)
    at runCallback (timers.js:693:18)
    at tryOnImmediate (timers.js:664:5)
    at processImmediate (timers.js:646:5)
    at process.topLevelDomainCallback (domain.js:121:23)
From previous event:
    at Object.<anonymous> (/usr/local/lib/node_modules/serverless/bin/serverless.js:38:39)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
    at startup (internal/bootstrap/node.js:266:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:596:3)

Apparently, the culprit is here:
this.options.environment = Object.assign(this.serverless.service.provider.environment, environment, dotenv.config({ path: path.join(config.servicePath, '.env.local') }).parsed)

and this.serverless.service.provider.environment is undefined...


Here my package.json:

{
  "name": "betility-graphql",
  "version": "1.2.0",
  "description": "Grpah QL Layer for Betility",
  "main": "graphql.js",
  "lint-staged": {
    "*.{json,css,md}": [
      "prettier --write",
      "git add"
    ],
    "*.js": [
      "prettier --write",
      "eslint --fix",
      "git add"
    ]
  },
  "scripts": {
    "precommit": "lint-staged",
    "lint": "eslint .",
    "startcache": "serverless offline start --skipCacheInvalidation",
    "start": "serverless offline start",
    "pretest": "npm run lint",
    "test": "jest --coverage",
    "test:watch": "npm test -- --watch"
  },
  "repository": {
    "type": "git",
    "url": "git+ssh://[email protected]:ccinvest/betility-graphql.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "homepage": "https://bitbucket.org/monamourdecomptable/backend-guideline#readme",
  "dependencies": {
    "apollo-server-lambda": "^2.8.1",
    "axios": "^0.19.0",
    "dotenv": "^8.0.0",
    "graphql": "^14.4.2"
  },
  "devDependencies": {
    "aws-sdk": "^2.454.0",
    "eslint": "^5.5.0",
    "eslint-config-airbnb-base": "^13.1.0",
    "eslint-config-prettier": "^3.0.1",
    "eslint-plugin-import": "^2.14.0",
    "eslint-plugin-jest": "^21.21.0",
    "eslint-plugin-prettier": "^2.6.2",
    "husky": "^0.14.3",
    "jest": "^23.5.0",
    "lint-staged": "^7.2.2",
    "prettier": "^1.14.2",
    "serverless": "^1.30.1",
    "serverless-domain-manager": "^3.2.6",
    "serverless-env-generator": "^1.4.1",
    "serverless-offline": "^5.9.0",
    "serverless-plugin-warmup": "^4.7.0-rc.1"
  }
}

I currently have a .env :
USER_BACKEND_DOMAIN=http://

and my environment.yml :
dev:
USER_BACKEND_DOMAIN: 'http:///'

After intialising the project, I run
sls env --attribute USER_BACKEND_DOMAIN --value "http://<IP-address>/" --stage dev
Then:
sls env create

Then I create my .env again (since it gets deleted by the "create" command

Finally, I run
sls offline start

and get the error mentioned. I also tried sls offline start --stage dev but I get the same output.

Where am I going wrong ?

Thanks

.env being removed after being generated

running

> sls env generate

first creates and then removes just generated .env. Sample output:

Serverless: DOTENV: Could not find .env file.
Serverless: Creating .env file...
Serverless: Removed .env file

same happens for sls package and other commands which seems to be expected according to after:deploy hooks, but shouldn't happen for a manual generation?

Running serverless v1.51.0

Not compatible with latest version of serverless (1.60.5)

What did you do?
sls env -a abc -v abc

What happened?
Serverless Error ---------------------------------------

"abc" is not a valid sub command. Run "serverless env" to see a more helpful error message for this command.
What should've happened?
An environment.yml file should have been generated with a variable 'abc' of value 'abc'.

What's the content of your serverless.yml file?

service: test

provider:
name: aws
runtime: nodejs12.x

functions:
hello:
handler: handler.hello
What's the output you get when you use the SLS_DEBUG=* environment variable (e.g. SLS_DEBUG=* serverless deploy)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.