Giter VIP home page Giter VIP logo

iopipe / iopipe-js-core Goto Github PK

View Code? Open in Web Editor NEW
123.0 123.0 20.0 3.08 MB

Observe and develop serverless apps with confidence on AWS Lambda with Tracing, Metrics, Profiling, Monitoring, and more.

Home Page: https://www.iopipe.com/

License: Apache License 2.0

JavaScript 99.78% Makefile 0.12% Dockerfile 0.10%
analytics aws aws-lambda debugging devops iopipe iopipe-agent javascript lambda monitoring profiling serverless

iopipe-js-core's Introduction

IOpipe Agent for JavaScript


Coverage Status npm version styled with prettier semantic-release

IOpipe is a serverless DevOps platform for organizations building event-driven architectures in AWS Lambda. IOpipe captures crucial, high-fidelity metrics for each Lambda function invocation. This data powers a flexibile and robust development and operations experience with features including tracing, profiling, custom metrics, and low-latency alerts. Get started today to quickly and confidently gain superior observability, identify issues, and discover anomalies in your connected applications.

Note: this library is a lower-level implementation than the package you might likely be looking for. Enjoy pre-bundled plugins like tracing and event info with @iopipe/iopipe

Installation

Install using your package manager of choice,

npm install @iopipe/core

or

yarn add @iopipe/core

If you are using the Serverless Framework to deploy your lambdas, check out our serverless plugin.

Usage

Configure the library with your project token (register for access), and it will automatically monitor and collect metrics from your applications running on AWS Lambda.

Example:

const iopipeLib = require('@iopipe/core');

const iopipe = iopipeLib({ token: 'PROJECT_TOKEN' });

exports.handler = iopipe((event, context) => {
  context.succeed('This is my serverless function!');
});

Custom metrics

You may add custom metrics to an invocation using context.iopipe.metric to add either string or numerical values. Keys have a maximum length of 256 characters, and string values are limited to 1024.

Example:

const iopipeLib = require('@iopipe/core');

const iopipe = iopipeLib({ token: 'PROJECT_TOKEN' });

exports.handler = iopipe((event, context) => {
  context.iopipe.metric('key', 'some-value');
  context.iopipe.metric('another-key', 42);
  context.succeed('This is my serverless function!');
});

Labels

You can label invocations using context.iopipe.label to label an invocation with a string value, with a limit of 128 characters.

Example:

const iopipeLib = require('@iopipe/core');

const iopipe = iopipeLib({ token: 'PROJECT_TOKEN' });

exports.handler = iopipe((event, context) => {
  context.iopipe.label('something-important-happened');
  context.succeed('This is my serverless function!');
});

Configuration

Methods

You can configure your iopipe setup through one or more different methods - that can be mixed, providing a config chain. The current methods are listed below, in order of precendence. The module instantiation object overrides all other config values (if values are provided).

  1. Module instantiation object
  2. IOPIPE_* environment variables
  3. An .iopiperc file
  4. An iopipe package.json entry
  5. An extends key referencing a config package
  6. Default values

Options

token (string: required)

If not supplied, the environment variable $IOPIPE_TOKEN will be used if present. Find your project token.

debug (bool: optional = false)

Debug mode will log all data sent to IOpipe servers to STDOUT. This is also a good way to evaluate the sort of data that IOpipe is receiving from your application. If not supplied, the environment variable $IOPIPE_DEBUG will be used if present.

const iopipe = require('@iopipe/core')({
  token: 'PROJECT_TOKEN',
  debug: true
});

exports.handler = iopipe((event, context, callback) => {
  // Do things here. We'll log info to STDOUT.
});

networkTimeout (int: optional = 5000)

The number of milliseconds IOpipe will wait while sending a report before timing out. If not supplied, the environment variable $IOPIPE_NETWORK_TIMEOUT will be used if present.

const iopipe = require('@iopipe/core')({ token: 'PROJECT_TOKEN', networkTimeout: 30000})

timeoutWindow (int: optional = 150)

By default, IOpipe will capture timeouts by exiting your function 150ms early from the AWS configured timeout, to allow time for reporting. You can disable this feature by setting timeoutWindow to 0 in your configuration. If not supplied, the environment variable $IOPIPE_TIMEOUT_WINDOW will be used if present.

const iopipe = require('@iopipe/core')({ token: 'PROJECT_TOKEN', timeoutWindow: 0})

plugins (array: optional)

Note that if you use the @iopipe/iopipe package, you get our recommended plugin set-up right away. Plugins can extend the functionality of IOpipe in ways that best work for you. Follow the guides for the plugins listed below for proper installation and usage on the @iopipe/core library:

Example:

const tracePlugin = require('@iopipe/trace');

const iopipe = require('@iopipe/core')({
  token: 'PROJECT_TOKEN',
  plugins: [tracePlugin()]
});

exports.handler = iopipe((event, context, callback) => {
  // Run your fn here
});

enabled (boolean: optional = True)

Conditionally enable/disable the agent. The environment variable $IOPIPE_ENABLED will also be checked.

url (string: optional)

Sets an alternative URL to use for the IOpipe collector. The environment variable $IOPIPE_COLLECTOR_URL will be used if present.

RC File Configuration

Not recommended for webpack/bundlers due to dynamic require.

You can configure iopipe via an .iopiperc RC file. An example of that is here. Config options are the same as the module instantiation object, except for plugins. Plugins should be an array containing mixed-type values. A plugin value can be a:

  • String that is the name of the plugin
  • Or an array with plugin name first, and plugin options second
{
  "token": "wow_token",
  "plugins": [
    "@iopipe/trace",
    ["@iopipe/profiler", {"enabled": true}]
  ]
}

IMPORTANT: You must install the plugins as dependencies for them to load properly in your environment.

package.json Configuration

Not recommended for webpack/bundlers due to dynamic require.

You can configure iopipe within a iopipe package.json entry. An example of that is here. Config options are the same as the module instantiation object, except for plugins. Plugins should be an array containing mixed-type values. A plugin value can be a:

  • String that is the name of the plugin
  • Or an array with plugin name first, and plugin options second
{
  "name": "my-great-package",
  "dependencies": {
    "@iopipe/trace": "^0.2.0",
    "@iopipe/profiler": "^0.1.0"
  },
  "iopipe": {
    "token": "wow_token",
    "plugins": [
      "@iopipe/trace",
      ["@iopipe/profiler", {"enabled": true}]
    ]
  }
}

IMPORTANT: You must install the plugins as dependencies for them to load properly in your environment.

Extends Configuration

Not recommended for webpack/bundlers due to dynamic require.

You can configure iopipe within a package.json or rc file by referencing a extends config package. An example of that is here. Config options are the same as the module instantiation object, except for plugins. Plugins should be an array containing mixed-type values. A plugin value can be a:

  • String that is the name of the plugin
  • Or an array with plugin name first, and plugin options second

For an example of a config package, check out @iopipe/config.

IMPORTANT: You must install the config package and plugins as dependencies for them to load properly in your environment.

License

Apache 2.0

iopipe-js-core's People

Contributors

adjohn avatar coreylight avatar dependabot[bot] avatar ewindisch avatar kolanos avatar mrickard avatar pselle avatar starsinmypockets avatar xerthesquirrel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

iopipe-js-core's Issues

post:invoke should not be called twice on Timeout

While working on changes to @iopipe/iopipe-plugin-profiler, I found that on timeouts post:invoke is called twice! I confirmed this by adding a console.log() in sendReport().


12:51:06
START RequestId: fd0fed0a-fbae-11e7-a3f1-859063e13748 Version: $LATEST

12:51:07
2018-01-17T17:51:07.163Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	Slack event:

12:51:07

12:51:07
2018-01-17T17:51:07.163Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	No image found in text.

12:51:07
2018-01-17T17:51:07.164Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	null

12:51:07
2018-01-17T17:51:07.164Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	sendReport called.

12:51:07
2018-01-17T17:51:07.165Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	PostInvoke run! 1

12:51:07
2018-01-17T17:51:07.179Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	invoked function context: {"callbackWaitsForEmptyEventLoop":true,"logGroupName":"/aws/lambda/slackbot-imgdesc-dev-event","logStreamName":"2018/01/17/[$LATEST]a9b3e575d58541e6bf0f190e27d88bac","functionName":"slackbot-imgdesc-dev-event","memoryLimitInMB":"512","functionVersion":"$LATEST","invokeid":"fd0fed0a-fbae-11e7-a3f1-859063e13748"

12:51:07
2018-01-17T17:51:07.427Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	Stopping profiling.

12:51:21
2018-01-17T17:51:21.518Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	sendReport called.

12:51:21
2018-01-17T17:51:21.519Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	PostInvoke run! 1

12:51:21
2018-01-17T17:51:21.538Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	invoked function context: {"callbackWaitsForEmptyEventLoop":true,"logGroupName":"/aws/lambda/slackbot-imgdesc-dev-event","logStreamName":"2018/01/17/[$LATEST]a9b3e575d58541e6bf0f190e27d88bac","functionName":"slackbot-imgdesc-dev-event","memoryLimitInMB":"512","functionVersion":"$LATEST","invokeid":"fd0fed0a-fbae-11e7-a3f1-859063e13748"

12:51:21
2018-01-17T17:51:21.600Z	fd0fed0a-fbae-11e7-a3f1-859063e13748	Stopping profiling.

12:51:21
END RequestId: fd0fed0a-fbae-11e7-a3f1-859063e13748
END RequestId: fd0fed0a-fbae-11e7-a3f1-859063e13748

12:51:21
REPORT RequestId: fd0fed0a-fbae-11e7-a3f1-859063e13748	Duration: 15012.01 ms	Billed Duration: 15000 ms Memory Size: 512 MB	Max Memory Used: 97 MB

Generate report after post:invoke, before pre:report

Currently, the report is generated inside of the report.send function.
We should separate the report generation from the report sending, and make sure that we insert the report generation in between the post:invoke and pre:report hooks.

Timeout expception

I am getting the following exception in IOpipe dashboard each time it tries to send the data, though no errors in cloudwatcg log:

Error: Timeout Exceeded.
at Timeout.<anonymous> (/var/task/node_modules/iopipe/dist/iopipe.js:851:38)
at ontimeout (timers.js:386:14)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5)

Uncaught TypeError: context.getRemainingTimeInMillis is not a function

We use the serverless-mocha-plugin to run our mocha unit tests before deploying to AWS.

https://github.com/SC5/serverless-mocha-plugin

Our unit tests run locally on Jenkins.

When we wrap our lambdas in iopipe, the following exception is thrown when the unit tests are run:

Uncaught TypeError: context.getRemainingTimeInMillis is not a function

This error is coming from iopipe code, and not our code under test. This means that our unit tests fail, and Jenkins will not deploy to AWS.

At the moment, this is blocking us from using iopipe :(

debug=true does not log to console

Speaking with a user, they were not getting their logs to cloudwatch when using the debug flag. Checking, I see we specified a . instead of a _ for the response_body variable.

Nesting IOpipe fails

The following fails:

var iopipeOuter = require('iopipe')()
var iopipeInner = require('iopipe')()

export.handler = iopipeOuter(iopipeInner(
 (event, callback) => { callback() }
))

I wanted to do this to perform profiling and debugging of the IOpipe library.

Serverless Framework: Syntax error when trying to use iopipe for 2 functions

My index.js file looks something like this:

const iopipe = require('iopipe')({clientId: process.env.IOPIPE_CLIENTID});

module.exports.parse = (event, context, callback) => {...};
module.exports.process = (event, context, callback) => {...};

If I wrap one function into iopipe, everything works great:

const iopipe = require('iopipe')({clientId: process.env.IOPIPE_CLIENTID});

module.exports.parse = iopipe((event, context, callback) => {...});
module.exports.process = (event, context, callback) => {...};

But if I do so for both functions, I'm getting a syntax error:

const iopipe = require('iopipe')({clientId: process.env.IOPIPE_CLIENTID});

module.exports.parse = iopipe((event, context, callback) => {...});
module.exports.process = iopipe((event, context, callback) => {...});
Syntax Error -------------------------------------------
 
     Unexpected token {
 
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
 
  Stack Trace --------------------------------------------
 
SyntaxError: Unexpected token {
    at new Function (<anonymous>)
    at copyCollection (/dev/lambdas/node_modules/deepcopy/lib/copy.js:62:14)
    at copy (/dev/lambdas/node_modules/deepcopy/lib/copy.js:17:10)
    at recursiveCopy (/dev/lambdas/node_modules/deepcopy/lib/index.js:73:35)
    at deepcopy (/dev/lambdas/node_modules/deepcopy/lib/index.js:35:10)
    at new_context (/dev/lambdas/node_modules/iopipe/index.js:163:23)
    at /dev/lambdas/node_modules/iopipe/index.js:207:17

"Serverless offline" plugin: Block-scoped declarations not yet supported outside strict mode

https://github.com/dherault/serverless-offline

[ 'SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode',
  'at Function (native)',
  'at copyCollection (/git/retraced/api/node_modules/iopipe/node_modules/deepcopy/lib/copy.js:62:14)',
  'at copy (/git/retraced/api/node_modules/iopipe/node_modules/deepcopy/lib/copy.js:17:10)',
  'at recursiveCopy (/git/retraced/api/node_modules/iopipe/node_modules/deepcopy/lib/index.js:73:35)',
  'at deepcopy (/git/retraced/api/node_modules/iopipe/node_modules/deepcopy/lib/index.js:35:10)',
  'at emitter.on.context.__defineSetter__ (/git/retraced/api/node_modules/iopipe/index.js:163:23)',
  'at /git/retraced/api/node_modules/iopipe/index.js:207:17' ]

used from: "iopipe": "0.0.20"

'use strict';

const iopipe = require('iopipe')({
  clientId: require('./lib/config/getConfig')().IOPipe.ClientID,
});

module.exports.default = iopipe(
  (event, context, cb) => {

Running local functions in a windows dev-env with SLS fails

When executing a functional using SLS w/ SLS-webpack, the function fails to execute when wrapped with iopipe. It appears the agent is not auto-mocking some of the Linux based commands in this type of environment. Once the function is deployed to AWS however it works normally.

callbackWaitsForEmptyEventLoop breaks in 0.0.13

Because we deepcopy context, the getter/setter are called for callbackWaitsForEmptyEventLoop instead of copying the getter/setter themselves. We need to preserve the getter/setter behavior such that callbackWaitsForEmptyEventLoop=false works correctly.

Publish v1.5.0 to NPM

We are getting a warning in iopipe console that a newer version is available but the newest version is not available on NPM

context.indexOf is not a function

I tried to integrate iopipe but haven't been able to get it working due to this error

2016-08-17T22:02:44.986Z    ecac0527-64c5-11e6-99d5-09ceaa62d15a
TypeError: context.indexOf is not a function 
at generateLog (/var/task/node_modules/iopipe/index.js:99:17) 
at /var/task/node_modules/iopipe/index.js:213:9

I'm on lambda using the nodejs4.3 runtime.

host vm_id should be string, not Buffer.

the vm_id should be a hex string!

                  "environment": {
                     "host": {
                        "vm_id": {
                           "type": "Buffer",
                           "data": [
                              48,
                              101,
                              102,
                              56,
                              53,
                              49,
                              48,
                              52,
                              45,
                              99,
                              50,
                              101,
                              102,
                              45,
                              52,
                              57,
                              52,
                              49,
                              45,
                              57,
                              48,
                              48,
                              51,
                              45,
                              48,
                              48,
                              100,
                              51,
                              50,
                              49,
                              50,
                              49,
                              48,
                              52,
                              57,
                              57
                           ]
                        }
                     },
                     "os": 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.