Giter VIP home page Giter VIP logo

iopipe-js-profiler's Introduction

IOpipe Agent for JavaScript


Coverage Status npm version styled with prettier semantic-release

IOpipe is a serverless DevOps platform for organizations building event-driven architectures in AWS Lambda. IOpipe captures crucial, high-fidelity metrics for each Lambda function invocation. This data powers a flexibile and robust development and operations experience with features including tracing, profiling, custom metrics, and low-latency alerts. Get started today to quickly and confidently gain superior observability, identify issues, and discover anomalies in your connected applications.

Note: this library is a lower-level implementation than the package you might likely be looking for. Enjoy pre-bundled plugins like tracing and event info with @iopipe/iopipe

Installation

Install using your package manager of choice,

npm install @iopipe/core

or

yarn add @iopipe/core

If you are using the Serverless Framework to deploy your lambdas, check out our serverless plugin.

Usage

Configure the library with your project token (register for access), and it will automatically monitor and collect metrics from your applications running on AWS Lambda.

Example:

const iopipeLib = require('@iopipe/core');

const iopipe = iopipeLib({ token: 'PROJECT_TOKEN' });

exports.handler = iopipe((event, context) => {
  context.succeed('This is my serverless function!');
});

Custom metrics

You may add custom metrics to an invocation using context.iopipe.metric to add either string or numerical values. Keys have a maximum length of 256 characters, and string values are limited to 1024.

Example:

const iopipeLib = require('@iopipe/core');

const iopipe = iopipeLib({ token: 'PROJECT_TOKEN' });

exports.handler = iopipe((event, context) => {
  context.iopipe.metric('key', 'some-value');
  context.iopipe.metric('another-key', 42);
  context.succeed('This is my serverless function!');
});

Labels

You can label invocations using context.iopipe.label to label an invocation with a string value, with a limit of 128 characters.

Example:

const iopipeLib = require('@iopipe/core');

const iopipe = iopipeLib({ token: 'PROJECT_TOKEN' });

exports.handler = iopipe((event, context) => {
  context.iopipe.label('something-important-happened');
  context.succeed('This is my serverless function!');
});

Configuration

Methods

You can configure your iopipe setup through one or more different methods - that can be mixed, providing a config chain. The current methods are listed below, in order of precendence. The module instantiation object overrides all other config values (if values are provided).

  1. Module instantiation object
  2. IOPIPE_* environment variables
  3. An .iopiperc file
  4. An iopipe package.json entry
  5. An extends key referencing a config package
  6. Default values

Options

token (string: required)

If not supplied, the environment variable $IOPIPE_TOKEN will be used if present. Find your project token.

debug (bool: optional = false)

Debug mode will log all data sent to IOpipe servers to STDOUT. This is also a good way to evaluate the sort of data that IOpipe is receiving from your application. If not supplied, the environment variable $IOPIPE_DEBUG will be used if present.

const iopipe = require('@iopipe/core')({
  token: 'PROJECT_TOKEN',
  debug: true
});

exports.handler = iopipe((event, context, callback) => {
  // Do things here. We'll log info to STDOUT.
});

networkTimeout (int: optional = 5000)

The number of milliseconds IOpipe will wait while sending a report before timing out. If not supplied, the environment variable $IOPIPE_NETWORK_TIMEOUT will be used if present.

const iopipe = require('@iopipe/core')({ token: 'PROJECT_TOKEN', networkTimeout: 30000})

timeoutWindow (int: optional = 150)

By default, IOpipe will capture timeouts by exiting your function 150ms early from the AWS configured timeout, to allow time for reporting. You can disable this feature by setting timeoutWindow to 0 in your configuration. If not supplied, the environment variable $IOPIPE_TIMEOUT_WINDOW will be used if present.

const iopipe = require('@iopipe/core')({ token: 'PROJECT_TOKEN', timeoutWindow: 0})

plugins (array: optional)

Note that if you use the @iopipe/iopipe package, you get our recommended plugin set-up right away. Plugins can extend the functionality of IOpipe in ways that best work for you. Follow the guides for the plugins listed below for proper installation and usage on the @iopipe/core library:

Example:

const tracePlugin = require('@iopipe/trace');

const iopipe = require('@iopipe/core')({
  token: 'PROJECT_TOKEN',
  plugins: [tracePlugin()]
});

exports.handler = iopipe((event, context, callback) => {
  // Run your fn here
});

enabled (boolean: optional = True)

Conditionally enable/disable the agent. The environment variable $IOPIPE_ENABLED will also be checked.

url (string: optional)

Sets an alternative URL to use for the IOpipe collector. The environment variable $IOPIPE_COLLECTOR_URL will be used if present.

RC File Configuration

Not recommended for webpack/bundlers due to dynamic require.

You can configure iopipe via an .iopiperc RC file. An example of that is here. Config options are the same as the module instantiation object, except for plugins. Plugins should be an array containing mixed-type values. A plugin value can be a:

  • String that is the name of the plugin
  • Or an array with plugin name first, and plugin options second
{
  "token": "wow_token",
  "plugins": [
    "@iopipe/trace",
    ["@iopipe/profiler", {"enabled": true}]
  ]
}

IMPORTANT: You must install the plugins as dependencies for them to load properly in your environment.

package.json Configuration

Not recommended for webpack/bundlers due to dynamic require.

You can configure iopipe within a iopipe package.json entry. An example of that is here. Config options are the same as the module instantiation object, except for plugins. Plugins should be an array containing mixed-type values. A plugin value can be a:

  • String that is the name of the plugin
  • Or an array with plugin name first, and plugin options second
{
  "name": "my-great-package",
  "dependencies": {
    "@iopipe/trace": "^0.2.0",
    "@iopipe/profiler": "^0.1.0"
  },
  "iopipe": {
    "token": "wow_token",
    "plugins": [
      "@iopipe/trace",
      ["@iopipe/profiler", {"enabled": true}]
    ]
  }
}

IMPORTANT: You must install the plugins as dependencies for them to load properly in your environment.

Extends Configuration

Not recommended for webpack/bundlers due to dynamic require.

You can configure iopipe within a package.json or rc file by referencing a extends config package. An example of that is here. Config options are the same as the module instantiation object, except for plugins. Plugins should be an array containing mixed-type values. A plugin value can be a:

  • String that is the name of the plugin
  • Or an array with plugin name first, and plugin options second

For an example of a config package, check out @iopipe/config.

IMPORTANT: You must install the config package and plugins as dependencies for them to load properly in your environment.

License

Apache 2.0

iopipe-js-profiler's People

Contributors

coreylight avatar dependabot[bot] avatar ewindisch avatar mrickard avatar pselle avatar starsinmypockets avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

iopipe-js-profiler's Issues

Switch to signer.<region>.iopipe.com hostnames

API Gateway is horribly broken when it comes to multi region support. So in the meantime we'll need to use signer.<region>.iopipe.com hostnames for the signer, similar to the way the collector works.

Add automatic labels

In order to make filtering invocations easier, the plugin should automatically label invocations with relevant labels.

Use gzip compression of data before uploading to S3

Use gzip to lower costs on s3 and save time on the network request.
May need some benchmarking to make sure we are actually saving time.
Can stream to the zlib module once we move to a streaming transport too ๐Ÿ˜Ž

Cert issue appearing on warm starts

With debug on, seeing the following in the logs, not on cold starts, on the following request:

2018-02-20T23:24:40.254Z [Requestid] (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: write EPROTO 140611409368896:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:794:

Upload seems to continue to work, but should investigate this issue.

Semantic release

This plugin could semantically release, now that the other JS plugins are following that format.

More granular tests

We do a full integration test with the IOpipe library, but do not verify the functionality of the cpu and heap profilers. We should improve these tests!

Support heap profiling

Users are likely to desire heap profiling. This might make sense in this plugin as we already install the dependency and will already be making calls to the s3-preauth-api and s3.

I did some research and while heap snapshots are documented, heap profiling is not documented for v8-profiler. It does seem to be implemented, but it's not clear if it's working correctly.

Tests do not pass under WSL

Windows Subsystem for Linux does not provide a fully compatible /proc. Tests do not pass due to looking for the boot_id.

Package fails to install on OSX

Following error occurs with yarn add @iopipe/profiler:

error /Users/kolanos/Web/iopipe/iopipe-js-core/testProjects/extendConfig/node_modules/v8-profiler: Command failed.
Exit code: 1
Command: sh
Arguments: -c node-pre-gyp install --fallback-to-build
Directory: /Users/kolanos/Web/iopipe/iopipe-js-core/testProjects/extendConfig/node_modules/v8-profiler
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using [email protected]
node-pre-gyp info using [email protected] | darwin | x64
node-pre-gyp info check checked for "/Users/kolanos/Web/iopipe/iopipe-js-core/testProjects/extendConfig/node_modules/v8-profiler/build/profiler/v5.7.0/node-v59-darwin-x64/profiler.node" (not found)
node-pre-gyp http GET https://node-inspector.s3.amazonaws.com/profiler/v5.7.0/node-v59-darwin-x64.tar.gz
node-pre-gyp http 404 https://node-inspector.s3.amazonaws.com/profiler/v5.7.0/node-v59-darwin-x64.tar.gz
node-pre-gyp ERR! Tried to download(404): https://node-inspector.s3.amazonaws.com/profiler/v5.7.0/node-v59-darwin-x64.tar.gz 
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected] (node-v59 ABI, unknown) (falling back to source compile with node-gyp) 
node-pre-gyp http 404 status code downloading tarball https://node-inspector.s3.amazonaws.com/profiler/v5.7.0/node-v59-darwin-x64.tar.gz 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp clean' (Error: spawn node-gyp ENOENT)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/kolanos/Web/iopipe/iopipe-js-core/testProjects/extendConfig/node_modules/node-pre-gyp/lib/util/compile.js:77:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:159:13)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:207:12)
node-pre-gyp ERR! stack     at onErrorNT (internal/child_process.js:389:16)
node-pre-gyp ERR! stack     at process._tickCallback (internal/process/next_tick.js:152:19)
node-pre-gyp ERR! System Darwin 16.7.0
node-pre-gyp ERR! command "/usr/local/Cellar/node/9.3.0_1/bin/node" "/Users/kolanos/Web/iopipe/iopipe-js-core/testProjects/extendConfig/node_modules/v8-profiler/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/kolanos/Web/iopipe/iopipe-js-core/testProjects/extendConfig/node_modules/v8-profiler
node-pre-gyp ERR! node -v v9.3.0
node-pre-gyp ERR! node-pre-gyp -v v0.6.39
node-pre-gyp ERR! not ok 
Failed to execute 'node-gyp clean' (Error: spawn node-gyp ENOENT)
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

add jwtAccess to report

The current release may be broken because we do not provide a jwtAccess key which seems now necessary for uploads of files with extensions.

Errors silently on local invoke

Attempted local invocations via sls invoke local and did not get profiling info in HUD.

Worked fine when running the function on AWS.

Add "enabled" flag

We need to know if an invocation has profiling data -- the plugin can report if it is enabled via an enabled flag in the metadata

Export to linux `perf`

The perf tool on Linux is a somewhat standard way of working with profiling information. v8 itself supports exporting to this tool already! It isn't clear if we can do this with the v8-profiler module or not.

filed as a possibly interesting feature / improvement.

Enabling profiling via env variable in Serverless Framework not working

When settings IOPIPE_ENABLE_PROFILER: true in serverless.yml file profiling is not getting enabled and I get an error in my CW logs with debug on

Node10
serverless-iopipe-layers: 0.3.3
arn:aws:lambda:us-east-1:146318645305:layer:IOpipeNodeJS10:1

2019-07-09T16:09:30.606Z    8013aca9-2f0e-4702-9aee-6954d5edc30a    ERROR    TypeError: Cannot read property 'getFileUploadMeta' of undefined
 at ProfilerPlugin.getFileUploadMeta (/opt/nodejs/node_modules/@iopipe/profiler/dist/index.js:124:23)
 at ProfilerPlugin.preInvoke (/opt/nodejs/node_modules/@iopipe/profiler/dist/index.js:96:28)
 at /opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:1984:30
 at Array.map (<anonymous>)
 at IOpipeWrapperClass._callee3$ (/opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:1980:44)
 at tryCatch (/opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:2244:40)
 at Generator.invoke [as _invoke] (/opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:2478:22)
 at Generator.prototype.(anonymous function) [as next] (/opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:2296:21)
 at step (/opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:1034:30)
 at /opt/nodejs/node_modules/@iopipe/core/dist/iopipe.js:1052:14

Include iopipe report in the profiler data

At the moment, it looks like there are no blockers to keeping the profiler running while the report to iopipe is sending. The profiler could be turned off and the profile data after the report is finished.
Alternatively, both requests could run asynchronously. Perhaps including an option to specify which is useful.

Document webpack configuration when using this plugin

when users include this module and use webpack, the binary library will cause errors unless the correct webpack configuration is used. We should document this.

We might also look at the serverless-webpack plugin and see if we can make changes there, too!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.