Giter VIP home page Giter VIP logo

cls-proxify's Introduction

cls-proxify Build Status Coverage Status Tweet

Logging on steroids with CLS and Proxy. A small library that proxies any arbitrary object with a proxy from Continuation-Local Storage a.k.a. CLS if found one. Super-useful for creating child loggers per each request with dynamic context from the request itself (e.g. adding request trace ID, adding request payload). Integrated with express, koa, fastify out-of-the-box.

Many thanks to @mcollina for the idea of combining Proxy and CLS.

Installation

npm i cls-proxify cls-hooked

Quick start

Express

TypeScript users, clsProxifyExpressMiddleware uses typings from @types/express. Please, run npm i -D @types/express

import { clsProxify } from 'cls-proxify'
import { clsProxifyExpressMiddleware } from 'cls-proxify/integration/express'
import * as express from 'express'

const logger = {
  info: (msg: string) => console.log(msg),
}
const loggerCls = clsProxify(logger)

const app = express()
app.use(
  clsProxifyExpressMiddleware((req) => {
    const headerRequestID = req.headers.Traceparent
    const loggerProxy = {
      info: (msg: string) => `${headerRequestID}: ${msg}`,
    }
    // this value will be accesible in CLS by key 'cls-proxify'
    // it will be used as a proxy for `loggerCls`
    return loggerProxy
  }),
)

app.get('/test', (req, res) => {
  loggerCls.info('My message!')
  // Logs `${headerRequestID}: My message!` into the console
  // Say, we send GET /test with header 'Traceparent' set to 12345
  // It's going to log '12345: My message!'
  // If it doesn't find anything in CLS by key 'cls-proxify' it uses the original `logger` and logs 'My message!'
})

Koa

TypeScript users, clsProxifyKoaMiddleware uses typings from @types/koa. Please, run npm i -D @types/koa

import { clsProxify } from 'cls-proxify'
import { clsProxifyKoaMiddleware } from 'cls-proxify/integration/koa'
import * as Koa from 'koa'

const logger = {
  info: (msg: string) => console.log(msg),
}
const loggerCls = clsProxify(logger)

const app = new Koa()
app.use(
  clsProxifyKoaMiddleware((ctx) => {
    const headerRequestID = ctx.req.headers.Traceparent
    const loggerProxy = {
      info: (msg: string) => `${headerRequestID}: ${msg}`,
    }
    // this value will be accesible in CLS by key 'cls-proxify'
    // it will be used as a proxy for `loggerCls`
    return loggerProxy
  }),
)

app.use((ctx) => {
  loggerCls.info('My message!')
  // Logs `${headerRequestID}: My message!` into the console
  // Say, we send GET / with header 'Traceparent' set to 12345
  // It's going to log '12345: My message!'
  // If it doesn't find anything in CLS by key 'cls-proxify' it uses the original `logger` and logs 'My message!'
})

Fastify

import { clsProxify } from 'cls-proxify'
import { clsProxifyFastifyPlugin } from 'cls-proxify/integration/fastify'
import * as fastify from 'fastify'

const logger = {
  info: (msg: string) => console.log(msg),
}
const loggerCls = clsProxify(logger)

const app = fastify()
app.register(clsProxifyFastifyPlugin, {
  proxify: (req) => {
    const headerRequestID = ctx.req.headers.Traceparent
    const loggerProxy = {
      info: (msg: string) => `${headerRequestID}: ${msg}`,
    }
    // this value will be accesible in CLS by key 'cls-proxify'
    // it will be used as a proxy for `loggerCls`
    return loggerProxy
  },
})

app.get('/test', (req, res) => {
  loggerCls.info('My message!')
  // Logs `${headerRequestID}: My message!` into the console
  // Say, we send GET /test with header 'Traceparent' set to 12345
  // It's going to log '12345: My message!'
  // If it doesn't find anything in CLS by key 'cls-proxify' it uses the original `logger` and logs 'My message!'
})

Any other framework or library

import { clsProxify, getClsHookedStorage } from 'cls-proxify'
import AbstractWebServer from 'abstract-web-server'

const logger = {
  info: (msg: string) => console.log(msg),
}
const loggerCls = clsProxify(logger)

const app = new AbstractWebServer()
// Assuming this AbstractWebServer supports some form of middlewares
app.use((request, response, next) => {
  // Assuming your request and response are event emitters
  getClsHookedStorage().namespace.bindEmitter(request)
  getClsHookedStorage().namespace.bindEmitter(response)

  getClsHookedStorage().namespace.run(() => {
    const headerRequestID = request.headers.Traceparent
    // this value will be accesible in CLS by key 'cls-proxify'
    // it will be used as a proxy for `loggerCls`
    const loggerProxy = {
      info: (msg: string) => `${headerRequestID}: ${msg}`,
    }
    getClsHookedStorage().set(loggerProxy)

    next()
  })
})

app.get('/test', (req, res) => {
  loggerCls.info('My message!')
  // Logs `${headerRequestID}: My message!` into the console
  // Say, we send GET /test with header 'Traceparent' set to 12345
  // It's going to log '12345: My message!'
  // If it doesn't find anything in CLS by key 'cls-proxify' it uses the original `logger` and logs 'My message!'
})

Set custom CLS storage

import { clsProxify, setClsHookedStorage, ClsHookedStorage, ClsProxifyStorage } from 'cls-proxify'
import AbstractWebServer from 'abstract-web-server'

// You can subclass existing ClsHookedStorage
class CustomClsStorage extends ClsHookedStorage {
  // Override namespace
  public readonly namespace = createNamespace('myNamespace')
  // Or override CLS key
  protected readonly key = 'yoda'
}
setClsHookedStorage(new CustomClsStorage())

// Or you can implement your own storage from scratch.
// Just make sure it conforms to `ClsProxifyStorage` interface.
class SecretStorage<T> implements ClsProxifyStorage<T> {
  set(proxy: T): void {}
  get(): T | undefined {}
}
setClsHookedStorage(new SecretStorage())

In depth

How it works

If you're struggling to grasp the idea behind CLS at these two articles: Request Id Tracing in Node.js Applications, A Pragmatic Overview of Async Hooks API in Node.js.

Take a look at this article which overviews how CLS works and covers the idea behind this library.

We wrap our original logger in a Proxy. Every time we try to access any property of that object we first check if there's an updated logger in CLS available. If it's there we take the property from it. If it's not we take the property from the original logger. Then for every request we create a CLS context using run and bindEmitter. Once the context is created we enhance our original logger with some extra data and put the updated logger in the context. Once we try to call any method of our logger we'll actually call the same method on our logger in CLS.

Does it work only for loggers?

No. You can proxify any object you want. Moreover you can even proxify functions and class constructors.

Here's a list of traps cls-proxify provides:

Take a look at the tests to get an idea of how you can utilize them.

Live demos

Troubleshooting

My context got lost

Note that some middlewares may cause CLS context to get lost. To avoid it use any third party middleware that does not need access to request ids before you use this middleware.

I'm experiencing a memory leak

Make sure you don't keep any external references to the objects inside of CLS. It may prevent them from being collected by GC. Take a look at this issues: #21, #11.

cls-proxify's People

Contributors

aigoncharov avatar dependabot[bot] avatar osher avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

cls-proxify's Issues

Recommended way to log with hapi-pino

I read this article about adding tracing to logs and it inspired me to want to enable this inside my hapi/hapi-pino project.

I registered hapi-pino as a hapi plugin just like the docs recommend (below). I'm wondering if it would make more sense to register a separate cls-pino-logger plugin or if it would be recommended to do it somehow inside of hapi-pino? Thanks in advance for any guidance.

await server.register({
    plugin: require('hapi-pino'),
    options: {
      prettyPrint: process.env.NODE_ENV !== 'production',
      // Redact Authorization headers, see https://getpino.io/#/docs/redaction
      redact: ['req.headers.authorization']
    }
  })

I created a plugin per the hapi plugin docs:

import { clsProxify, clsProxifyNamespace, setClsProxyValue } from 'cls-proxify'
import * as Pino from "pino";

const logger = Pino();
const loggerCls = clsProxify('clsKeyLogger', logger)

const handler = function (request, h) {
    clsProxifyNamespace.bindEmitter(request);
    clsProxifyNamespace.bindEmitter(request.response);

    clsProxifyNamespace.run(() => {
        const headerRequestID = request.headers.Traceparent
        // this value will be accesible in CLS by key 'clsKeyLogger'
        // it will be used as a proxy for `loggerCls`
        const loggerProxy = {
            info: (msg: string) => `${headerRequestID}: ${msg}`,
        }
        setClsProxyValue('clsKeyLogger', loggerProxy)
    })
};

exports.plugin = {
    name: 'cls-trace-logger',
    register: function (server, options) {
        server.route({ method: 'GET', path: '/test/cls', handler });

        loggerCls.info('My message!');
    }
};

But I get this error:
[```
1597365833550] ERROR (55306 on ip-192-168-2-50.ec2.internal): request error
err: {
"type": "AssertionError",
"message": "can only bind real EEs",
"stack":
AssertionError [ERR_ASSERTION]: can only bind real EEs

how to use with koa1

i use with koa1 like this but it only console 1
it seems clsProxifyKoaMiddleware is not worked

app.use(
    function *(next){
      console.log(1)
      clsProxifyKoaMiddleware(
         'clsKeyLogger',
         ctx => {
           console.log(2);
         }
       )
      yield next;
    }
  );

Fastify Plugin Decorators

Hello!

Your project looks awesome. Just out of curiosity, do you have any plans to add Fastify plugin decorators to it?

Error on Koa.js with bunyan

On this code:

const Koa = require('koa');
const bunyan = require('bunyan');
const {clsProxify} = require('cls-proxify');
const {clsProxifyKoaMiddleware} = require('cls-proxify/integration/koa');

const app = new Koa();

const logger = bunyan.createLogger({
  name: 'cls-test',
  streams: [
    {
      level: 'error',
      stream: process.stderr,
    },
  ]
});

const loggerCls = clsProxify('clsKeyLogger', logger);

app.use(clsProxifyKoaMiddleware('clsKeyLogger', (ctx) => {
  return logger.child({
    requestId: Math.random(),
    level: 'error',
  });
}));

app.use(async (ctx, next) => {
  try {
    await next();
  } catch (err) {
    loggerCls.error(err);
  }
});

app.use(async ctx => {
  ctx.throw(new Error('boom'));
});

app.listen(3000, () => { console.log('Server listening on port', 3000); });

I'm got an error:

 TypeError [ERR_INVALID_ARG_TYPE]: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received undefined
      at new NodeError (node:internal/errors:278:15)
      at WriteStream.Writable.write (node:internal/streams/writable:322:13)

Koa & pino - not work

node: 11.7.0
OS: MacOS

I've created a logger module:

import pino, { Logger } from 'pino';
import { clsProxify } from 'cls-proxify';
import { clsProxifyKoaMiddleware } from 'cls-proxify/integration/koa';
import uuidv4 from '../../common/utils/uuid';

let logger: Logger;

switch (process.env.NODE_ENV) {
    case 'development':
        logger = pino({ prettyPrint: { colorize: true } });
        break;

    case 'production':
        const dest = pino.extreme(); // logs to stdout with no args
        logger = pino(dest);
        break;

    default:
        throw new Error('Logger must be initialized! Set up process.env.NODE_ENV');
}

const loggerCls = clsProxify('clsKeyLogger', logger);

export const middleware = () =>
    clsProxifyKoaMiddleware('clsKeyLogger', () => {
        const headerRequestID = uuidv4(); 
        const loggerProxy = {
            info: (msg: string) => {`${headerRequestID}: ${msg}`,
        };
        return loggerProxy;
    });

setInterval(() => loggerCls && process.env.NODE_ENV === 'production' && loggerCls.flush(), 10000).unref();

export default loggerCls;

and then use it in app

import http from 'http';
import Koa from 'koa';
import send from 'koa-send';
import serve from 'koa-static';
import body from 'koa-body';
import logger, { middleware } from './utils/getServerLogger';

const app = new Koa();
const server = http.createServer(app.callback());

app.use(middleware());
app.use(body());

app.use(async (ctx, next) => {
   logger.info('Now', Date.now());
   await nex();
})

......

there is no above log item in log - everything else is present but not the item above

what is wrong in the code?

No private property forwarding

Hi,

Based on the current implementation, proxy method calls are not able to update the wrapped object private members. It is related to the No private property forwarding topic.

I was wondering if implementing the proposed solution in the above link would be acceptable for this project:

get(target, prop, receiver) {
    const value = target[prop];
    if (value instanceof Function) {
      return function (...args) {
        return value.apply(this === receiver ? target : this, args);
      };
    }
    return value;
}

I am not sure about side-effects and implications this could have for other use cases, but this does fix my issue.

If this is an option, I will gladly submit a PR. I will use a locally patched version of this library until then.

Thank you

What would be the recommended way to use with Hapi and specifically hapi-pino

I read this article about adding tracing to logs and it inspired me to want to enable this inside my hapi/hapi-pino project.

I registered hapi-pino as a hapi plugin just like the docs recommend (below). I'm wondering if it would make more sense to register a separate cls-pino-logger plugin or if it would be recommended to do it somehow inside of hapi-pino? Thanks in advance for any guidance.

await server.register({
    plugin: require('hapi-pino'),
    options: {
      prettyPrint: process.env.NODE_ENV !== 'production',
      // Redact Authorization headers, see https://getpino.io/#/docs/redaction
      redact: ['req.headers.authorization']
    }
  })

I created a plugin per the hapi plugin docs:

import { clsProxify, clsProxifyNamespace, setClsProxyValue } from 'cls-proxify'
import * as Pino from "pino";

const logger = Pino();
const loggerCls = clsProxify('clsKeyLogger', logger)

const handler = function (request, h) {
    clsProxifyNamespace.bindEmitter(request);
    clsProxifyNamespace.bindEmitter(request.response);

    clsProxifyNamespace.run(() => {
        const headerRequestID = request.headers.Traceparent
        // this value will be accesible in CLS by key 'clsKeyLogger'
        // it will be used as a proxy for `loggerCls`
        const loggerProxy = {
            info: (msg: string) => `${headerRequestID}: ${msg}`,
        }
        setClsProxyValue('clsKeyLogger', loggerProxy)
    })
};

exports.plugin = {
    name: 'cls-trace-logger',
    register: function (server, options) {
        server.route({ method: 'GET', path: '/test/cls', handler });

        loggerCls.info('My message!');
    }
};

But I get this error:

1597365833550] ERROR (55306 on ip-192-168-2-50.ec2.internal): request error
    err: {
      "type": "AssertionError",
      "message": "can only bind real EEs",
      "stack":
          AssertionError [ERR_ASSERTION]: can only bind real EEs

How to add co-relationId to pino logger with middy in serverless application using cls-proxify

Our logger.js that uses pino library looks like this:

  import pino from "pino";
   const logger = pino({ name: 'viewLinc', level: process.env.STAGE == 'prod' ? 'info' : 'debug' });
   export default logger;

When using logger, we simply say

    import logger.js
    logger.info("bla bla bla:)

we are also using middy in the application. Middy generates traces with requestId in traces in the ServiceLens but we do not see our custom logs there. For that matter, we will need to add our requestId/co-relationId to the logger object.

Common-middleware.js in our application looks like this:

    export default handler => middy(handler).use([
    loggerMiddleware({ logger: logger }), 
    //.....other middleware 
    httpErrorHandler(), ]);

How can I centrally add co-relationId in my custom logs using cls-proxify so that they will show up in the traces without having to leave it up to the developers to do it?

FR: provide namespace to use

There are cases when the app already has a namespace and it would be very helpful to use the same namespace in cls proxify.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.