krakenjs / shortstop Goto Github PK

Enables use of protocols in configuration.

License: Other

JavaScript 100.00%

shortstop's Introduction

shortstop

Sometimes JSON just isn't enough for configuration needs. Occasionally it would be nice to use arbitrary types as values, but JSON is necessarily a subset of all available JS types. shortstop enables the use of protocols and handlers to enable identification and special handling of json values.

var fs = require('fs');
var shortstop = require('shortstop');

function buffer(value) {
    return new Buffer(value);
}


var resolver, json;
resolver = shortstop.create();
resolver.use('buffer', buffer);
resolver.use('file', fs.readFile);

json = {
    "secret": "buffer:SGVsbG8sIHdvcmxkIQ==",
    "ssl": {
        "pfx": "file:foo/bar",
        "key": "file:foo/baz.key",
    }
};

resolver.resolve(json, function (err, data) {
    console.log(data);
    // {
    //     "secret": <Buffer ... >,
    //     "ssl" {
    //         "pfx": <Buffer ... >,
    //         "key": <Buffer ... >
    //     }
    // }
});

API

shortstop.create([parent]);

parent (Object, optional) - An optional shortstop resolver. Returns a resolver instance.

resolver.use(protocol, handler);

protocol (String) - The protocol used to identify a property to be processed, e.g. "file"
handler (Function) - The implementation of the given protocol with signature function (value, [callback])

This method returns a function when invoked will remove the handler from the stack for this protocol.

resolver.resolve(data, callback);

data (Object) - The object, containing protocols in values, to be processed.
callback (Function) - The callback invoked when the processing is complete with signature function (err, result).

resolver.resolveFile(path, callback);

path (String) - The path to a file which is, or exports, JSON or a javascript object.
callback (Function) - The callback invoked when the processing is complete with signature function (err, result).

Multiple handlers

Multiple handlers can be registered for a given protocol. They will be executed in the order registered and the output of one handler will be the input of the next handler in the chain.

var fs = require('fs'),
var path = require('path'),
var shortstop = require('shortstop');

function resolve(value) {
    if (path.resolve(value) === value) {
        // Is absolute path already
        return value;
    }
    return path.join(process.cwd(), value);
}


var resolver, json;
resolver = shortstop.create();
resolver.use('path', resolve);
resolver.use('file', resolve);
resolver.use('file', fs.readFile);

json = {
    "key": "file:foo/baz.key",
    "certs": "path:certs/myapp"
};

resolver.resolve(json, function (err, data) {
    console.log(data);
    // {
    //     "key": <Buffer ... >,
    //     "certs": "/path/to/my/certs/myapp"
    // }
});

Removing Handlers

When registered, handlers return an unregister function you can call when you no longer want a handler in the chain.

var path = require('path');
var shortstop = require('shortstop');


function resolve(value) {
    if (path.resolve(value) === value) {
        // Is absolute path already
        return value;
    }
    return path.join(process.cwd(), value);
}

var resolver, unuse, json;
resolver = shortstop.create();
unuse = resolver.use('path', resolve);
json = { "key": "path:foo/baz.key" };

resolver.resolve(json, function (err, data) {
    console.log(data);
    // {
    //     "key": "/path/to/my/foo/baz.key"
    // }

    unuse();

    resolver.resolve(json, function (err, data) {
        console.log(data);
        // {
        //     "key": "path:foo/baz.key"
        // }
    });
});

shortstop's People

Contributors

Stargazers

Watchers

Forkers

nvdnkpr indieisaconcept sstedman sumanboss pvenkatakrishnan internetsadboy alexdavis-bf-node aredridel kleopatra999 omni-tools exzyle osukaa isabella232 ivanskydev mengzhou44

shortstop's Issues

can we store function using shortstop?

can we store function using shortstop?
I am curious if that is possible.

Object key order not preserved

Even though object keys in Javascript are not explicitly ordered, most known JS implementations do respect key order (in the order they were created) - same in NodeJS.

However, shortstop handles resolving in a parallel manner, causing the key order to change. It would be good if this would be preserved as expected.

Async Support

Add support for async protocol handlers.

Proposed API

create([parent]);
use(protocol, handler); // handler can be sync or async: function (value, [cb])
resolve(obj, callback);
resolveFile(obj, callback);

The naive implementation of "unuse" is bad news and buggy.

Unless handlers are unused in reverse-order of registration, this impl has unintended consequences/side-effects. (If an item prior to the given item is removed the index value of the later handlers is no longer accurate, thus the wrong element is removed.)

https://github.com/paypal/shortstop/blob/master/lib/resolver.js#L58

Providing opt-in default implementations for common protocols

Shortstop is a generic implementation at the moment, but why didn't we provide default implementations for some of the more basic types which could be opted into.

Happy to PR it, I just wanted to get everyone's opinions first since we haven't chatted about it.

Synchronous resolve?

I'm trying to use shortstop with the config module. I have a default.js file in my config folder where I create and export my configuration object by require()-ing a bunch of JSON files. The JSON values can contain shortstop directives. So before exporting the final configuration object for use by the node-config module, I would like to apply shortstop resolution to it.

I am not able to do this currently because shortstop.resolve is an async method and the module.exports will not wait for it to complete. So is there a Sync variation of the resolve method?

Note: I've already looked into the confit module, but I cannot use it due to my specific requirements.

multiple protocols on one property

Are there any plans on implementing a way to handle multiple nested protocols. Right now on my app I'm using the glob and require handlers and looking for a way to merge those two without having to create a custom requireGlob handler. Looking into something like this:

{
  "files": "require:glob:/myDir/**/*.json"
}

Where the output from the glob handler is then sent into the require. This doesn't have to be the final syntax, it could also be something like:

{
  "files": "glob:/myDir/**/*.json | require"
}

Looking for opinions, I could add the change if we agree on adding something like this. Not really looking into this exact scenario but more into the functionality of having multiple nested protocols.

NPM High vulnerability on async dependency

Detailed Description

npm audit report reported a High vulnerability on the async dependency used in the shortstop dependency

Possible Implementation

For versions 0.x.x, 1.x.x and 2.x.x use an async version >= 2.6.4 for 3.x.x versions use >= 3.2.2
More details can be found here: GHSA-fwr7-v2mv-hh25

Environment Details

Version used: 1.0.3
Environment versions: Node v12.19.0, npm v6.14.13
Operating System and version: OSX 10.15.7 (Catalina)

Protocol Filters

@totherik I recently added the concept of Protocol Filters. It provides the means to register protocols which can be aliased or match a specific pattern via regex.

The use case I have is for when I have values which contain urls ( more on that later ).

Essentially the change is as follow:

resolver.use('http', '^(https?):', handler);                // ( overrides default regex )  or
resolver.use('file', '^({protocol}|fileSync):', handler);   // ( ^(file|fileSync): )        or
resolver.use('select', ['filter'], handler);                // ( ^(select|filter): )

I'm working towards adding generic async support for protocols to allow the loading of remote configuration files and other async based tasks. Without this addition, I would have to do the following.

before

var data = {
    "config1": "request:http://www.domain.com/config.json",
    "config2": "request:https://www.domain.com/config.json"
}

I wanted it to be neater hence adding, filters/aliases.

after

var data = {
    "config1": "http://www.domain.com/config.json",
    "config2": "https://www.domain.com/config.json"
}

This change also changes the regex for determining if a value begins with a valid protocol pattern.

I was in two minds about this, should protocols be validated like normal protocols as per the RFC spec or just any valid characters.

I've intentionally just kept it fairly relaxed for now but Regex's are my "achilles heel" :-) so you may have a better alternative.

I've not yet raised a pull request on this since there are still a few outstanding to be merged in but wanted to share the idea nonetheless.

Code can be seen here: https://github.com/indieisaconcept/shortstop/tree/protocol-filters