havsar / node-ts-cache Goto Github PK

View Code? Open in Web Editor NEW

49.0 3.0 24.0 651 KB

Simple and extensible caching module supporting decorators

Home Page: https://www.npmjs.com/package/node-ts-cache

License: MIT License

TypeScript 100.00%

node-ts-cache's Introduction

node-ts-cache

Simple and extensible caching module supporting decorators.

Install

npm install node-ts-cache

Note: The underlying storage layer must be installed separately.

Storages

Storage	Install
memory	`npm install node-ts-cache-storage-memory`
node-fs	`npm install node-ts-cache-storage-node-fs`
ioredis	`npm install node-ts-cache-storage-ioredis`

Usage

With decorator

Caches function response using the given options. Works with the above listed storages. By default, uses all arguments to build an unique key.

@Cache(container, options)

options:
- ttl: (Default: 60) Number of seconds to expire the cachte item
- isLazy: (Default: true) If true, expired cache entries will be deleted on touch. If false, entries will be deleted after the given ttl.
- isCachedForever: (Default: false) If true, cache entry has no expiration.
- calculateKey(data => string): (Default: JSON.stringify combination of className, methodName and call args)
  - data:
    - className: The class name for the method being decorated
    - methodName: The method name being decorated
    - args: The arguments passed to the method when called

Note: @Cache will consider the return type of the function. If the return type is a thenable, it will stay that way, otherwise not.

import { Cache, CacheContainer } from 'node-ts-cache'
import { MemoryStorage } from 'node-ts-cache-storage-memory'

const userCache = new CacheContainer(new MemoryStorage())

class MyService {
    @Cache(userCache, {ttl: 60})
    public async getUsers(): Promise<string[]> {
        return ["Max", "User"]
    }
}

Directly

import { CacheContainer } from 'node-ts-cache'
import { MemoryStorage } from 'node-ts-cache-storage-memory'

const myCache = new CacheContainer(new MemoryStorage())

class MyService {
    public async getUsers(): Promise<string[]> {
        const cachedUsers = await myCache.getItem<string[]>("users")

        if (cachedUsers) {
            return cachedUsers
        }

        const newUsers = ["Max", "User"]

        await myCache.setItem("users", newUsers, {ttl: 60})

        return newUsers
    }
}

Logging

This project uses debug to log useful caching information. Set environment variable DEBUG=node-ts-cache to enable logging.

Development & Testing

This project follows the monorepo architecture using lerna. To start development and run tests for all the packages, run:

git clone [email protected]:havsar/node-ts-cache.git
cd node-ts-cache
npm i
npm run bootstrap
npm run test

node-ts-cache's People

Contributors

Stargazers

Watchers

node-ts-cache's Issues

how to use redisCluster with ioRedis

Hello, in ioredis, I only see the instance of redis, not the instance of cluster. Can I use rediscluster at present?

If you cache a result value that's a false boolean value, the cache will always miss

I created a cached method using the decorator and backed by NodeFsStorage. If the method returns a boolean that you want to cache, and the boolean value is false, the cache will always get hit. I'd expect it to respect the ttl.

TTL is defined as seconds but used as milliseconds

in /packages/core/src/cache-container/cache-container.ts

the finalOptions.ttl is defined in seconds, but used in setTimeout() method, which assumes milliseconds.
If you are not using lazy cache eviction, this causes an item to be removed after 60 milliiseconds instead of 60 seconds

if (!finalOptions.isLazy) {
    setTimeout(() => {
        this.unsetKey(key)

        debug(`Expired key ${key} removed from cache`)
    }, finalOptions.ttl)
}

expose information "if" ttl is over

Currently, as I see it, you implicitly get information about the ttl being over, as soon as the cache returns undefined.

I'd prefer a solution where I additionally explicitly get some meta-data for the following use case:

If isLazy is enabled, I will get a value back. But I may want to trigger some refetching in the background to rehydrate the cache.

Property 'promisifyAll' does not exist on type 'typeof Bluebird'

Hi. I've installed the module but i get this when try to instantiate the cache:

[13:55:46] Error: TypeScript error: node_modules/node-ts-cache/src/storages/fsjsonstorage.ts(4,21): Error TS2339: Property 'promisifyAll' does not exist on type 'typeof Bluebird'. at formatError (C:\Users\Loki\AppData\Roaming\npm\node_modules\gulp-cli\lib\versioned\^4.0.0\format-error.js:20:10) at Gulp.<anonymous> (C:\Users\Loki\AppData\Roaming\npm\node_modules\gulp-cli\lib\versioned\^4.0.0\log\events.js:31:15)

Property 'promisifyAll' does not exist on type 'typeof Bluebird'

node_modules/node-ts-cache/src/storage/fs.json.storage.ts:4:21 - error TS2339: Property 'promisifyAll' does not exist on type 'typeof Bluebird'.

4 const Fs = Bluebird.promisifyAll(require('fs'))
                      ~~~~~~~~~~~~

node_modules/node-ts-cache/src/storage/redis.storage.ts:7:10 - error TS2339: Property 'promisifyAll' does not exist on type 'typeof Bluebird'.

7 Bluebird.promisifyAll(Redis.RedisClient.prototype)
           ~~~~~~~~~~~~

node_modules/node-ts-cache/src/storage/redis.storage.ts:8:10 - error TS2339: Property 'promisifyAll' does not exist on type 'typeof Bluebird'.

8 Bluebird.promisifyAll(Redis.Multi.prototype)

NodeJS version: 10.16.3
typescript version: 3.4.4

The error is happening when I am trying to compile my typescript code.

cleanup todos

imo this repository could use a good cleanup:

remove lerna and just use workspaces - lerna is dead and not necessary for a project of this scale (npm and yarn support workspaces out of the box)
move tests next to code
take a close look onto the (dev)dependencies - I think some should be dev&peers, some are unnecessary alltogether
clean up tests to use jest for everything, remove assert and sinon

Add option for no expiration

I realize we can implement our own strategies, but as a general addition to the ExpirationStrategy, I think it would be nice to have an option to cache indefinitely. This could also skip a check on expiration date.

PS - Great library!!

getting typescript compilation error

Hi. I've installed the module but i get the attached issue when I tried to build my app.

Please do let me know on how to resolve this.

FYI:
node: v10.16.0
tsc: 3.7.2

how it works if user refresh the page

Hi,
consider a situation we have thousands of items to be cached , but there is an option on the page that users can go to a second page and can return to that page again , are those cached objects still available ? or if user hit refresh what would happen to the cached objects ?

how to update cache

i was use the cache with decorator, when i update the DB data and how to update the cache ?

[Bug] Using inside a NodeJS project - error TS2300: Duplicate identifier 'beforeEach'

Including this inside my TypeScript-based NodeJS project, I started receiving the errors below. After some digging around, I noticed that @types/mocha is declared as a dependency of this project.

Is it actually needed for some important purposes? I'm sure it's not a blocking point for many of the users of this library since they'll not use it in the same context that I do, but still, somebody somewhere may experience the same. Any chance we could drop that dependency?


> [email protected] tsc /home/ubuntu/www/Private/RSS-News-Feed-Portal/backend
> node ./node_modules/typescript/bin/tsc

node_modules/@types/mocha/index.d.ts:2680:13 - error TS2300: Duplicate identifier 'beforeEach'.

2680 declare var beforeEach: Mocha.HookFunction;
                 ~~~~~~~~~~

  ../../../node_modules/@types/jasmine/index.d.ts:59:18
    59 declare function beforeEach(action: ImplementationCallback, timeout?: number): void;
                        ~~~~~~~~~~
    'beforeEach' was also declared here.
  ../../../node_modules/@types/jasminewd2/index.d.ts:13:18
    13 declare function beforeEach(action: (done: DoneFn) => Promise<void>, timeout?: number): void;
                        ~~~~~~~~~~
    and here.

node_modules/@types/mocha/index.d.ts:2698:13 - error TS2300: Duplicate identifier 'afterEach'.

2698 declare var afterEach: Mocha.HookFunction;
                 ~~~~~~~~~

  ../../../node_modules/@types/jasmine/index.d.ts:66:18
    66 declare function afterEach(action: ImplementationCallback, timeout?: number): void;
                        ~~~~~~~~~
    'afterEach' was also declared here.
  ../../../node_modules/@types/jasminewd2/index.d.ts:14:18
    14 declare function afterEach(action: (done: DoneFn) => Promise<void>, timeout?: number): void;
                        ~~~~~~~~~
    and here.

node_modules/@types/mocha/index.d.ts:2714:13 - error TS2300: Duplicate identifier 'describe'.

2714 declare var describe: Mocha.SuiteFunction;
                 ~~~~~~~~

  ../../../node_modules/@types/jasmine/index.d.ts:23:18
    23 declare function describe(description: string, specDefinitions: () => void): void;
                        ~~~~~~~~
    'describe' was also declared here.

node_modules/@types/mocha/index.d.ts:2735:13 - error TS2300: Duplicate identifier 'xdescribe'.

2735 declare var xdescribe: Mocha.PendingSuiteFunction;
                 ~~~~~~~~~

  ../../../node_modules/@types/jasmine/index.d.ts:25:18
    25 declare function xdescribe(description: string, specDefinitions: () => void): void;
                        ~~~~~~~~~
    'xdescribe' was also declared here.

node_modules/@types/mocha/index.d.ts:2749:13 - error TS2300: Duplicate identifier 'it'.

2749 declare var it: Mocha.TestFunction;
                 ~~

  ../../../node_modules/@types/jasmine/index.d.ts:34:18
    34 declare function it(expectation: string, assertion?: ImplementationCallback, timeout?: number): void;
                        ~~
    'it' was also declared here.
  ../../../node_modules/@types/jasminewd2/index.d.ts:10:18
    10 declare function it(expectation: string, assertion?: (done: DoneFn) => Promise<void>, timeout?: number): void;
                        ~~
    and here.

node_modules/@types/mocha/index.d.ts:2770:13 - error TS2300: Duplicate identifier 'xit'.

2770 declare var xit: Mocha.PendingTestFunction;
                 ~~~

  ../../../node_modules/@types/jasmine/index.d.ts:44:18
    44 declare function xit(expectation: string, assertion?: ImplementationCallback, timeout?: number): void;
                        ~~~
    'xit' was also declared here.
  ../../../node_modules/@types/jasminewd2/index.d.ts:12:18
    12 declare function xit(expectation: string, assertion?: (done: DoneFn) => Promise<void>, timeout?: number): void;
                        ~~~
    and here.

../../../node_modules/@types/jasmine/index.d.ts:23:18 - error TS2300: Duplicate identifier 'describe'.

23 declare function describe(description: string, specDefinitions: () => void): void;
                    ~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2714:13
    2714 declare var describe: Mocha.SuiteFunction;
                     ~~~~~~~~
    'describe' was also declared here.

../../../node_modules/@types/jasmine/index.d.ts:25:18 - error TS2300: Duplicate identifier 'xdescribe'.

25 declare function xdescribe(description: string, specDefinitions: () => void): void;
                    ~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2735:13
    2735 declare var xdescribe: Mocha.PendingSuiteFunction;
                     ~~~~~~~~~
    'xdescribe' was also declared here.

../../../node_modules/@types/jasmine/index.d.ts:34:18 - error TS2300: Duplicate identifier 'it'.

34 declare function it(expectation: string, assertion?: ImplementationCallback, timeout?: number): void;
                    ~~

  node_modules/@types/mocha/index.d.ts:2749:13
    2749 declare var it: Mocha.TestFunction;
                     ~~
    'it' was also declared here.

../../../node_modules/@types/jasmine/index.d.ts:44:18 - error TS2300: Duplicate identifier 'xit'.

44 declare function xit(expectation: string, assertion?: ImplementationCallback, timeout?: number): void;
                    ~~~

  node_modules/@types/mocha/index.d.ts:2770:13
    2770 declare var xit: Mocha.PendingTestFunction;
                     ~~~
    'xit' was also declared here.

../../../node_modules/@types/jasmine/index.d.ts:59:18 - error TS2300: Duplicate identifier 'beforeEach'.

59 declare function beforeEach(action: ImplementationCallback, timeout?: number): void;
                    ~~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2680:13
    2680 declare var beforeEach: Mocha.HookFunction;
                     ~~~~~~~~~~
    'beforeEach' was also declared here.

../../../node_modules/@types/jasmine/index.d.ts:66:18 - error TS2300: Duplicate identifier 'afterEach'.

66 declare function afterEach(action: ImplementationCallback, timeout?: number): void;
                    ~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2698:13
    2698 declare var afterEach: Mocha.HookFunction;
                     ~~~~~~~~~
    'afterEach' was also declared here.

../../../node_modules/@types/jasminewd2/index.d.ts:10:18 - error TS2300: Duplicate identifier 'it'.

10 declare function it(expectation: string, assertion?: (done: DoneFn) => Promise<void>, timeout?: number): void;
                    ~~

  node_modules/@types/mocha/index.d.ts:2749:13
    2749 declare var it: Mocha.TestFunction;
                     ~~
    'it' was also declared here.

../../../node_modules/@types/jasminewd2/index.d.ts:12:18 - error TS2300: Duplicate identifier 'xit'.

12 declare function xit(expectation: string, assertion?: (done: DoneFn) => Promise<void>, timeout?: number): void;
                    ~~~

  node_modules/@types/mocha/index.d.ts:2770:13
    2770 declare var xit: Mocha.PendingTestFunction;
                     ~~~
    'xit' was also declared here.

../../../node_modules/@types/jasminewd2/index.d.ts:13:18 - error TS2300: Duplicate identifier 'beforeEach'.

13 declare function beforeEach(action: (done: DoneFn) => Promise<void>, timeout?: number): void;
                    ~~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2680:13
    2680 declare var beforeEach: Mocha.HookFunction;
                     ~~~~~~~~~~
    'beforeEach' was also declared here.

../../../node_modules/@types/jasminewd2/index.d.ts:14:18 - error TS2300: Duplicate identifier 'afterEach'.

14 declare function afterEach(action: (done: DoneFn) => Promise<void>, timeout?: number): void;
                    ~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2698:13
    2698 declare var afterEach: Mocha.HookFunction;
                     ~~~~~~~~~
    'afterEach' was also declared here.


Found 16 errors.

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] tsc: `node ./node_modules/typescript/bin/tsc`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] tsc script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/ubuntu/.npm/_logs/2019-02-15T14_10_04_123Z-debug.log

interface RedisClient -> getAsync

Where's the implementation of interface RedisClient which is defined in custom.d.ts? or why those 4 methods are marked with ??

storage-node-fs with string parameter

Error when trying to cache a method with a string parameter

Example:

import { Cache, CacheContainer } from 'node-ts-cache';
import { NodeFsStorage } from 'node-ts-cache-storage-node-fs';
const userCache = new CacheContainer(new NodeFsStorage('cache/example.json'));

class MyService {
  @Cache(userCache, { ttl: 10 })
  public async getUser(name) {
    if (name == 'name1') {
      return { name: 'LONGGTEXTTTTTTTTTT' };
    } else if (name == 'name2') {
      return undefined;
    }
  }
}
const s = new MyService();
s.getUser('name1').then(console.log);
s.getUser('name2').then(console.log);

Error:

at JSON.parse (<anonymous>)
    at NodeFsStorage.<anonymous> (/home/node_modules/node-ts-cache-storage-node-fs/dist/index.js:111:54)

The cache filed generated is invalid:

{"MyService:getUser:[\"name2\"]":{"meta":{"ttl":10000,"createdAt":1623514436658}}}content":{"name":"LONGGTEXTTTTTTTTTT"}}}

Use real TTL with Redis

For disk / memory usage improvement with redis I think it's better to not handle the TTL manually.
And also a TTL in milliseconds would be better imho.

race condition / dead locks

Hi,
I'm experencing some weird dead locks since I use node-ts-cache. I guess there must be some kind of race condition when setting / getting caches in parallel. Anyone else experienced this so far?

I'm looking into it, just want to clearify if someone else found something about this already.

regards
simon

How to clear cache for a single item?

I found there is a function clear() and expect it to clear the full cache. But how to I clear the cache only for a single value?

If this is not integrated please consider it as a feature request.

myCache.clearItem('key');