Giter VIP home page Giter VIP logo

loopback4-ratelimiter's Introduction

ARC By SourceFuse logo

npm version Sonar Quality Gate Synk Status GitHub contributors downloads License Powered By LoopBack 4

Overview

A simple loopback-next extension for rate limiting in loopback applications. This extension uses express-rate-limit under the hood with redis, memcache and mongodDB used as store for rate limiting key storage using rate-limit-redis, rate-limit-memcached and rate-limit-mongo

Install

npm install loopback4-ratelimiter

Usage

In order to use this component into your LoopBack application, please follow below steps.

  • Add component to application.
this.component(RateLimiterComponent);
  • Minimum configuration required for this component is given below.

For redis datasource, you have to pass the name of a loopback4 datasource

this.bind(RateLimitSecurityBindings.CONFIG).to({
  name: 'redis',
  type: 'RedisStore',
});

For memcache datasource

this.bind(RateLimitSecurityBindings.CONFIG).to({
  client: memoryClient,
  type: 'MemcachedStore',
});

For mongoDB datasource

this.bind(RateLimitSecurityBindings.CONFIG).to({
  name: 'mongo',
  type:'MongoStore';
  uri: 'mongodb://127.0.0.1:27017/test_db',
  collectionName: 'expressRateRecords'
});
  • By default, ratelimiter will be initialized with default options as mentioned here. However, you can override any of the options using the Config Binding. Below is an example of how to do it with the redis datasource, you can also do it with other two datasources similarly.
const rateLimitKeyGen = (req: Request) => {
  const token =
    (req.headers &&
      req.headers.authorization &&
      req.headers.authorization.replace(/bearer /i, '')) ||
    '';
  return token;
};

......


this.bind(RateLimitSecurityBindings.CONFIG).to({
  name: 'redis',
  type: 'RedisStore',
  max: 60,
  keyGenerator: rateLimitKeyGen,
});

EnabledbyDefault

enabledByDefault option in Config Binding will provide a configurable mode. When its enabled (default value is true),it will provide a way to ratelimit all API's except a few that are disabled using a decorator.

To disable ratelimiting for all APIs except those that are enabled using the decorator, you can set its value to false in config binding option.

this.bind(RateLimitSecurityBindings.CONFIG).to({
  name: 'redis',
  type: 'RedisStore',
  max: 60,
  keyGenerator: rateLimitKeyGen,
  enabledByDefault:false
});
  • The component exposes a sequence action which can be added to your server sequence class. Adding this will trigger ratelimiter middleware for all the requests passing through.
export class MySequence implements SequenceHandler {
  constructor(
    @inject(SequenceActions.FIND_ROUTE) protected findRoute: FindRoute,
    @inject(SequenceActions.PARSE_PARAMS) protected parseParams: ParseParams,
    @inject(SequenceActions.INVOKE_METHOD) protected invoke: InvokeMethod,
    @inject(SequenceActions.SEND) public send: Send,
    @inject(SequenceActions.REJECT) public reject: Reject,
    @inject(RateLimitSecurityBindings.RATELIMIT_SECURITY_ACTION)
    protected rateLimitAction: RateLimitAction,
  ) {}

  async handle(context: RequestContext) {
    const requestTime = Date.now();
    try {
      const {request, response} = context;
      const route = this.findRoute(request);
      const args = await this.parseParams(request, route);

      // rate limit Action here
      await this.rateLimitAction(request, response);

      const result = await this.invoke(route, args);
      this.send(response, result);
    } catch (err) {
      ...
    } finally {
      ...
    }
  }
}
  • This component also exposes a method decorator for cases where you want tp specify different rate limiting options at API method level. For example, you want to keep hard rate limit for unauthorized API requests and want to keep it softer for other API requests. In this case, the global config will be overwritten by the method decoration. Refer below.
const rateLimitKeyGen = (req: Request) => {
  const token =
    (req.headers &&
      req.headers.authorization &&
      req.headers.authorization.replace(/bearer /i, '')) ||
    '';
  return token;
};

.....

@ratelimit(true, {
  max: 60,
  keyGenerator: rateLimitKeyGen,
})
@patch(`/auth/change-password`, {
  responses: {
    [STATUS_CODE.OK]: {
      description: 'If User password successfully changed.',
    },
    ...ErrorCodes,
  },
  security: [
    {
      [STRATEGY.BEARER]: [],
    },
  ],
})
async resetPassword(
  @requestBody({
    content: {
      [CONTENT_TYPE.JSON]: {
        schema: getModelSchemaRef(ResetPassword, {partial: true}),
      },
    },
  })
  req: ResetPassword,
  @param.header.string('Authorization') auth: string,
): Promise<User> {
  return this.authService.changepassword(req, auth);
}
  • You can also disable rate limiting for specific API methods using the decorator like below or use the skip handler
@ratelimit(false)
@authenticate(STRATEGY.BEARER)
@authorize(['*'])
@get('/auth/me', {
  description: ' To get the user details',
  security: [
    {
      [STRATEGY.BEARER]: [],
    },
  ],
  responses: {
    [STATUS_CODE.OK]: {
      description: 'User Object',
      content: {
        [CONTENT_TYPE.JSON]: AuthUser,
      },
    },
    ...ErrorCodes,
  },
})
async userDetails(
  @inject(RestBindings.Http.REQUEST) req: Request,
): Promise<AuthUser> {
  return this.authService.getme(req.headers.authorization);
}

Middleware Sequence Support

As action based sequence will be deprecated soon, we have provided support for middleware based sequences. If you are using middleware sequence you can add ratelimit to your application by enabling ratelimit action middleware. This can be done by binding the RateLimitSecurityBindings.CONFIG in application.ts :

this.bind(RateLimitSecurityBindings.RATELIMITCONFIG).to({
  RatelimitActionMiddleware: true,
});

this.component(RateLimiterComponent);

This binding needs to be done before adding the RateLimiter component to your application. Apart from this all other steps will remain the same.

Skip Handler

By default all the paths are rate limited based on the configuration provided, but can be skipped using the skip handler.

Following is the example of an handler that returns true if the path starts with /obf/ such as /obf/css/style.css, /obf/fonts, /obf/stats etc.

const obfPath = process.env.OBF_PATH ?? '/obf';

this.bind(RateLimitSecurityBindings.CONFIG).to({
  name: RedisDataSource.dataSourceName,
  type: 'RedisStore',
+  skip: (request, response) => {
+    const isOBFSubpath = Boolean(
+      request.path.match(new RegExp(`^/$+{obfPath.replace(/^\//, '')}/.+`)),
+    );
+    return !!isOBFSubpath;
  },
});

Feedback

If you've noticed a bug or have a question or have a feature request, search the issue tracker to see if someone else in the community has already created a ticket. If not, go ahead and make one! All feature requests are welcome. Implementation time may vary. Feel free to contribute the same, if you can. If you think this extension is useful, please star it. Appreciation really helps in keeping this project alive.

Contributing

Please read CONTRIBUTING.md for details on the process for submitting pull requests to us.

Code of conduct

Code of conduct guidelines here.

License

MIT

loopback4-ratelimiter's People

Contributors

akshatdubeysf avatar ankurbansalsf avatar arpit1503khanna avatar barleendhaliwal avatar dependabot[bot] avatar gautam23-sf avatar harshitmitsf avatar jyoti-13 avatar raghavarorasf avatar samarpan-b avatar semantic-release-bot avatar sf-kansara avatar sf-sahil-jassal avatar sfdevops avatar shubhamp-sf avatar surbhi-sharma1 avatar tyagi-sunny avatar yeshamavani avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

loopback4-ratelimiter's Issues

Test case coverage lacking

Describe the bug
Test case coverage missing and not up to the mark

To Reproduce
We should have at least 75% unit test case coverage for this package.

Need refactoring as middleware

Is your feature request related to a problem? Please describe.
As Action-based sequence is now being phased out, we need to use middleware-based sequence.

Describe the solution you'd like
The same package but loaded as middleware :)

Stale Bot missing in the repository

Describe the bug
Currently the issues and PR never closed even if inactive.
They should be closed automatically.

To Reproduce
Steps to reproduce the behavior:

  1. Create a new issue/Pr
  2. Observe it.
  3. Even after no activity it stays open.

Expected behavior
Inactive issues/Pr should be closed automatically.

Request for more detailed and customizable changelog

Is your feature request related to a problem? Please describe.
Right now the changelog created for releases is not well in detail and informative.
Request to generate detailed changelog.

Describe the solution you'd like
Can use different npm packages available

Please add functionality to set multiple configs, ie. Limit requests per minute & per day in 1 request

Is your feature request related to a problem? Please describe.
We want to limit a request 3 per minute, but max 30 per day,

Describe the solution you'd like
Allow multiple options

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Remove support for node v14

Is your feature request related to a problem? Please describe.
Node v14 reaching its end of life this month. Loopback removes support for node v14 in all of its packages we depend on.

Sourceloop packages/services also currently have v12 and v14 as the supported versions.

Describe the solution you'd like
Remove the support for node v14 and v12. And add the support for the latest LTS version v18.

Describe alternatives you've considered
__

Additional context
__

semantic-release : patch release via chore

Describe the bug
Changes for semantic-release
when dependencies are updated with chore type new version is not released

To Reproduce
try updating the dependencies using chore(deps)

Expected behavior
when dependencies are updated with chore type new version must be released

Redis Datasource no longer working in config

Describe the bug

Until recently, to set a redis store for ratelimited, we just had to create a datasource, and pass the name of that loopback4 datasource in the config, in the latest version, the existing datasource can not be used, instead a redis client is expected to be created and that is to be passed in the config.

Expected behavior
The config for Redis type store should accept a datasource name instead of a Redis client.

Correct the changelog Format

Describe the bug
Right now Issue description is not visible in the changelog
To Reproduce
Steps to reproduce the behavior:

  1. Release a new version
  2. check the changelog
  3. Issue Description not visible
  4. Issue link not clickable

Option to apply ratelimiting over a single API

Describe the bug
As of the now, the module provides a way to ratelimit all api except a few that are disabled using a decorator. There should be a way to ratelimit only a few of the many available apis.

Expected behavior
There should be a configurable default mode of ratelimiting for a microservice, if it is false, ratelimiting should be disabled for all APIs except those that are enabled using the decorator.

Support for Middleware Sequence?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

How can I use this with the new Middleware sequence?
I can't register the action the old way anymore.

Describe the solution you'd like
A clear and concise description of what you want to happen.

I need some way to add this to sequence.ts

import {MiddlewareSequence} from '@loopback/rest';

export class MySequence extends MiddlewareSequence {}

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Mongodb connections peak and reach 500

Describe the bug
When using this extension with MongoDB Atlas, the number of connections peak and reaches 500.
To Reproduce
Steps to reproduce the behavior:
1-Here is the configuration used on Loopback:

this.bind(RateLimitSecurityBindings.CONFIG).to({ name: 'MongoDbDataSource', type: 'MongoStore', uri: db_url, collectionName: 'expressRateRecords', windowMs: 60 * 60 * 1000, max: 42, standardHeaders: true, // Return rate limit info in the RateLimit-headers legacyHeaders: false, // Disable theX-RateLimit- headers });
2. @ratelimit(true) on the controller endpoint
3. The number of connections increased until reaching 500 on Atlas.
4. When I turned ratelimiter off on this endpoint
@ratelimit(false)
the problem is solved and the number of connections is kept at 7!
Expected behavior
MongoDB connections shouldn't peak.
Screenshots

image

What should I do to fix this?
Thanks

sync-docs action failing

Describe the bug
check why the sync arc-docs github action is failing

To Reproduce
Steps to reproduce the behavior:

  1. merge a commit to master branch
  2. Observe the logs
  3. Find the reason

Expected behavior
Should actually sync the docs to arc-docs repo

Dist folder should not be committed to the git repo

Describe the bug
Dist folder should not be committed to the git repo

To Reproduce
Steps to reproduce the behavior:
Whenever a new version is released using semantic-release a dist folder is also committed to the repository

Expected behavior
dist folder should not be present in the repo

Interceptor support to avoid sequence changes

Is your feature request related to a problem? Please describe.
The action based sequences are being phased out by loopback4, so it would be great if this library provides a middleware or interceptor to handle the ratelimiting.

Describe the solution you'd like
A interceptor that is bound by the ratelimiter component.

Additional context
Another issue with sequence based approach is that using this component would require a custom sequence which is not always possible.

datasource not found error

I am using the following configuration in application.ts but I am getting that
The key 'datasources.undefined' is not bound to any value in context:

this.bind(RateLimitSecurityBindings.CONFIG).to({
  RatelimitActionMiddleware: true,
  name: 'redis',
  type: 'RedisStore',
  windowMs: 10 * 1000,
  max: 3,
  message: "limit reached",
  standardHeaders: true
});
this.component(RateLimiterComponent as any);

Package Update - loopback4-ratelimiter

Describe the bug
remove all current vulnerability present in loopback4-ratelimiter

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Changelog to be generated

Is your feature request related to a problem? Please describe.
After the release process has been moved to github action changelog is not being generated.

Describe the solution you'd like
Invoke the release notes js file for changelog generation.

Describe alternatives you've considered
Using the same approach as used during jenkins release process

Construct signature return types 'RateLimiterComponent' and 'Component' are incompatible.

Argument of type 'typeof RateLimiterComponent' is not assignable to parameter of type 'Constructor'.
Construct signature return types 'RateLimiterComponent' and 'Component' are incompatible.
The types of 'bindings' are incompatible between these types.
Type 'import("F:/FTEL/Source/Train/demo-loopback/node_modules/loopback4-ratelimiter/node_modules/@loopback/context/dist/binding").Binding[]' is not assignable to type 'import("F:/FTEL/Source/Train/demo-loopback/node_modules/@loopback/context/dist/binding").Binding[]'.
Type 'Binding' is missing the following properties from type 'Binding': source, refresh

49 this.component(RateLimiterComponent);

ratelimit decorator is not working

Hi there, ratelimit decorator is not working.

  • Using sequence it is working fine, but I want to rate limit for a specific api. I have removed it from the sequence.
  • I am using a newly created Redis data source named 'redis'. I can see keys getting created in redis when implemented rate limit through the sequence.
  • As mentioned in #9, windowMs is not getting overriden.

controller.ts

  @ratelimit(true, {
      max: 4
  })
  @post('/add_user', {
    responses: {
      '200': {
        description: 'Add User',
        content: {
          'application/json': {
            schema: {
              type: 'object',
              properties: {
                is_success: {
                  type: 'boolean',
                },
              },
            },
          },
        },
      },
    },
  })
  async addUser(): Promise<{is_success: boolean}> {
      return {is_success: true}
  }

application.ts

    this.component(RateLimiterComponent);
    this.bind(RateLimitSecurityBindings.CONFIG).to({
      name: 'redis',
      max: 10,
      windowMs: 900000, // #9 
      message: 'Too many requests, please try again later.'
    });

package.json

  "engines": {
      "node": ">=10.16"
  },
  "dependencies": {
      "@loopback/authentication": "^7.0.1",
      "@loopback/authentication-jwt": "^0.7.1",
      "@loopback/authorization": "0.7.1",
      "@loopback/boot": "^3.0.1",
      "@loopback/core": "^2.10.1",
      "@loopback/cron": "^0.4.1",
      "@loopback/repository": "^3.0.1",
      "@loopback/rest": "^7.0.1",
      "@loopback/rest-explorer": "^3.0.1",
      "@loopback/service-proxy": "^3.0.1",
      "loopback-connector-kv-redis": "3.0.0",
      "loopback4-ratelimiter": "^2.2.0",
   }
   "devDependencies": {
      "@loopback/build": "^6.2.4",
      "@types/node": "^10.17.35",
      "typescript": "~4.0.2"
    }

@samarpan-b can you please check? Thanks in advance!

Semantic release

Is your feature request related to a problem? Please describe.
Adding semantic release for automatic release of packages.

Describe the solution you'd like
Using npm semantic-release

Describe alternatives you've considered

Additional context

Update all dependencies

  • update all dependencies ( including dev and peer dependencies)
  • use node version 16 for the same

Redis rate limiter

Im implementing a rate limiter in my loopback4 project and I have some issues when trying to customize the values for the rate-limit config. Im using the redis-loopback-connector.
For example:

this.bind(RateLimitSecurityBindings.CONFIG).to({
     name: 'redis',
      max: 20, // limit each IP to 20 requests per windowMs
      windowMs: 1 * 1000, // one second
      keyGenerator: rateLimitKeyGen,
    });

Im trying to set the max parameter to 20 and windowMs to 1000 ms, but it looks like the windowMs parameter is being ignored because when Im testing it is using the default value 6000 ms.
Does anyone know how I can fix this?

loopback version updates

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Setup Release Process via GH Actions

Is your feature request related to a problem? Please describe.
After the sourceloop release processes regression on jenkins the release of packages needs to be done locally.

Describe the solution you'd like
Set up a manually dispatch-able github action to publish releases.

Describe alternatives you've considered
The possible alternate is to publish packages locally but that requires keeping the credentials environment already setup.

Additional context
__

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.