Implement Server Connect API rate limiting

/me waiting for JonL’s NodeJS rate limiter :grimacing:

You really need it? The apps that I am currently working on don’t have public endpoint so I don’t need it at the moment but you should be able to create a middleware extension and use:

1 Like

Yeah. Eff it.

Here you are:

npm install express-rate-limit

/extensions/server_connect/routes/rateLimiter.js
const rateLimit = require('express-rate-limit')

exports.before = function (app) {
    const limiter = rateLimit({
        windowMs: 15 * 60 * 1000, // 15 minutes
        max: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes)
        standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
        legacyHeaders: false, // Disable the `X-RateLimit-*` headers
    })
    app.use(limiter)
}

Result after spamming my endpoint:

429 Too Many Requests

2 Likes

Moved to feature request as this is something we should implement in the core @patrick

6 Likes

This is also a good explanation of rate limiting techniques:

1 Like

I would suggest at least having 2 types of limiters:

  • A global one for SC as a whole(similar to my middleware extension). Options could be added here.
    Wappler 11-08-2022 10.21.19 000067

  • A SC flow one. Options could be added in the settings. Default values would be the ones specified for global limiter.

The SC flow one should be able to override the global one.

1 Like

Since this is now a feature request, I’m listing the options I’m looking forward:

  • Rate limiter SC step
  • Ability to select a custom key for the rate limiter through the data binding picker (e.g.: so I can pick something from $_SESSION, or the default $_SERVER.REMOTE_ADDR for regular IP address limiting)
  • Ability to define window time (or equivalent) of the rate limiter (data binding picker)
  • Ability to define max requests in window time (or equivalent) of the rate limiter (data binding picker)
  • Ability to use Redis for rate-limiting (for distributed rate-limiting)

If it reduces complexity, we could skip the global rate limiter, because one could put a rate limiter in the Globals steps:

And one might want to use several rate limiting steps simultaneously (e.g.: IP address based, and then session based), and these should not void one another

3 Likes

A bit off topic :

Could you please tell a bit more about this? Are you securing some how your API calls or there is an option so that API call would be possible only while visiting the website, not just via curl or Postman?

All the SC flows(except auth SC flows) are secured so nobody can run them if they are not logged in or don’t have permission to do it. First step is a library flow with a security restrict action.

1 Like

Do you need to apply some sort of logic in the SC flow before running the limiter step?

Yes, different rate limits depending on the user’s role/plan/subscription

1 Like

Got it. Makes sense.

Bump, please vote :smiley:

Hi! Any news about this? Need to create some public endpoints for end users right now.
Anyone any tips to keep in mind?

2 Likes

Me too, public endpoints are on my agenda, rate limiting for this is critical.

1 Like

Bump... I'm working on an API right now that will have both public and private access. I'd like to rate limit the public side of it so this tool would be very useful.

Also bump. A very common use case for this is a support ticket request (or password reset) for external users that can't login/signup.

Internal authenticated users can easily make 50+ get/post API requests in 3 seconds legitimately, while I really wouldn't want someone on the outside making 50+ support tickets in 3 seconds ha. This fuzziness makes the simple global limiter approach in the express-rate-limit middleware useless.

I know instead of app.use(limiter) we can call app.post(‘/limitedAPIRoute’, limiter, (req, res, next) => {}) to manual call-out each API we need instead, but when adding in different limits per different API's etc, the js becomes more of a config file we need to mentally keep updated everytime an API gets moved/renamed/destroyed/etc in the native solution.

5 Likes

How do you differentiate between internal (logged in users only?) and external?

Do you use the security provider storing a cookie or do you have other authentication api header?

For the rate limiting to work efficiently, you want to have it at the highest level, so preferably globally so suspicious calls can be denied directly without causing too much server load.

But for that to work users should be divided in groups as quickly and as easy as possible.

Also reminder of my suggestions where I solve such challenges @George:

1 Like