Document updated on Jan 28, 2022

Stateless Cluster Throttling: Optimize API Performance

The stateless rate-limiting (service Enterprise or endpoint types) is the recommended approach for almost all scenarios. As the API Gateway does not have any centralization, the limits apply individually to each running instance of KrakenD.

Working in a cluster implies applying the limits taking into account the deployment size. For instance, if you want to use a limit of 100 reqs/s in a specific endpoint, the configuration will need to consider the number of nodes in the cluster.

Let’s say that you have a cluster deployed with three instances of KrakenD. If you write a max_rate=100 in the configuration, your ecosystem is limiting to 300 reqs/s in total as three servers are receiving balanced traffic.

When you have to do small math like this, through the Flexible Configuration, you might inject environment variables when starting KrakenD with the total number of machines you have. For instance:

{
    "endpoint": "/limited-endpoint",
    "extra_config": {
      "qos/ratelimit/router": {
          "max_rate": {{ env "NUM_PODS" | div 100 }}
        }
    }
}

You can execute krakend like this:

$FC_ENABLE=1 \
FC_OUT=compiled.json \
NUM_PODS=3 krakend check -c krakend.json

And you get in the compiled.json the following content:

{
    "endpoint": "/limited-endpoint",
    "extra_config": {
      "qos/ratelimit/router": {
          "max_rate": 33
        }
    }
}