News New Look, Same Vision: KrakenD’s Website Redesign

You are viewing a previous version of KrakenD Community Edition (v2.5), go to the latest version

Document updated on Oct 11, 2022

API Throttling and Timeout Management

Being KrakenD, an API Gateway that talks to other services, controlling the waiting times for different aspects is crucial. KrakenD will allow you to fine-tune these settings.

The timeouts can apply to:

  • The duration of the whole pipe (from user request to user response)
  • The HTTP request related timeouts

Additionally, you can control the number of maximum IDLE connections.

Default timeout

The timeout key in the krakend.json at the root level is used to apply a default timeout for those endpoints that do not specify one. The duration you write in the timeout represents the whole duration of the pipe, so it counts the time all your backends take to respond and the processing of all the components involved in the endpoint (the request, fetching data, manipulation, etc.). You can see it is an end-user timeout, the maximum amount of time a user will wait for a response.

{
    "version": 3,
    "timeout": "2000ms"
}

Endpoint timeout

Probably the most convenient way to work with timeouts is by having a default timeout value in the root level with reasonable values. The default timeout will apply to all endpoints not setting a value and then set a timeout to those endpoints having a very different nature (like uploading files or heavy processing).

To do so, place it inside the desired endpoint:

{
    "version": 3,
    "timeout": "2000ms",
    "endpoints": [
        {
            "endpoint": "/default-timeout"
        },
        {
            "endpoint": "/different-timeout",
            "timeout": "1s"
        }
    ]
}

The example above will use 1 second timeout for the /different-timeout endpoint and 2 seconds (expressed in milliseconds) for any the other endpoint. Note that the backend section is omitted for better reading.

Context deadline exceeded

What happens when the timeout is reached? KrakenD will exhaust the specified timeout trying to fetch and process the content, but if that happens, the user will receive an HTTP status code 500 Internal Server Error when there is no content to return.

If you connected to more than one backend and you have at least a valid response, the gateway will return a 200 status code with a partial response (the backends that worked)

The logs will show context deadline exceeded every time that you hit a timeout.

Golden rule for setting timeouts

When setting timeouts, you always have to respect the following rule:

Client timeout > KrakenD timeout > Your API timeout

From the consumer to the provider, the timeouts need to decrease. For instance, your javascript application should have a timeout greater than KrakenD timeouts, and KrakenD should have a greater timeout than your backend API.

HTTP Server and Transport timeouts

In addition to the timeouts, you can configure the specific HTTP transport timeouts and the HTTP server timeout.

For more information, see:

All the settings in these sections work just like the pipeline timeouts. However, if you place the values in an endpoint, they will override the default setting.

Scarf

Unresolved issues?

The documentation is only a piece of the help you can get! Whether you are looking for Open Source or Enterprise support, see more support channels that can help you.

See all support channels