Body manipulation and generation

Document updated on Feb 9, 2023

The body generator modifier allows you to craft the body you send to a backend through templates, and enables injecting other values from the user request such as the original body, headers, querystrings or URL parameters.

The body generator does not necessarily need that the endpoint sends data, as it works even when there is no input body from the user. It lets you specify the body content you want to send to the final service, and you can reuse parts of the request (such as headers) to form a new body.

The most common uses cases are:

• Body manipulation or transformation
• Add headers or query strings to the body
• Convert a backend POST into a GET endpoint, while still sending data
• Enrich user’s body with additional data
• Build your gateway without having yet a client (set mock data in the POST)

Configuration for the body generator

The body generator modifier has the following options available:

* Required one of: path , or template

How body generation works?

To use the body generator modifier, you need to write a template (inline as base64 or clear text in an external file). For example, the following sample.json.tmpl represents a body in JSON format you would send to your backend:

{
  "update":{
    "user_id": "{{ .req_params.User }}",
    "email": "{{ .req_body.email }}"
  }
}

When a request to the backend arrives, this template replaces the {{ .req_params.User }} with the {user} value of the the URL (e.g., /foo/{user}), and the {{ .req_body.email }} with an email field passed in the user request body. The final render is the content of the body you will send to the backend.

The example uses a JSON format, but you can write the template using any other format you need, and the configuration option content_type lets the backend know the right type. In this case it should equal application/json to match the content.

Template definition

When you write the content of the body, you do it in a Go text template (similar to Helm, Kubernetes, and other systems). The template engine parses the content and replaces variables with the format {{ .variable }}, but you can use all the power of templates and introduce conditionals, loops, and other checks.

The following variables are available in the template you will use to construct a body:

.req_body

It contains the data sent by the user in the body request. You can reuse the body of the user sent in several formats, to compose the final body you will send to the backend server.

The .req_body is initially empty unless the following requirements are met:

• The template has at least a .req_body declaration
• The Content-Type is declared in the input_headers of the endpoint. The content type is necessary to determine how to parse the request body and make it available to the template. This is not the content_type configuration option you will send to the backend server, although it could match. The following content types are the only ones that will work when submitting data to KrakenD (otherwise, there will be an error):
  • application/json
  • application/xml
  • text/xml
  • application/x-www-form-urlencoded
  • multipart/form-data
  • text/plain

For instance, a user request like:

$curl -XPOST -d '{"foo": "bar"}' -H'Content-Type: application/json' http://localhost:8080/hello

Allows you to use in a template .req_body.foo, which will translate into bar. At the same time you could decide to write a template in XML format using these values and switch to a content_type different when reaching the backend server.

.req_params

It contains all the parameters you have declared in the endpoint as {placeholders}. To access the parameters, use the first letter capitalized.

For instance, an endpoint defined like this:

{
  "endpoint": "/foo/{bar}"
}

Allows you to use in a template .req_params.Bar and contains the value of the request in {bar}.

.req_headers

It contains all the headers allowed in the endpoint, not the ones sent by the user. It means that the endpoint needs to declare in input_headers each header you would like to access. For instance:

{
  "endpoint": "/foo/{bar}",
  "input_headers": ["X-Header"]
}

Allows you to use in a template .req_headers["X-Header"]. Notice that we are NOT accessing this variable as .req_headers.X-Header in this case because it contains the special char -, and that is a minus sign on the template.

When input_headers is set to ["*"], all headers sent by the client are in the variable, although this practice might lead to potential security threads and is discouraged. Add only those that you will actually use.

.req_querystring

It contains all the query strings allowed to pass in the endpoint. As with headers, the endpoint must declare the list in input_query_strings. For instance

{
  "endpoint": "/foo/{bar}",
  "input_query_strings": ["query","limit"]
}

Allows you to use in a template .req_querystring.query or .req_querystring.limit.

When input_query_strings is set to ["*"], then all query strings sent by the client are in the variable, although this practice might lead to potential security threads and is discouiraged.

.req_path

The path that KrakenD will use to connect the backend server. It matches the url_pattern of the configuration.

Body generator modifier example

Let’s show how this works with a testable example.

Supose you have a POST endpoint on KrakenD where a user sends a text, and we want to POST this content modified, along with additional parts of the input to build the body. The user request would be:

$curl -XPOST -d '{"text": "hello"}' -H'Content-Type: application/json' http://localhost:8080/bodygenerator/10

But we want to receive in our backend something like:

$curl -XPOST -d '{"message": "User said hello", "id": 10}' -H'Content-Type: application/json' http://backend/url

Where the text is renamed to message and we also include the id passed in the URL. The reproducible configuration would look like this:

{
  "$schema": "https://www.krakend.io/schema/v2.2/krakend.json",
  "version": 3,
  "host": ["http://localhost:8080"],
  "debug_endpoint": true,
   "endpoints": [
        {
            "endpoint": "/bodygenerator/{id}",
            "method": "GET",
            "input_headers": [ "Content-Type" ],
            "backend": [
                {
                    "url_pattern": "/__debug/test/{id}",
                    "method": "POST",
                    "encoding": "json",
                    "extra_config": {
                        "modifier/body-generator": {
                            "path": "./body.json.tmpl",
                            "content_type": "application/json",
                            "debug": true
                        }
                    },
                    "host": [ "http://localhost:8080" ]
                }
            ]
        }
    ]
}

And the configuration refers to a file body.json.tmpl which would contain the following content:

{
  "id": "{{ .req_params.Id }}",
  "message": "User said {{ .req_body.text }}"
}

When running KrakenD and calling the endpoint you would have the expected replacement. These are the important takeaways from this configuration:

• The path defines where the external template is, using a relative dir ./ based on KrakenD’s working directory, but it can also be an absolute path.
• You can use template instead, but then you need to convert the template above into a base64 string.

The important takeways from the template are:

• The variable req_params (and not the rest) is accessing parameters with the first letter in uppercase (Id).
• All injected variables start with a dot . and use one of the variables defined in the variables section.
• All template blocks are delimited by {{ }}.

Debugging the POST template

While working with the body generator modifier, you might find it useful to set the debug flag to true. This flag (that you should not use in production) outputs the following information in the console (when the debug level is DEBUG):

• All the variables available in the template
• The final generated body, after compiling the template and injecting the variables
• The content type to send to the backend server

Use the flag for faster development! But remove it in production. It is designed for developer reading of the logs (multiline content), and not for machine processing of the lines.

It is also important to look at the rest of the log lines, as in case the templates cannot be rendered or found (the relative path could be different than you expected) you will see lines showing the problem:

KRAKEND DEBUG: [BACKEND: /foo][body-generator] open ./body.json.tmpl: no such file or directory

Embeding templates in base64

You can embed the template in the configuration as a base64 instead of referencing it as an external file. There are several ways you can do this.

Written inline in the template using flexible configuration:

"modifier/body-generator": {
      "template": "{{ `{
          "id": "{{ .req_params.Id }}",
          "message": "User said {{ .req_body.text }}"
          }` | b64enc }}",
      "content_type": "application/json",
      "debug": true
}

As you can see the backtick delimiters write the template as it is, and at the end it pipes it to the b64enc function.

Loaded as a partial template with base64 encoding and flexible configuration:

"modifier/body-generator": {
      "template": "{{ include "body.json.tmpl" | b64enc }}",
      "content_type": "application/json",
      "debug": true
}

Copy paste the value from a terminal:

$base64 -w 0 body.json.tmpl
ewogICJpZCI6ICJ7eyAucmVxX3BhcmFtcy5JZCB9fSIsCiAgIm1lc3NhZ2UiOiAie3sgLnJlcV9ib2R5Lm1lc3NhZ2UgfX0iCn0=

Notice that we add -w 0 because we don’t want new lines that would break the configuration.