Extended Flexible Configuration

Document updated on Aug 23, 2023

The Extended Flexible Configuration allows you to express your KrakenD configuration using multiple files, reusing code blocks, and introduces a templates system. It is called “Extended” because it uses an engine more capable than its open source alternative, but it remains 100% compatible.

In its simplest form, you can do things like:

{
    "version": 3,
    "extra_config": {
        "$ref": "./service/extra_config.json"
    }
}

As you can guess, in the example above, the $ref is replaced by the content of the external file, much like it works in the JSON Schema specification.

You may need this feature when you have different environments with slight differences, work with a group of people, or avoid copying and pasting configurations in the various endpoints.

There are also what we call settings, files with your custom data structures that you can access in a variable tree. For instance, if you have a student.json or student.yaml with the content:

name: john

Then you could access its attributes like this:

{
    "version": 3,
    "extra_config": {
        "$ref": "{{ .student.name }}/service/extra_config.json"
    }
}

As you can see from the example above, we have introduced the concept of templates now, a very simple one. We are loading the extra_config.json file from a dynamic location that is dictated by a file student.yaml, in this case from john/service/extra_config.json.

But templates can get very complex. You can use the syntax of the Go templates, which opens the door to much more flexible configurations. Here’s another example:

{
    "version": 3
    {{ if .student }}
    ,{{ template "service/components.tmpl" .student }}
    {{ end }}
}

The example above is a template with a conditional, rendering into a JSON file, but you can encode your configuration files in any of the supported formats (json, yaml, toml, etc.), as the template engine is agnostic of its contents.

See in the following sections how to work with templates.

Configuration

Activating the Extended Flexible Configuration requires creating a behavioural file that defines how the engine parses the injected content. This file sets things like the paths to your content or options to parse it.

Copy and paste the following JSON file and save it with any name, for instance flexible_config.json:

 {
    "settings": {
        "paths": ["settings"],
        "allow_overwrite": true,
        "dir_field_prefix": "",
        "allowed_suffixes": [
            ".yaml",
            ".json"
        ]
    },
    "partials": {
        "paths": ["partials"]
    },
    "templates": {
        "paths": ["templates"]
    },
    "out": "result.json"
}

Create an initial directory structure to support what you defined in the paths above:

$mkdir -p {partials,settings,templates}

Now you can run the software using krakend run or krakend check and pass the environment variable FC_CONFIG=flexible.config.json (pointing to the filename you chose). If you come from the open-source version of the flexible configuration, you can delete any previous FC_ environment variables.

For instance, if your configuration is in a krakend.tmpl file:

$docker run --rm -v "$PWD:/etc/krakend/" -e "FC_CONFIG=/etc/krakend/flexible_config.json" krakend/krakend-ee check -c krakend.tmpl
Parsing configuration file: krakend.tmpl
Syntax OK!

Finally, you can debug the hierarchy of the settings tree by passing an environment variable FC_DEBUG=true.

This is all you need to start using the Extended flexible configuration.

The behavior file

The Extended Flexible Configuration behaves according to its configuration file. As you have seen above, you must express it in JSON format.

The structure of this file represents the following:

• settings defines where are your data sources and several options to parse them.
• partials defines from where to load snippets of text that don’t evaluate.
• templates defines from where to load the Go templates.
• out sets the filename that saves the resulting configuration after rendering the template.

Settings

Settings are files expressed in JSON, YAML, ENV, TOML, INI or properties files (Java). Their expected extensions are .json, .yaml, .env, .yml, .toml, .ini, .prop, .properties, or .props.

Settings option paths

The paths attribute in settings defines the list of all data sources, and they can be absolute directories (e.g.: /etc/krakend/settings), relative to the working dir (e.g., ./settings) or individual files (e.g.,: ./override.yaml). Directories are included recursively, with all the contents of subfolders. For instance, you could have:

{
    "settings": {
        "paths": [
            "./settings/common",
            "./settings/production",
            "override.yaml"
        ]
    }
}

When all directories and files are parsed, you can access them in the templates using the dot notation .. It means that if you, for instance, have a file under ./settings/production/a/b/c/file.json and you defined the paths as above, you can access its contents in the templates as {{ .a.b.c.file }} (notice the starting dot). All contents under the paths you defined are placed at the root level of the settings tree.

You can add multiple files and directories in the paths. The engine will load recursively and, in the order defined, all files and subdirectories you have specified in the paths. Their content is merged if you have repeated file names with different extensions (e.g.: file.yaml and file.json).

Settings option allow_overwrite

If there are conflicts when converting directories and file contents into the settings tree (they use the same name path), the allow_overwrite flag lets you define what you want to do. When overwrite is allowed, the last parsed file replaces any conflicting variables in the tree.

There might also be directory names that clash with file names or keys inside the data structures. For instance, you have ./settings/foo.json and ./settings/foo/common.json. This would resolve to {{ .foo }} and {{ .foo.common }}. The last one could replace a key common if it exists in foo.json.

Settings option dir_field_prefix

You have the opportunity to rename the paths of directories with a prefix. For instance, if you use "dir_field_prefix": "dir_", then the same scenario would resolve to {{ .foo }} and {{ .dir_foo.common }} which gives no room to conflict.

Settings option allowed_suffixes

And finally, you can have more files in the settings directories than the flexible configuration will load. To filter which extensions or ending paths you want to load, you can use the allowed_suffixes list. For instance:

{
    "settings": {
    "allowed_suffixes": [
            ".yaml",
            ".prod.json"
        ]
    }
}

The example above would only load YAML files or that end in .prod.json.

Partials

The partials are treated as raw text files, and they are inserted in the placeholder “as is”, and no template evaluation of its content is performed.

The paths list accepts one single directory because, unlike settings and templates, you specify the full relative path when invoked. E.g.: {{ include "dir1/dir2/file.txt" }}

{
    "partials": {
        "paths": [
            "./partials"
        ]
    }
}

Templates

When you start KrakenD, the configuration file passed with the -c flag is treated as the base Go template (documentation), and you can make use of all the power the template engine brings.

The templates attribute in the behavior configuration defines from where to load Go templates that are compiled into the final configuration file.

You must add a paths entry inside the templates setting. It is also a list that defines multiple directories or individual template files. For instance:

{
    "templates": {
        "paths": [
            "templates",
            "some_file.tmpl"
        ]
    }
}

In the example above, we pass a templates folder and an individual template file some_file.tmpl.

All paths are traversed recursively, meaning that all insider templates are also available if there are subfolders. The relative path is removed from the template name when there are subdirectories.

For instance, a directory structure like dir1/dir2/file.tmpl lets you load the template as {{ template "file.tmpl" }}, and as you can see, there is no dir1 or dir2 when you reference it.

When there are conflicts (you have the same filename in different folders), you will see a warning in the logs, and the engine will rename the template to {{ template "dir2_file.tmpl"}}, using the last directory name as a prefix. If there was also a conflict, you would have {{ template "dir1_dir2_file.tmpl"}}, and so on. But unless there are conflicts, the template base name is all you need.

To start working with templates, read Flexible Configuration Templates

Out file

Our final attribute of the behavior file is the out. It specifies a filename where the result of compiling the templates is stored. The directory and the filename you use must be writeable by the KrakenD process. It let’s you see the resulting file after rendering all the templates. It has no other purpose than debugging the resulting template.

For instance:

{
    "out": "result.json"
}

$ref: Reference external files

One of the main differences between the Extended Flexible Configuration and the open source one is that you can reference other configuration files using a reference, even if you don’t want to use Go templates.

The $ref keyword will inject the content of the referenced file, allowing you to work with non-template files (but the templates are welcome, too!). It does not matter if you have the configuration in YAML, INI, or JSON. For instance, when you add a $ref keyword, the content of the referenced file will replace the $ref block.

For instance, given that you have a websockets.json as follows:

{
    "url_pattern": "/ws/{room}",
    "disable_host_sanitize": true,
    "host": [ "ws://chat:8888" ]
}

You can include it in the configuration as follows:

{
  "endpoint": "/chat/{room}",
  "backend": [
    { "$ref": "./backends/websockets.json" }
  ]
}

And KrakenD resolves this to:

{
  "endpoint": "/chat/{room}",
  "backend": [
    {
        "url_pattern": "/ws/{room}",
        "disable_host_sanitize": true,
        "host": [ "ws://chat:8888" ]
    }
  ]
}

In addition, you can also use data pointers, so you can use # to traverse a specific route inside the referenced file. For instance:

{
  "endpoint": "/chat/{room}",
  "backend": [
    {
        "url_pattern": "/ws/{room}",
        "disable_host_sanitize": true,
        "host": { "$ref": "./backends/websockets.json#host" }
    }
  ]
}

In the example above, we import just one field from the referenced file.

Using $ref compared to template syntax is better for maintenance because the complexity is much lower.

Testing the configuration

We recommend using a Docker compose file to work faster with flexible configuration.

Save the following docker-compose.yml and do a docker-compose up. This setup with the :watch image will allow you to work locally with the latest version of KrakenD and apply the changes automatically whenever you change a source file.

version: "3"
services:
  krakend:
    image: krakend/krakend-ee:watch
    volumes:
      - "./:/etc/krakend/"
    environment:
      - FC_CONFIG=flexible_config.json
      - FC_DEBUG=true
    command: ["run","-dc","krakend.tmpl"]

As the flexible configuration is composed of several pieces, it’s easy to make a mistake at some point. Test the syntax of all the files with the krakend check command and pay attention to the output to verify there aren’t any errors. When there are errors, the output contains information to help you resolve it, e.g.:

ERROR parsing the configuration file: loading flexible-config settings:
- backends.json: invalid character '}' looking for beginning of object key string

The setting out in the behavior file (flexible_config.json above) writes the content of the final file in a known path, so you can check its contents at any time.

If you don’t use docker-compose, you can also use flexible configuration as follows:

$FC_CONFIG=flexible_config.json \
FC_DEBUG=true \
krakend check -t -d -c "$PWD/krakend.tmpl"