Document updated on Jun 13, 2020
Flexible Configuration: template-based config
The Flexible Configuration allows you to declare the configuration using multiple files and use a templates system, opening the door to multi-environment configurations and code reuse.
The Flexible Configuration enables a template processor based on Go templates and is enriched with Sprig functions and KrakenD functions.
You can encode your configuration files in any of the supported formats (json
, yaml
, toml
, etc.), as the template is agnostic of its contents.
Use the Flexible Configuration when you need to:
- Split an extensive configuration into several files
- Inject variables, like environment settings, into the configuration
- Use placeholders and reusable code blocks to avoid repeating code
- Organize your code better when multiple developers are modifying the gateway
- Manage KrakenD using multiple repositories (and merging the files in the CI)
- Track changes, avoid conflicts, and review code more easily
- Have the full power of a templates system!
How it works
The activation of the Flexible Configuration requires injecting the environment variable FC_ENABLE=1
when running a krakend run
or check
command. Still, you can use additional variables depending on the features you’d like to enable.
Flexible Configuration variables
The list of all recognized environment variables is:
FC_ENABLE=1
: Activates Flexible Configuration. You can use1
or any other value (but0
won’t disable it!). The file passed with the--config
flag is the base template and contains the references to any other templates.FC_TEMPLATES=path/to/templates
: The path to thetemplates
directory. These are evaluated using the Go templating system.FC_SETTINGS=path/to/settings
: The path to thesettings
directory. Settings are JSON files that you can use to fill values in the templates, much similar to env files in other applications, but richer as you can use multiple files, structures, and nesting.FC_PARTIALS=path/to/partials
: The path to thepartials
directory. Partial files are pieces of text that DON’T EVALUATE, and they are inserted in the placeholder “as is”.FC_OUT=file.json
: Saves the resulting configuration after rendering the template. It’s required when you don’t use JSON content (e.g.,FC_OUT=krakend.yml
), or when you need to pass the output to another program (likecheck --lint
).
For instance, let’s write a simple template simple.tmpl
(go template emulating a json format):
{
"version": {{add 2 1}}
}
If the template system works, the server will start with a value "version": 3
. We can test it with (Docker example):
Execute a simple template
$docker run --rm -v "$PWD:/etc/krakend/" -e "FC_ENABLE=1" -e "FC_OUT=result.json" krakend/krakend-ee check -c simple.tmpl
Parsing configuration file: simple.tmpl
Syntax OK!
We know that it works because KrakenD will fail with a different version
value, but we can also check the contents of result.json
for debugging purposes.
Let’s now introduce the rest of the optional variables. For instance, let’s assume you decided to organize your code as follows:
.
├── krakend.tmpl
└── config
├── partials
│ └── file.txt
├── templates
│ ├── telemetry.tmpl
│ └── endpoints.tmpl
└── settings
├── prod
| └── urls.json
└── dev
└── urls.json
Then you could run KrakenD from the terminal with this command:
Enabling flexible configuration with your custom dirs
$FC_ENABLE=1 \
FC_SETTINGS="config/settings/prod" \
FC_PARTIALS="config/partials" \
FC_TEMPLATES="config/templates" \
FC_OUT="output.json" \
krakend run -c "krakend.tmpl"
In the example above, notice that the FC_SETTINGS
includes the path to the prod
uction folder. This is how you would set a specific environment. You might inject here an env var if you have multiple environments. The directory structure is completely up to you.
:watch
image to speed up your development time.Template syntax
The configuration file passed with the -c
flag is treated as a Go template (documentation), and you can make use of all the power the template engine brings. In addition, the templating system is overloaded with Sprig functions, and KrakenD functions, adding more features.
The data evaluations or control structures are easily recognized as they are surrounded by {{
and }}
. Any other text outside these delimiters is unprocessed text copied to the output as it is.
KrakenD-specific functions
To use the partials, templates, and settings as defined in the environment variables, you have the following functions available:
{{ marshal .var }}
: Insert a JSON structure taking the content from.var
{{ include "file.txt" }}
: Insert the content of thefile.txt
“as is”. You can use any extension in these files.{{ template "file.tmpl" . }}
: Renders the Go templatefile.tmpl
passing all its variables as context. The context is the final.
you can see in the block. Thefile.tmpl
can access the context using{{ . }}
. The context can be a simple value (like a string) or an object with nested elements.
See below for further explanation and examples.
Insert values from settings files
In the FC_SETTINGS
directory, you can save multiple .json
files with data structures inside that you can reuse in the templates. A filename.json
is immediately available as the variable {{ .filename }}
in the template.
For instance, if you have a file settings/urls.json
with the following content:
{
"users_api": "https://users-api.mycompany.com",
"inventory_api": "https://inventory-api.mycompany.com",
"3rdparty": {
"github": "https://api.github.com"
}
}
You can access type in the template {{ .urls.users_api }}
, and get https://users-api.mycompany.com
when rendered. Or you could use {{ .urls.3rdparty.github }}
and get https://api.github.com
.
As you can see, the first word after the dot is the filename (without the extension), and the following dots traverse the objects to the final value.
Insert structures from settings files
When instead of a single value you need to insert a JSON structure (several elements), you need to use marshal
.
{{ marshal .urls }}
The example would write the entire content of the urls.json
file.
Include an external file
To insert the content of an external partial file in-place use:
{{ include "partial_file_name.txt" }}
The content inside the partial template is not parsed, and is inserted as is in plain text. The file is assumed to live inside the directory defined in FC_PARTIALS
and can have any name and extension. Filenames referenced are case sensitive, and although your host operating system might work with case insensitive files (e.g., A docker volume on Mac), when copied to a Docker image not respecting the case will fail.
Include and process a sub-template
While the include
is only meant to paste the content of a plain text file, the template
gives you all the power of Go templating. The syntax is as follows:
{{ template "template_name.tmpl" context }}
The template template_name.tmpl
is executed and processed. The value of context
is passed in the template as the context, meaning that the sub-template can access it using the dot {{ . }}
. This context variable could be an object, such as {{ template "environment.tmpl" .urls }}
, but it can also be another type, like a string: ``{{ template “environment.tmpl” “production” }}`.
Go templates allow you to introduce handy stuff like conditionals or loops and allow you to create powerful configurations.
Sprig functions
Complementing the Go built-in template language, and the KrakenD functions, the templates also adds more than 100 commonly used functions.
For instance, you could get a string from an environment variable COMMIT_SHA
, and truncate it to 8 chars with:
{
"$schema": "https://www.krakend.io/schema/v2.2/krakend.json",
"version": 3,
"name": "Configuration commit {{ env "COMMIT_SHA" | trunc 8}}"
}
Sprig provides many functions in the following categories:
- String Functions:
trim
,wrap
,randAlpha
,plural
and more.- String List Functions:
splitList
,sortAlpha
and more.
- String List Functions:
- Integer Math Functions:
add
,max
,mul
and more.- Integer Slice Functions:
until
,untilStep
- Integer Slice Functions:
- Float Math Functions:
addf
,maxf
,mulf
and more. - Date Functions:
now
,date
and more. - Defaults Functions:
default
,empty
,coalesce
,fromJson
,toJson
,toPrettyJson
,toRawJson
,ternary
- Encoding Functions:
b64enc
,b64dec
and more. - Lists and List Functions:
list
,first
,uniq
and more. - Dictionaries and Dict Functions:
get
,set
,dict
,hasKey
,pluck
,dig
,deepCopy
and more. - Type Conversion Functions:
atoi
,int64
,toString
and more. - Path and Filepath Functions:
base
,dir
,ext
,clean
,isAbs
,osBase
,osDir
,osExt
,osClean
,osIsAbs
- Flow Control Functions:
fail
- Advanced Functions
- UUID Functions:
uuidv4
- OS Functions:
env
,expandenv
- Version Comparison Functions:
semver
,semverCompare
- Reflection:
typeOf
,kindIs
,typeIsLike
and more. - Cryptographic and Security Functions:
derivePassword
,sha256sum
,genPrivateKey
and more. - Network:
getHostByName
- UUID Functions:
Testing the configuration
As the configuration is now composed of several pieces, it’s easy to make a mistake at some point. Test the syntax of all the files with the krakend check
command and pay attention to the output to verify there aren’t any errors.
You might also want to use the flag FC_OUT
to write the content of the final file in a known path, so you can check its contents:
Checking the configuration
$FC_ENABLE=1 \
FC_SETTINGS="$PWD/config/settings" \
FC_PARTIALS="$PWD/config/partials" \
FC_TEMPLATES="$PWD/config/templates" \
FC_OUT=out.json \
krakend check -t -d -c "$PWD/config/krakend.json"
When there are errors, the output contains information to help you resolve it, e.g.:
ERROR parsing the configuration file: loading flexible-config settings:
- backends.json: invalid character '}' looking for beginning of object key string
Tips & Tricks
The flexible configuration is a very simple tool (this documentation in fact is way larger than its implementation), but it can hold very complex setups. Here there are a few tips and tricks that you can use.
Directory boilerplate
You can create the Flexible Configuration directory structure depicted above with the following:
Template boilerplate
$mkdir -p config/{partials,settings,templates} config/settings/{prod,test}
Work faster with a docker-compose
Save the following docker-compose.yml
and do a docker-compose up
. This setup with the :watch
image will allow you to work locally with the latest version of KrakenD and apply the changes automatically every time you change a source file.
version: "3"
services:
krakend:
image: krakend/krakend-ee:watch
volumes:
- "./:/etc/krakend/"
environment:
- FC_ENABLE=1
- FC_OUT=/etc/krakend/out.json
- FC_PARTIALS=/etc/krakend/config/partials
- FC_SETTINGS=/etc/krakend/config/settings/test
- FC_TEMPLATES=/etc/krakend/config/templates
command: ["run","-dc","krakend.tmpl"]
Understanding the context
When you call a subtemplate from another template, you are in a {{ range }}
(loop), or use a {{ with }}
, you are using a specific context. The context is the variables you have available inside that block. When calling templates from templates, make sure to add the final dot .
to pass all the settings files to the next template or pass those variables that are needed:
{{ template "hello.tmpl" . }}
: the hello template receives all setting files and works as its calling template.{{ template "hello.tmpl" .urls.users_api }}
: receives only the string value of the users API.{{ template "hello.tmpl" "hello world" }}
: receives only a constant string
Using the $ notation
Similarly, when you are making a loop with a range
or accessing a with
, the variables inside are relative to its context one more time. But to access outsider variables you can use the $
notation. For instance: {{ $.urls.users_api }}
How to separate objects with commas
A lot of times, you need to iterate content and separate it using a comma. To do so, place the comma insertion at the beginning instead of the end when the index in the loop is not zero:
{{ range $index, $endpoint := .endpoints_list }}
{{if $index}},{{end}}
{
"endpoint": "{{ $endpoint.path }}",
"backend": []
}
{{end}}
Remove white spaces and line breaks
When you use code {{ blocks }}
on your templates, you can add a left dash {{- blocks }}
to suppress preceding whitespaces and linebreaks or a right dash {{ blocks -}}
to remove the following ones, or both {{- blocks -}}
.
Inserting an external file as base64
A few fields in KrakenD require you to set their value in base64 format instead of the raw counterpart. For example, sometimes you want to version control the raw file in an external file and reference it as base64. To do so, you could have a template render_as_base64.tmpl
with the following content:
{{/* Notice the dashes (-) at the beginning and end of the following code.
They remove all spaces and linebreaks that appear before and after when the result outputs. */}}
{{- $raw_content := include . -}}
{{- $raw_content | b64enc -}}
And call it in the krakend.tmpl
like this:
{
"version": 3,
"some_base_64_value": "{{template "render_as_base64.tmpl" "file.json"}}"
}
Load dynamically all templates in a directory
Sometimes you want to “drag and drop” a new template into a folder and load it automatically in the configuration without referencing it. To do this, you must preprocess the template directory’s contents into the file you will include.
Let’s imagine your krakend.tmpl
looks like this:
{
"version": 3,
"$schema": "https://www.krakend.io/schema/v2.2/krakend.json",
"endpoints": [
{{ include "endpoints.tmpl" . }}
]
}
The endpoints.tmpl
should contain the list of all the templates we want to include. One of the limitations of the Go {{ include}}
is that you cannot use a variable to load a template, so we must pre-generated the contents.
There are many ways to generate this, and the following command is to save you time if you want to achieve this:
Dynamic file
$tree -J -P "ep_*.tmpl" -I "endpoints.tmpl" \
| jq -r ' ( .[0].contents[].name | "{{ template \"\(.)\" . }}," )' \
| sed '$s/,$//' > endpoints.tmpl
The command above will use tree
to scan the contents of the template folder, using a theoretical pattern and generate the endpoints.tmpl
file. The flags are:
-J
generates the tree output in JSON format.-P
scans only for a particular pattern. In our example, templates beginning withep_
.-I
excludes the fileendpoints.tmpl
- The
jq
command prints in raw (-r
) so quotes are not escaped and surrounds the template with the template syntax - The final
sed
removes the last comma - The output file
endpoints.tmpl
could look like this:
cat endpoints.tmpl
{{ template "ep_users.tmpl" . }}
{{ template "ep_checkout.tmpl" . }}
{{ template "ep_catalogue.tmpl" . }}
{{ template "ep_starred.tmpl" . }}
{{ template "ep_auth.tmpl" . }}
Practical example
To demonstrate the usage of the flexible configuration, we will reorganize a configuration file into several pieces. This is a simple example to see the basics of the templates system:
.
└── config
├── krakend.tmpl
├── partials
│ └── rate_limit_backend.tmpl
└── settings
├── endpoint.json
└── service.json
partials/rate_limit_backend.tmpl
In this file, we have written the content of the rate limit configuration for a backend. This file is inserted when included “as is”:
"qos/ratelimit/proxy": {
"max_rate": "100",
"capacity": "100"
}
settings/service.json
In the settings directory, we write all the files whose values can be accessed as variables.
{
"port": 8090,
"default_hosts": [
"https://catalog-api-01.srv",
"https://catalog-api-02.srv",
"https://catalog-api-03.srv"
],
"extra_config": {
"security/http": {
"allowed_hosts": [],
"ssl_proxy_headers": {
"X-Forwarded-Proto": "https"
},
"ssl_certificate": "/opt/rsa.cert",
"ssl_private_key": "/opt/rsa.key"
}
}
}
settings/endpoint.json
This file declares a couple of endpoints that feed on a single backend:
{
"example_group": [
{
"endpoint": "/users/{id}",
"backend": "/v1/users?userId={id}"
},
{
"endpoint": "/posts/{id}",
"backend": "/posts?postId={id}"
}
]
}
krakend.tmpl
Finally, let’s introduce the base template. It inserts the content of other files using include
, uses the variables declared in the settings files, and writes json content with marshal
.
Have a look at the highlighted lines:
{
"version": 3,
"port": {{ .service.port }},
"extra_config": {{ marshal .service.extra_config }},
"host": {{ marshal .service.default_hosts }},
"endpoints": [
{{ range $idx, $endpoint := .endpoint.example_group }}
{{if $idx}},{{end}}
{
"endpoint": "{{ $endpoint.endpoint }}",
"backend": [
{
"url_pattern": "{{ $endpoint.backend }}",
"extra_config": {
{{ include "rate_limit_backend.tmpl" }}
}
}
]}
{{ end }}
]
}
- The
.service.port
is taken from theservice.json
file. - The
extra_config
in the third line is inserted as a JSON object using themarshal
function from theservice.json
as well. - A
range
iterates the array found underendpoint.json
and keyexample_group
. The variables inside the range are relative to theexample_group
content. - An
include
in the extra_config inserts the content as is. - Also notice the little trick
{{if $idx}},{{end}}
inside the loop. When it is not in the first element0
, it will add a comma to prevent breaking the JSON format.
Notice that there is a {{ range }}
. If you wanted to use it inside a template, and not the base file, you would need to include it inside a sub-template with {{ template "template.tmp" .endpoint.example_group }}
.
To parse the configuration, use:
Check and test (-t) the config
$FC_ENABLE=1 \
FC_SETTINGS="$PWD/config/settings" \
FC_PARTIALS="$PWD/config/partials" \
krakend check -t -d -c "$PWD/config/krakend.tmpl"