Document updated on Oct 24, 2022
Exporting metrics and events to InfluxDB
KrakenD can expose very detailed metrics to provide a monitoring dashboard. One of the richest monitoring solutions at the metrics level is the combination of Extended metrics with the native Influx exporter. The two components let you send detailed metrics to InfluxDB and draw them later on our preconfigured Grafana dashboard.
InfluxDB configuration
Notice that this document describes two different implementations of InfluxDB:
- Native InfluxDB exporter (recommended)
- OpenCensus InfluxDB exporter
Native InfluxDB configuration
Pushing data to InfluxDB using the native middleware requires adding two different configuration pieces:
- Enabling the extended metrics (collecting the information)
- Enabling InfluxDB (pushing the data)
You can accomplish it with the following snippet.
{
"version": 3,
"extra_config": {
"telemetry/influx":{
"address":"http://influxdb:8086",
"ttl":"25s",
"buffer_size":0,
"db": "krakend",
"username": "your-influxdb-user",
"password": "your-influxdb-password"
},
"telemetry/metrics": {
"collection_time": "30s",
"listen_address": "127.0.0.1:8090"
}
}
}
Fields of Telemetry via influx
address
* string- The complete url of the influxdb including the port if different from defaults in http/https.
buffer_size
integer- The buffer size is a protection mechanism that allows you to temporarily store datapoints for later reporting when Influx is unavailable. If the buffer is
0
, reported metrics that fail are discarded immediately. If the buffer is a positive number, KrakenD creates a buffer with the number of datapoints set. When the buffer is full because the Influx server keeps failing, newer datapoints replace older ones in the buffer. db
string- Name of the InfluxDB database (Influx v1) or the bucket name (Influx v2).Defaults to
"krakend"
password
string- Password to authenticate to InfluxDB. In Influx v2, you also need to add grant access with
influx v1 auth
. ttl
*- TTL against Influx.Specify units using
ns
(nanoseconds),us
orµs
(microseconds),ms
(milliseconds),s
(seconds),m
(minutes), orh
(hours). username
string- Username to authenticate to InfluxDB.
See below how to configure InfluxDB, and you are ready to publish a Grafana dashboard.
OpenCensus InfluxDB exporter
InfluxDB is a time series database designed to handle high write and query loads.
The Opencensus exporter allows you export data to InfluxDB for monitoring metrics and events. Enabling it only requires you to add as exporter
the influxdb
entry.
The following configuration snippet sends data to your InfluxDB:
{
"telemetry/opencensus": {
"sample_rate": 100,
"exporters": {
"influxdb": {
"address": "http://192.168.99.100:8086",
"db": "krakend",
"timeout": "1s",
"username": "your-influxdb-user",
"password": "your-influxdb-password"
}
}
}
}
As with all OpenCensus exporters, you can add optional settings in the telemetry/opencensus
level:
Fields of "telemetry/opencensus"
enabled_layers
- Lets you specify what data you want to export. All layers are enabled by default unless you declare this section.
backend
boolean- Reports the activity between KrakenD and your servicesDefaults to
false
pipe
boolean- Reports the activity at the beginning of the proxy layer. It gives a more detailed view of the internals of the pipe between end-users and KrakenD, having into account merging of different backends.Defaults to
false
router
boolean- Reports the activity between end-users and KrakenDDefaults to
false
reporting_period
integer- The number of seconds passing between reports. If duration is less than or equal to zero, it enables the default behavior of each exporter.Defaults to
0
sample_rate
integer- A number between 0 (no requests at all) and 100 (all requests) representing the percentage of sampled requests you want to send to the exporter. Sampling the 100% of the requests is generally discouraged when the relationship between traffic and dedicated resources is sparse.Defaults to
0
Then, the exporters
key must contain an influxdb
entry with the following properties:
Fields of "telemetry/opencensus": { "exporters":{} }
influxdb
object- Exports data to InfluxDB: A time series database designed to handle high write and query loads.
address
string- The URL (including port) where your InfluxDB is installed.Example:
"http://192.168.99.100:8086"
db
string- The database nameExample:
"krakend"
password
string- The password to access the databaseExample:
"kr4k3nd"
timeout
string- The maximum time you will wait for InfluxDB to respond.Specify units using
ns
(nanoseconds),us
orµs
(microseconds),ms
(milliseconds),s
(seconds),m
(minutes), orh
(hours).Example:"2s"
username
string- The influxdb username to access the databaseExample:
"krakend"
Setting up Influx
For InfluxDB v2.x, we have included in our Telemetry Dashboards the files that create the authorization part.
For InfluxDB v1.x (older) the process is straightforward and requires you nothing else than start an Influx instance with the desired configuration.
Influx v2
If you use Docker, you can start InfluxDB as part of a docker-compose file. You need to specify in the configuration above the same data you used to run InfluxDB. For instance, the following docker-compose.yml
sets the credentials you need to reflect in the KrakenD configuration.
version: "3"
services:
influxdb:
image: influxdb:2.4
environment:
- "DOCKER_INFLUXDB_INIT_MODE=setup"
- "DOCKER_INFLUXDB_INIT_USERNAME=krakend-dev"
- "DOCKER_INFLUXDB_INIT_PASSWORD=pas5w0rd"
- "DOCKER_INFLUXDB_INIT_ORG=my-org"
- "DOCKER_INFLUXDB_INIT_BUCKET=krakend"
- "DOCKER_INFLUXDB_INIT_RETENTION=1w"
- "DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=my-super-secret-auth-token"
ports:
- "8086:8086"
volumes:
- "./config/telemetry/influx:/docker-entrypoint-initdb.d"
krakend:
image: devopsfaith/krakend:2.4
volumes:
- ./krakend:/etc/krakend
ports:
- "8080:8080"
In the fields db
, username
, and password
of the component configuration reflect the same values as in DOCKER_INFLUXDB_INIT_BUCKET
, DOCKER_INFLUXDB_INIT_USERNAME
, and DOCKER_INFLUXDB_INIT_PASSWORD
accordingly.
The Influx volume below must have the contents of the influx initdb script, that it will create the authorization needed to let KrakenD push the metrics.
Manual configuration
If you don’t want to use the automated docker-compose above, the manual steps to create the auth are:
Connect to your influx
Term
$docker exec -it influxdb /bin/bash
Create a configuration:
Term
$influx config create --config-name krakend-config \
--host-url http://localhost:8086 \
--org my-org \
--token my-super-secret-auth-token \
--active
Make sure to replace the values below as follows:
config-name
is a random name to identify your configurationorg
Your organization name as written during the setup of InfluxDBtoken
The one you started influx withhost-url
The address where the influxdb is running (inside Docker is as shown)
Access with a web browser to http://localhost:8086
and select BUCKETS from the UI. You’ll see that there is an ID next to the bucket you created during the Setup. Copy the bucket ID.
And now launch the last command in the shell:
Term
$influx v1 auth create \
--read-bucket b492e6f8f3b13aaa \
--write-bucket b492e6f8f3b13aaa \
--username krakend-dev
Replace the ID of the buckets above with the ID you just copied, and the username as in the docker compose. The shell will ask for your password.
Now your configuration should work and start sending data to InfluxDB:
{
"version": 3,
"extra_config": {
"telemetry/metrics": {
"collection_time": "60s",
"listen_address": ":8090"
},
"telemetry/influx": {
"address": "http://localhost:8086",
"ttl": "25s",
"buffer_size": 100,
"db": "krakend_db",
"username": "user",
"password": "password"
}
}
}
Make sure to type in db
the bucket name you created on InfluxDB and the username
and password
as well.
Influx v1
When using InfluxDB v1.x, you need to specify in the configuration above the same data you used to run InfluxDB. For instance, the following docker-compose sets the credentials you need to reflect in the KrakenD configuration.
version: "3"
services:
influxdb:
image: influxdb:1.8
environment:
- "INFLUXDB_DB=krakend"
- "INFLUXDB_USER=krakend-dev"
- "INFLUXDB_USER_PASSWORD=pas5w0rd"
- "INFLUXDB_ADMIN_USER=admin"
- "INFLUXDB_ADMIN_PASSWORD=supersecretpassword"
ports:
- "8086:8086"
krakend:
image: devopsfaith/krakend:2.4
volumes:
- ./krakend:/etc/krakend
ports:
- "8080:8080"
In the fields db
, username
, and password
of the component configuration reflect the same values as in INFLUXDB_DB
, INFLUXDB_USER
, and INFLUXDB_USER_PASSWORD
accordingly.