Automated End-to-End (E2E) tests

Last update: Nov 18, 2022

In addition to checking the syntax of your KrakenD configuration and ensuring that the gateway can start, you can run integration tests to guarantee that all the active software components from beginning to end have the expected flow and that the gateway returns what you planned.

The krakend e2e command launches the integration tests.

Creating e2e test files

In essence, you must create a specs folder and place the test files inside. Then, KrakenD will run all the tests declared in the folder and exit with the proper status code. All tests must be deterministic and reproducible.

The default directory structure for specs is:

/etc/krakend
├── krakend.json
└── specs
    ├── test-foo.json
    ├── test-bar.json
    └── some-other-test.json

Each test file is a very simple .json file, and it doesn’t need a specific name. It contains an in (the request you want to do to KrakenD) and an out (what you expect as the response). In addition, the following attributes are recognized in each test spec:

• in: The parameters used to build the request to a running KrakenD with your configuration
  • method (optional): The request method. Defaults to GET.
  • url: The full URL to the endpoint you want to test
  • header (optional): An optional map of headers to include in the request
• out: The expected response from the gateway
  • status_code (integer): The expected status code
  • body (optional): The expected returned body by the response as a string or JSON object. Remove this body field when you don’t want to check its contents, when the response does not have a body, or when you want to use the schema instead.
  • schema (optional): A JSON Schema object that validates the response. If the response matches the schema definition, the test passes.
  • header (optional): Any header included in the response. When the value is empty, you don’t expect that header.

For instance, save your first test under specs/test1.json and run it with any krakend configuration:

{
    "@comment": "Makes sure that the debug endpoint returns a status ok",
    "in": {
        "method": "GET",
        "url": "http://localhost:8080/__debug/something",
        "header": {
            "User-Agent": "krakend e2e tool"
        }
    },
    "out": {
        "status_code": 200,
        "body": {
          "status": "ok"
        },
        "header": {
            "content-type": ["application/json; charset=utf-8"],
            "Cache-Control": [""],
            "X-Krakend-Completed": ["true"]
        }
    }
}

In the example above, the response must contain the content-type and X-Krakend-Completed with the specified values, and the Cache-Control header cannot be present.

A more simplified one that only checks a 200 and a specific body could be:

{
    "@comment": "Makes sure that the debug endpoint returns a status ok",
    "in": {
        "url": "http://localhost:8080/__debug/something"
    },
    "out": {
        "status_code": 200,
        "body": {
          "status": "ok"
        }
    }
}

Testing against a JSON schema

When the response of your backend is not consistent, instead of checking the response against a body, you can test it against a JSON Schema, using the schema attribute.

For instance, our health endpoint returns the date after every request. It would be impossible to test it literally, as it keeps changing. The following test would make sure that it works by comparing the response with a schema:

{
    "@comment": "Makes sure that the health endpoint contains three fields with the right types",
    "in": {
        "url": "http://localhost:8080/__health"
    },
    "out": {
        "status_code": 200,
        "schema": {
          "$schema": "http://json-schema.org/draft-07/schema#",
          "required": ["agents","now","status"],
          "properties": {
            "agents": {
                "type": "object"
            },
            "now": {
                "type": "string"
            },
            "status": {
                "type": "string",
                "enum": ["ok"]
            }
          }
        }
    }
}

Running the e2e tests

The e2e command can run without additional flags if you use the default naming, but it has several run options. It looks like this:

$krakend e2e -h

╓▄█                          ▄▄▌                               ╓██████▄µ
▐███  ▄███╨▐███▄██H╗██████▄  ║██▌ ,▄███╨ ▄██████▄  ▓██▌█████▄  ███▀╙╙▀▀███╕
▐███▄███▀  ▐█████▀"╙▀▀"╙▀███ ║███▄███┘  ███▀""▀███ ████▀╙▀███H ███     ╙███
▐██████▌   ▐███⌐  ,▄████████M║██████▄  ║██████████M███▌   ███H ███     ,███
▐███╨▀███µ ▐███   ███▌  ,███M║███╙▀███  ███▄```▄▄` ███▌   ███H ███,,,╓▄███▀
▐███  ╙███▄▐███   ╙█████████M║██▌  ╙███▄`▀███████╨ ███▌   ███H █████████▀
                     ``                     `'`

Version: 2.3.0

Executes an end-to-end test for the gateway based on the configuration file and a set of specs.

Usage:
  krakend e2e [flags]

Examples:
krakend e2e -c config.json -s specs

Flags:
  -c, --config string    Path to the krakend configuration file. (default "./krakend.json")
  -d, --delay duration   The delay for the delayed backend endpoint. (default 200ms)
  -e, --envar string     Comma separated list of patterns to use to filter the envars to pass (set to ".*" to pass everything).
  -h, --help             help for e2e
  -l, --log string       Path for storing the server logs. (default "./e2e.log")
  -r, --no-redirect      Disable redirects at the http client.
  -p, --port int         The port for the mocked backend api. (default 8081)
  -s, --specs string     Path to the specs folder. (default "./specs")

When you run the tests, KrakenD will tell you the failing ones with a [KO] and the working ones with an [OK]. For instance:

$krakend e2e
[OK] test1
1 test(s) completed
Total time: 1.102928274s

The e2e command starts and shuts downs two services during the tests:

• A KrakenD instance running on port 8080 or the port you have defined in your configuration, on which all test requests are sent.
• An additional backend service on port 8081 (or the one you define with krakend e2e -p) with a few utility endpoints that you can use to complement your testing (see below)

Skipping tests

If you want to skip a test temporarily, rename the test to a non .json extension. For instance, you can rename test1.json to test1.json.skip.

Using mock data

The point of integration tests is to test KrakenD configuration (not the backend content itself). Therefore, all tests expect reproducible outputs. For example, suppose your test includes getting data from a source that changes between different executions (like relative dates, timestamps, etc.). In that case, you cannot use body and need to use a schema instead.

Suppose you don’t want to use a schema but a body for any reason. Then, to mitigate this scenario, you can use the deny attribute to remove specific fields from the tests, or you can use mock data for testing instead. An easy way to have fake data is to create a mock folder with static JSON content and offer it via the static-filesystem plugin. For instance:

{
    "version": 3,
    "$schema": "https://www.krakend.io/schema/v3.json",
    "host": [
        "http://localhost:8080"
    ],
    "plugin": {
        "folder": "./plugins/",
        "pattern": ".so"
    },
    "extra_config": {
        "plugin/http-server": {
            "name": [
                "static-filesystem"
            ],
            "static-filesystem": {
                "path": "./tests/mock/",
                "prefix": "/mock/"
            }
        }
    },
    "endpoints": [
        {
            "@comment": "Uses KrakenD as a backend with mocked data",
            "endpoint": "/test/1",
            "backend": [
                {
                    "url_pattern": "/mock/sample.json"
                }
            ]
        }
    ]
}

With this configuration, whenever you ask for a route /mock/ to the gateway, it will return the file inside the folder. With a mock, it’s easier to pass your tests.

E2E utility backend service

When running the tests, the command line will start on port 8081(by default), a backend service with the following endpoints that you can include in your tests.

You can see the code of these endpoints below here.

In addition to the endpoints below, you can also use the echo and debug endpoints of KrakenD (not this service)

Endpoint /param_forwarding/*

An echo endpoint that returns an object containing a map with the request details:

• path: The URL requested to the backend
• query: The different query strings passed to the backend
• headers: All the headers that reached the backend
• foo: An additional object with a hardcoded value 42
• body: A string with the data passed in the request’s body. Only dumped when you call the backend with the query string ?dump_body=1.

For example, a call to http://localhost:8081/param_forwarding/hey/yo?a=1&b=1&dumb_body=1 produces the response:

{
    "path": "/param_forwarding/hey/yo",
    "query": {
        "a": 1,
        "b": 2,
        "dump_body": 1
    },
    "headers": {
        "Accept-Encoding": [
            "gzip"
        ],
        "User-Agent": [
            "KrakenD Version 2.3.0"
        ],
        "X-Forwarded-Host": [
            "localhost:8080"
        ]
    },
    "foo": 42,
    "body": {}
}

Endpoint /xml

Returns hardcoded content in XML format. Useful to test a mix of JSON and XML encodings:

<?xml version="1.0" encoding="UTF-8"?>
<user type="admin">
  <name>Elliot</name>
  <social>
    <facebook>https://facebook.com</facebook>
    <twitter>https://twitter.com</twitter>
    <youtube>https://youtube.com</youtube>
  </social>
</user>

Endpoint /collection/*

Returns a collection of 10 objects (an array) with the number of iteration (i) and the path used. For instance, calling http://localhost:8081/collection/hi-there produces:

[
    {
        "i": 0,
        "path": "/collection/hi-there"
    },
    {
        "i": 1,
        "path": "/collection/hi-there"
    },
    {
        "i": 2,
        "path": "/collection/hi-there"
    },
    {
        "i": 3,
        "path": "/collection/hi-there"
    },
    {
        "i": 4,
        "path": "/collection/hi-there"
    },
    {
        "i": 5,
        "path": "/collection/hi-there"
    },
    {
        "i": 6,
        "path": "/collection/hi-there"
    },
    {
        "i": 7,
        "path": "/collection/hi-there"
    },
    {
        "i": 8,
        "path": "/collection/hi-there"
    },
    {
        "i": 9,
        "path": "/collection/hi-there"
    }
]

Endpoint /delayed/*

Returns an echo endpoint (as in /param_forwarding), but it delays the response for 200ms or any other value you pass using the -d flag when running the tests.

Endpoint /redirect/*

Returns an HTTP redirection to /param_forwarding/ using the status code passed by query string with ?status=301. For instance, http://localhost:8081/redirect/hi-there?status=302`.

Endpoint /jwk/symmetric

Returns a fake signing key that validates demo JWT tokens. To be used when you set the jwk_url if you don’t want to issue real tokens

{
  "keys": [
    {
      "kty": "oct",
      "alg": "A128KW",
      "k": "GawgguFyGrWKav7AX4VKUg",
      "kid": "sim1"
    },
    {
      "kty": "oct",
      "k": "AyM1SysPpbyDfgZld3umj1qzKObwVMkoqQ-EstJQLr_T-1qS0gZH75aKtMN3Yj0iPS4hcgUuTwjAzZr1Z9CAow",
      "kid": "sim2",
      "alg": "HS256"
    }
  ]
}

The key above validates the following Authorization: Beaerer demo token:

bearer eyJhbGciOiJIUzI1NiIsImtpZCI6InNpbTIifQ.eyJhdWQiOiJodHRwOi8vYXBpLmV4YW1wbGUuY29tIiwiZXhwIjoxNzM1Njg5NjAwLCJpc3MiOiJodHRwczovL2tyYWtlbmQuaW8iLCJqdGkiOiJtbmIyM3Zjc3J0NzU2eXVpb21uYnZjeDk4ZXJ0eXVpb3AiLCJyb2xlcyI6WyJyb2xlX2EiLCJyb2xlX2IiXSwic3ViIjoiMTIzNDU2Nzg5MHF3ZXJ0eXVpbyJ9.htgbhantGcv6zrN1i43Rl58q1sokh3lzuFgzfenI0Rk