gRPC client and gRPC to REST conversion

Document updated on May 25, 2023

The gRPC client integration allows KrakenD to connect to upstream services using gRPC. If you haven’t enabled the gRPC server functionality, then the content is automatically transformed to a regular REST API, as shown in the picture.

If, on the other hand, you enable a gRPC server, then you can have a gRPC-to-gRPC communication.

gRPC client configuration

To use the gRPC client, you must declare two extra configuration entries in the settings:

1. grpc: The catalog with all the protocol buffer definitions, at the service level. See the catalog definition.
2. backend/grpc: This is the connection to the specific gRPC services in the catalog at the backend level.

For example:

{
  "version": 3,
  "extra_config": {
    "grpc": {
      "@comment": "The catalog loads all .pb files passed or contained in directories",
      "catalog": [
        "grpcatalog/flights/fligths.pb",
        "grpcatalog/known_types",
        "grpcatalog/third_parties"
      ]
    }
  },
  "endpoints": [
    {
      "@comment": "Feature: GRPC",
      "endpoint": "/flights",
      "input_query_strings": ["*"],
      "backend": [
        {
          "host": ["localhost:4242"],
          "url_pattern": "/flight_finder.Flights/FindFlight",
          "extra_config": {
            "backend/grpc": {}
          }
        }
      ]
    }
  ]
}

At the backend level, the inclusion of the backend/grpc object with no content is enough, although there are more optional settings you can add, both for the TLS connection with the gRPC service and how to treat the input parameters or backend response:

A few important requirements you should have in mind when adding backend configurations:

• host The array of hosts does not have a protocol prefix (no http or https, just host:port).
• url_pattern: Cannot contain variables and must be the full name of the gRPC service and method call (for example: "/pizzeria_sample.Pizzeria/ListMenu")

Ignored attributes (their values or presence is ignored):

• method
• is_collection

Regarding the endpoint configuration, the output_encoding you set in the endpoint must be different than no-op.

The backend response after the gRPC interaction accepts all the manipulation options as with REST calls.

For instance:

{
    "@comment": "Feature: GRPC",
    "endpoint": "/flights",
    "input_query_strings": ["*"],
    "backend": [
    {
        "host": ["localhost:4242"],
        "url_pattern": "/flight_finder.Flights/FindFlight",
        "extra_config": {
            "backend/grpc": {
                "client_tls": {
                    "@comment": "Allow untrusted certificates in development stage",
                    "allow_insecure_connections": true
                }
            }
        }
    }
    ]
}

Passing user parameters

To pass dynamic parameters from the user to your gRPC service, you must use query strings, a body, headers (sent as metadata), or {placeholders} in the endpoints.

When you use query strings, you must always include the necessary input_query_strings. You can even pass all the query strings with a wildcard * because only those that match the gRPC call will pass.

Headers must be listed in the endpoint’s input_headers array when you use them. In addition, the field header_mapping under the backend/grpc section allows you to rename those headers to gRPC metadata. All headers in the gRPC backend will be sent with their original (but lowercased) name or using the provided header_mapping.

When you use a {placeholder} in the URL of the endpoint, the variable name capitalizes the first letter. You must refer to a variable {foo} as Foo.

For instance:

{
    "@comment": "Feature: GRPC",
    "endpoint": "/flights/{date}",
    "input_query_strings": ["*"],
    "input_headers": ["*"],
    "backend": [
    {
        "host": ["localhost:4242"],
        "url_pattern": "/flight_finder.Flights/FindFlight",
        "extra_config": {
            "backend/grpc": {
                "header_mapping": {
                    "X-Tenant": "customerid"
                },
                "client_tls": {
                    "@comment": "Allow untrusted certificates in development stage",
                    "allow_insecure_connections": true
                }
            }
        }
    }
    ]
}

In addition, you can map the original input query string parameters to other keys using the input_mapping property.

For instance, the following example takes a request with ?lat=123&lon=456 and sends a request to the gRPC service with a renamed object. The Date (notice the first letter is uppercased) is taken from the URL, while the coordinates are taken from the query string:

{
    "@comment": "Feature: GRPC",
    "endpoint": "/flights/{date}",
    "input_query_strings": ["*"],
    "backend": [
    {
        "host": ["localhost:4242"],
        "url_pattern": "/flight_finder.Flights/FindFlight",
        "extra_config": {
            "backend/grpc": {
                "input_mapping": {
                    "lat": "where.latitude",
                    "lon": "where.longitude",
                    "Date": "when.departure"
                }
            }
        }
    }
    ]
}

The gRPC service receives:

{
    "where": {
        "latitude": 123,
        "longitude": 456
    },
    "when": {
        "departure": "2023-07-09"
    }
}

In addition to query parameters, you can pass a body through the flag use_request_body. When both the params/query strings and the body input are enabled, the parameters and query string are set first and the JSON body second. This means the body will overwrite colliding values set by the params/query strings (you pass the same fields in both). Or said otherwise: params/query values will only apply when those values are not set in the body payload.

Finally, fields for the GRPC body can also be taken from the headers data (but query string values and URL parameters take precedence if provided over the values found in the headers).

Limitations

For child messages (or child objects), the input params are expected to be in dot notation, e.g., some_field.child.grand_child=10.

It accepts lists (“repeated” in gRPC nomenclature) of basic types with repeated notation: a=1&a=2&a=3

Or also when the array is inside an object, e.g., a.b.c=1&a.b.c=2&a.b.c=3

But it cannot fill arrays that contain other arrays or arrays that contain other objects (that could include other arrays) because it is difficult to know where a value should be put.

For example, for an object

{
    "a": {
       "b": [
           {
               "x": [1, 2]
           },
           {
               "x": [5, 6]
           }
       ],
       "c": "something"
    }
}

What should be the output for a.b.x=1&a.b.x=2?

Should it be: {"a": {"b": [{"x": [1, 2]}]}} or {"a": {"b": [{"x": [1]}, {"x": [2]}]}}?