Document updated on Mar 1, 2020
Since | v1.0 |
---|---|
Namespace | backend/pubsub/publisher backend/pubsub/subscriber |
Log prefix | [BACKEND: schema://host][PubSub] |
Scope | backend |
Source | krakend/krakend-pubsub |
You can connect an endpoint to multiple publish/subscribe backends, helping you integrate with event driven architectures.
For instance, a frontend client can push events to a queue using a REST interface. Or a client could consume a REST endpoint that is plugged to the last events pushed in a backend. You can even validate messages and formats as all the KrakenD available middleware can be used. The list of supported backend technologies is:
To add pub/sub functionality to your backends include the namespaces backend/pubsub/subscriber
and backend/pubsub/publisher
under the extra_config
of your backend
section.
The host
key defines the desired driver, and the actual host is usually set in an environment variable outside of KrakenD:
For a subscriber:
{
"host": ["schema://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "url"
}
}
}
| Subscription URL according to the selected driver |
For a publisher:
{
"host": ["schema://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/publisher": {
"topic_url": "url"
}
}
}
| Topic URL according to the selected driver |
See the specification of each individual technology.
Example (RabbitMQ):
Set the envvar RABBIT_SERVER_URL='guest:guest@localhost:5672'
and add in the configuration:
{
"host": ["amqp://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "myexchange"
}
}
}
Google’s Cloud Pub/Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications.
The configuration you need to use is:
host
: gcppubsub://
url
for topics: "projects/myproject/topics/mytopic"
or the shortened form "myproject/mytopic"
url
for subscriptions: "projects/myproject/subscriptions/mysub"
or the shortened form "myproject/mysub"
GOOGLE_APPLICATION_CREDENTIALS
, see Google documentation.Example:
{
"host": ["gcppubsub://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "projects/myproject/subscriptions/mysub"
}
}
}
NATS.io is a simple, secure and high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.
Configuration:
host
: nats://
NATS_SERVER_URL
url
: mysubject
No query parameters are supported.
Example:
{
"host": ["nats://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "mysubject"
}
}
}
Amazon Simple Notification Service (SNS) is a highly available, durable, secure, fully managed pub/sub messaging service that enables you to decouple microservices, distributed systems, and serverless applications. Amazon SNS provides topics for high-throughput, push-based, many-to-many messaging
AWS SNS sets the url
without any host
or environment variables, e.g:
{
"host": ["awssns:///arn:aws:sns:us-east-2:123456789012:mytopic"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "?region=us-east-2"
}
}
}
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
AWS SQS sets the url
without any host
or environment variables, e.g:
Url: awssqs://sqs-queue-url
{
"host": ["awssqs://sqs.us-east-2.amazonaws.com/123456789012"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "/myqueue?region=us-east-2"
}
}
}
Microsoft Azure Service Bus supports a set of cloud-based, message-oriented middleware technologies including reliable message queuing and durable publish/subscribe messaging. These “brokered” messaging capabilities can be thought of as decoupled messaging features that support publish-subscribe, temporal decoupling, and load balancing scenarios using the Service Bus messaging workload.
Configuration:
host
: azuresb://
SERVICEBUS_CONNECTION_STRING
mytopic
mytopic?subscription=mysubscription
Note that for subscriptions, the subscription name must be provided in the ?subscription=
query parameter.
Example:
{
"host": ["azuresb://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "mytopic"
}
}
}
RabbitMQ is one of the most popular open source message brokers.
Rabbit can alternatively be configured using the AMQP component.
Configuration:
host
: rabbit://
RABBIT_SERVER_URL
url
for topics: myexchange
url
for subscriptions: myqueue
No query parameters are supported.
Example:
{
"host": ["rabbit://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "myexchange"
}
}
}
Apache Kafka is a distributed streaming platform.
Kafka connection requires KrakenD >= 1.1
.
host
: kafka://
KAFKA_BROKERS
pointing to the server(s), e.g: KAFKA_BROKERS=192.168.1.100:9092
Kafka subscriptions:
{
"host": ["kafka://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/subscriber": {
"subscription_url": "group?topic=mytopic"
}
}
}
Kafka topics:
{
"host": ["kafka://"],
"disable_host_sanitize": true,
"extra_config": {
"backend/pubsub/publisher": {
"topic_url": "mytopic"
}
}
}
The documentation is only a piece of the help you can get! Whether you are looking for Open Source or Enterprise support, see more support channels that can help you.