Apache Kafka

Apache Kafka is an open-source distributed event streaming platform for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

It can be used for many purposes but principally serves as an implementation of a high performance and scalable distributed message queeu that lets you decouple your front end applications (and the Cloud CMS UI Server) from back end API notifications.

Cloud CMS Application Server

The Cloud CMS Application Server is a middle-tier cluster that sits between your mobile/web application and the Cloud CMS API. It provides a wide range of request-time services for your front-end application for fast performance, object caching and more.

It features integration with Apache Kafka as a means of listening to events that occur within the Cloud CMS API. When content is created, updated or deleted within the Cloud API, notifications are received and placed onto an Apache Kafka topic.

The Cloud CMS Application Server cluster hears these messages and responds by invalidating cache and doing the necessary bookkeeping to keep the middle-tier consistent with changes from your editorial team.

How it works

You will need to run Apache Kafka on your own (either as a standalone service or a standalone container).
Apache Kafka needs to be accessible to the API and the UI via a host and port.

1. When the editorial team makes changes to content inside of Cloud CMS, the Cloud CMS API produces notification messages that are pushed to an Apache Kofka topic.
2. The Cloud CMS Application Server consumes messages from this Apache Kafka topic.
3. When messages are received, the Cloud CMS Application server automatically invalidates cache state for your client facing applications.

Configure the Cloud CMS API

To configure the Cloud CMS API to use the kafka notification provider, you can adjust your docker.properties file to have something like this:

gitana.default.application.deployer.uiserver.notifications.enabled=true
gitana.default.application.deployer.uiserver.notifications.providerType=kafka
gitana.default.application.deployer.uiserver.notifications.topic=cloudcms.ui.topic
gitana.default.application.deployer.uiserver.notifications.configuration.username=
gitana.default.application.deployer.uiserver.notifications.configuration.password=<host>
gitana.default.application.deployer.uiserver.notifications.configuration.host=<port>
gitana.default.application.deployer.uiserver.notifications.configuration.port=9202

Where the value of gitana.default.application.deployer.uiserver.notifications.configuration.host is the host name of the Kafka broker to connect to.

The value of gitana.default.application.deployer.uiserver.notifications.configuration.port is typically 9092.

Note that this sends notifications to the cloudcms.ui.topic topic. Change these as you require.

Configure the Cloud CMS Application Server

To configure the Cloud CMS Application Server, you can add the following to your environment upon startup:

CLOUDCMS_NOTIFICATIONS_ENABLED=true
CLOUDCMS_NOTIFICATIONS_KAFKA_BROKERS=<host1>:<port1>,<host2>:<port2>
CLOUDCMS_NOTIFICATIONS_KAFKA_TOPIC=cloudcms.ui.topic

Note that the UI server supports connecting to multiple Kafka brokers (though only 1 is required). Once again, adjust these settings as you require.

Alternatively, you can specify this via the server configuration. Here is an example of how this might look:

"notifications": {
    "enabled": true,
    "type": "kafka",
    "configuration": {
        "brokers": "kafka1:9092,kafka2:9092,kafka3:9092",
        "topic": "cloudcms.ui.topic"
    }
}

And then restart your Node app. When your Node app comes online, you should see a message indicating that the connection to Apache Kafka was established and is good to go.