Broadway has a queue of built-in Actors that manage the handling of Pub or Sub asynchronous messages and can subscribe to Apache Kafka messages.
Since Fabric publishes CDC messages to Kafka, a dedicated CDC consumer can be defined in Broadway to subscribe CDC messages in a Broadway flow.
To customize Fabric Studio and add an additional CDC consumer for a Broadway flow, do the following:
Go to the project tree, right click the project name and select Open Folder.
Open the [project name].k2proj file to be edited.
Edit the DataChangeIndicators tag by adding the DataChange tag. By default, the DataChangeIndicators contain the Search consumer. Set the consumer name in the name and set the Option in the Options tag to data. See the following example:
<DataChangeIndicators>
<DataChange name="Search" enabled="true">
<Options>
<option>keyword</option>
<option>data</option>
<option>date</option>
</Options>
</DataChange>
<DataChange name="CDCBroadway" enabled="true">
<Options>
<option>data</option>
</Options>
</DataChange>
</DataChangeIndicators>
Select the LU table columns to be published to the CDC topic of Broadway.
Click for more information about CDC Implementation Steps.
Create a new Interface. Set the Interface Type to PubSub Configuration. In the config.ini, populate the Bootstrap Servers using the host and port of the Kafka server. For example: localhost:9092. Note that several servers delimited by a comma can be set.
Click for more information about PubSub Configuration interface.
Define a Broadway flow. Add the built-in Subscribe Actor and set its input parameters as follows:
Set the interface according to the settings in the Kafka interface added to the Fabric project.
Set the topic according to the CDC consumer name (CDCBroadway in the above example).
Notes:
Broadway has a queue of built-in Actors that manage the handling of Pub or Sub asynchronous messages and can subscribe to Apache Kafka messages.
Since Fabric publishes CDC messages to Kafka, a dedicated CDC consumer can be defined in Broadway to subscribe CDC messages in a Broadway flow.
To customize Fabric Studio and add an additional CDC consumer for a Broadway flow, do the following:
Go to the project tree, right click the project name and select Open Folder.
Open the [project name].k2proj file to be edited.
Edit the DataChangeIndicators tag by adding the DataChange tag. By default, the DataChangeIndicators contain the Search consumer. Set the consumer name in the name and set the Option in the Options tag to data. See the following example:
<DataChangeIndicators>
<DataChange name="Search" enabled="true">
<Options>
<option>keyword</option>
<option>data</option>
<option>date</option>
</Options>
</DataChange>
<DataChange name="CDCBroadway" enabled="true">
<Options>
<option>data</option>
</Options>
</DataChange>
</DataChangeIndicators>
Select the LU table columns to be published to the CDC topic of Broadway.
Click for more information about CDC Implementation Steps.
Create a new Interface. Set the Interface Type to PubSub Configuration. In the config.ini, populate the Bootstrap Servers using the host and port of the Kafka server. For example: localhost:9092. Note that several servers delimited by a comma can be set.
Click for more information about PubSub Configuration interface.
Define a Broadway flow. Add the built-in Subscribe Actor and set its input parameters as follows:
Set the interface according to the settings in the Kafka interface added to the Fabric project.
Set the topic according to the CDC consumer name (CDCBroadway in the above example).
Notes: