All configuration parameters are located in the common_area_config section of the config.ini file located in the Fabric Home directory of each Fabric Node.
Fabric allows users to use server's RAM to distribute messages (in a single-node configuration only) in order to avoid the need for Kafka configurations. It is very important to note that in that case, overall data persistency cannot be ensured.
# Messages distribution mode - MEMORY / KAFKA
MESSAGES_BROKER_TYPE=MEMORY
Internal distribution of long snapshot / transaction:
MESSAGES_INTERNAL_BROKER_TYPE=PUB_SUB
When a message is not processed properly, Fabric allows for a configurable number of retrials to be executed.
# Max retry for a poisoned message
PROCESS_UPDATE_MESSAGE_RETRIES_COUNT=3
# Max idle time when consuming snapshot messages, if processing hangs on longer the consumer will return IDLE_TIMOUT
# 16.6 hrs
CONSUMER_IDLE_TIME=60000
OPERATION_RETRIES_COUNT=3
Using the affinity parameter, the table synchronization process can be allocated to a specific node (by default, no affinity is set).
SYNC_JOBS_AFFINITY=10.23.10.11
When set to true, all topics for tables being dropped will be removed automatically:
DELETE_TOPICS_ON_DROP=true
If 2500 insert commands are required, each bulk of 1000 commands is sent to Kafka, while the update content is kept in Cassandra. These 2500 inserts are divided into 3 transactions, the first 2 containing 1000 rows to insert, and the third one 500.
TRANSACTION_BULK_SIZE=1000
MAX_TRANSACTIONS_COMMIT=100
# defines maximum snapshot size - in the case below set to 100 MB
#SNAPSHOT_SIZE=100
# in seconds - Default is one week
# below has been set to one day
COMMONS_TABLE_TTL=86400
All the Kafka connection settings are defined in the [default_pubsub] section of the config.ini and are applicable across various Fabric processes including the Common DB connection to Kafka.
When it is required to have different Kafka settings for Common DB, it can be done using the [common_area_pubsub] section. This section does not have to include all the parameters, but only those which should override the default section's settings.
Click for more information about PubSub Configuration.
COMMONS_SNAP_TABLE=snapshots
Snapshot table TTL in seconds - Default is one week
#COMMONS_TABLE_TTL=8640
Maximum idle time when consuming snapshot messages
CASSANDRA_WAIT_MESSAGE_TIMEOUT=60000
Fabric provides the following statistics about CommonDB-related operations. These can be viewed sing the JMX metrics tool located at: http://<IP address>:3213/static/status/status.html
commonUpdate
Common area update messages count per table formatted as follow: count, timestamp, sinceDate.
commonTransactions
Common area transaction/commit count per table formatted as follow: count, timestamp, sinceDate.
commonKafkaRead
Common area Kafka message read count per table formatted as follow:
count, timestamp, sinceDate.
commonKafkaCommit
Common area Kafka commit count per table" formatted as follow:
count, timestamp, sinceDate.
commonSnapshot
Common area snapshot bulk and statement count per table formatted as follow:
last, totalLast, average, count, timestamp, sinceDate.
commonSnapshotDuration
Common area snapshot duration per table formatted as follow:
last, totalLast, average, count, timestamp, sinceDate.
commonMessagesBulkProcessDuration
Common area process bulk messages duration, including failures and retries, formatted as follow:
last, totalLast, average, count, timestamp, sinceDate.
All configuration parameters are located in the common_area_config section of the config.ini file located in the Fabric Home directory of each Fabric Node.
Fabric allows users to use server's RAM to distribute messages (in a single-node configuration only) in order to avoid the need for Kafka configurations. It is very important to note that in that case, overall data persistency cannot be ensured.
# Messages distribution mode - MEMORY / KAFKA
MESSAGES_BROKER_TYPE=MEMORY
Internal distribution of long snapshot / transaction:
MESSAGES_INTERNAL_BROKER_TYPE=PUB_SUB
When a message is not processed properly, Fabric allows for a configurable number of retrials to be executed.
# Max retry for a poisoned message
PROCESS_UPDATE_MESSAGE_RETRIES_COUNT=3
# Max idle time when consuming snapshot messages, if processing hangs on longer the consumer will return IDLE_TIMOUT
# 16.6 hrs
CONSUMER_IDLE_TIME=60000
OPERATION_RETRIES_COUNT=3
Using the affinity parameter, the table synchronization process can be allocated to a specific node (by default, no affinity is set).
SYNC_JOBS_AFFINITY=10.23.10.11
When set to true, all topics for tables being dropped will be removed automatically:
DELETE_TOPICS_ON_DROP=true
If 2500 insert commands are required, each bulk of 1000 commands is sent to Kafka, while the update content is kept in Cassandra. These 2500 inserts are divided into 3 transactions, the first 2 containing 1000 rows to insert, and the third one 500.
TRANSACTION_BULK_SIZE=1000
MAX_TRANSACTIONS_COMMIT=100
# defines maximum snapshot size - in the case below set to 100 MB
#SNAPSHOT_SIZE=100
# in seconds - Default is one week
# below has been set to one day
COMMONS_TABLE_TTL=86400
All the Kafka connection settings are defined in the [default_pubsub] section of the config.ini and are applicable across various Fabric processes including the Common DB connection to Kafka.
When it is required to have different Kafka settings for Common DB, it can be done using the [common_area_pubsub] section. This section does not have to include all the parameters, but only those which should override the default section's settings.
Click for more information about PubSub Configuration.
COMMONS_SNAP_TABLE=snapshots
Snapshot table TTL in seconds - Default is one week
#COMMONS_TABLE_TTL=8640
Maximum idle time when consuming snapshot messages
CASSANDRA_WAIT_MESSAGE_TIMEOUT=60000
Fabric provides the following statistics about CommonDB-related operations. These can be viewed sing the JMX metrics tool located at: http://<IP address>:3213/static/status/status.html
commonUpdate
Common area update messages count per table formatted as follow: count, timestamp, sinceDate.
commonTransactions
Common area transaction/commit count per table formatted as follow: count, timestamp, sinceDate.
commonKafkaRead
Common area Kafka message read count per table formatted as follow:
count, timestamp, sinceDate.
commonKafkaCommit
Common area Kafka commit count per table" formatted as follow:
count, timestamp, sinceDate.
commonSnapshot
Common area snapshot bulk and statement count per table formatted as follow:
last, totalLast, average, count, timestamp, sinceDate.
commonSnapshotDuration
Common area snapshot duration per table formatted as follow:
last, totalLast, average, count, timestamp, sinceDate.
commonMessagesBulkProcessDuration
Common area process bulk messages duration, including failures and retries, formatted as follow:
last, totalLast, average, count, timestamp, sinceDate.