CommonDB Table Initialization
When a new node comes up online or rejoins the current Fabric cluster, all common tables must also be updated on this node. There are two enrollment options:
Option 1: Directly from Kafka
A new node connects directly to each Kafka topic for each Reference table to check whether a snapshot is available:
- The table is regularly synced as defined in its sync schedule and therefore a snapshot is available.
- A snapshot has already been created due to a similar request generated by another node. Note that in both cases, depending on the size of the update, the table's content is synced either from the Kafka message payload or from the Cassandra keyspace.
Option 2: From another node
A new node is online and requests an update:
- No snapshot is available in the corresponding Kafka topic. The new node requests for a snapshot to be created.
- The request is picked up by a node that in turn, prepares the snapshot and puts it either in Kafka (short) or Cassandra (long).
- The new node waits for the snapshot by listening to the relevant Kafka queue message to be published by the node preparing the snapshot.
Specific Cases Triggering Fabric Sessions Actions
What Happens When I Deploy a New Reference Table ?
- A new table/index is created in the CommonDB.
- A new Kafka topic is created if a new table has been added.
- A new Kafka consumer is created for each node.
- New sync jobs are started, also if the deployment process failed so as not to prevent existing synchronization processes.
What Happens When I Deploy an Existing Reference Table ?
- All currently running Reference table synchronization jobs stop.
- New sync jobs are started also if the deployment process failed so as not to prevent existing synchronization processes.
- All existing configuration parameters are used such as sync_job_retry_interval on the coordinating node.
What Happens When I Remove a Reference Table ?
- The Reference table is dropped from the CommonDB.
- All running Reference table synchronization jobs stop.
- All local topic consumers and producers on each node are dropped.
What Happens When CommonDB is Dropped ?
When running the following drop lutype k2_ref; command from any Fabric Node the following occurs across the entire Fabric cluster:
- All existing tables in common.db are dropped.
- All existing sync jobs are stopped.
- All existing consumer/producers are terminated.
CommonDB Table Initialization
When a new node comes up online or rejoins the current Fabric cluster, all common tables must also be updated on this node. There are two enrollment options:
Option 1: Directly from Kafka
A new node connects directly to each Kafka topic for each Reference table to check whether a snapshot is available:
- The table is regularly synced as defined in its sync schedule and therefore a snapshot is available.
- A snapshot has already been created due to a similar request generated by another node. Note that in both cases, depending on the size of the update, the table's content is synced either from the Kafka message payload or from the Cassandra keyspace.
Option 2: From another node
A new node is online and requests an update:
- No snapshot is available in the corresponding Kafka topic. The new node requests for a snapshot to be created.
- The request is picked up by a node that in turn, prepares the snapshot and puts it either in Kafka (short) or Cassandra (long).
- The new node waits for the snapshot by listening to the relevant Kafka queue message to be published by the node preparing the snapshot.
Specific Cases Triggering Fabric Sessions Actions
What Happens When I Deploy a New Reference Table ?
- A new table/index is created in the CommonDB.
- A new Kafka topic is created if a new table has been added.
- A new Kafka consumer is created for each node.
- New sync jobs are started, also if the deployment process failed so as not to prevent existing synchronization processes.
What Happens When I Deploy an Existing Reference Table ?
- All currently running Reference table synchronization jobs stop.
- New sync jobs are started also if the deployment process failed so as not to prevent existing synchronization processes.
- All existing configuration parameters are used such as sync_job_retry_interval on the coordinating node.
What Happens When I Remove a Reference Table ?
- The Reference table is dropped from the CommonDB.
- All running Reference table synchronization jobs stop.
- All local topic consumers and producers on each node are dropped.
What Happens When CommonDB is Dropped ?
When running the following drop lutype k2_ref; command from any Fabric Node the following occurs across the entire Fabric cluster:
- All existing tables in common.db are dropped.
- All existing sync jobs are stopped.
- All existing consumer/producers are terminated.
