TDM - Reference Implementation

TDM enables users to extract Reference tables from several source environments and to save them into the Cassandra DB. It also enables saving different versions of a Reference table and source environment into the Cassandra DB and creating and executing TDM load tasks to copy them from the Cassandra DB into the target environment.

Note that the TDM Reference solution is not based on Fabric Reference tables and that the tables are saved in Cassandra.

A TDM implementation has the following steps:

Step 1 - Populate trnRefList Translation

The list of Reference tables available for TDM extract tasks is populated in the trnRefList translation object. Populate trnRefList with the list of available Reference tables for each LU. The following settings must be populated for each record:

  • lu_name, populated by the LU name.
  • ID, populated by a sequence.
  • reference_table_name, populated by the Reference table.
  • schema_name, populated by the source DB schema name that stores the Reference table.
  • interface_name, the Reference table's source interface.
  • target_schema_name, populated by the target DB Schema's name that stores the Reference table.
  • target_interface_name, name of the Reference table's target interface.

Example:

Input

Output

lu_name

ID

reference_table_name

schema_name

interface_name

target_schema_name

target_interface_name

Customer

1

CUSTOMER_TYPE CUST CRM_DB TAR_CUST CRM_DB

Customer

2

ADDRESS_ZIP_CODES CUST CRM_DB TAR_CUST CRM_DB

Billing

3

BILL_TYPE BILL BILLING_DB TAR_BILL BILLING_DB

Note that is it also possible to populate the list of target's PK fields in trnRefList. These fields can be later used to customized the load flow to run an Upsert on the target reference table.

Step 2 - Creating Cassandra Tables for Reference Tables

Run the fnValidateAndRebuildRefTables job to create an empty Cassandra table for each Reference table in trnRefList. The tables are created in the k2view_tdm keyspace.

Step 3 - Create and Execute TDM Extract Tasks

TDM extract tasks store the selected Reference data in the related Cassandra DB table.

Click for more information on how the TDM stores Reference tables in the Cassandra DB.

Step 4 - Create and Execute TDM Load Tasks

The TDM GUI enables creating TDM load tasks to copy Reference tables that have been successfully extracted into the Cassandra DB. The execution of a load task runs the TDMReferenceLoader Broadway flow to copy the task's reference tables to the target environment.

Click for more information about generic TDM Broadway flows.

Previous

TDM - Reference Implementation

TDM enables users to extract Reference tables from several source environments and to save them into the Cassandra DB. It also enables saving different versions of a Reference table and source environment into the Cassandra DB and creating and executing TDM load tasks to copy them from the Cassandra DB into the target environment.

Note that the TDM Reference solution is not based on Fabric Reference tables and that the tables are saved in Cassandra.

A TDM implementation has the following steps:

Step 1 - Populate trnRefList Translation

The list of Reference tables available for TDM extract tasks is populated in the trnRefList translation object. Populate trnRefList with the list of available Reference tables for each LU. The following settings must be populated for each record:

  • lu_name, populated by the LU name.
  • ID, populated by a sequence.
  • reference_table_name, populated by the Reference table.
  • schema_name, populated by the source DB schema name that stores the Reference table.
  • interface_name, the Reference table's source interface.
  • target_schema_name, populated by the target DB Schema's name that stores the Reference table.
  • target_interface_name, name of the Reference table's target interface.

Example:

Input

Output

lu_name

ID

reference_table_name

schema_name

interface_name

target_schema_name

target_interface_name

Customer

1

CUSTOMER_TYPE CUST CRM_DB TAR_CUST CRM_DB

Customer

2

ADDRESS_ZIP_CODES CUST CRM_DB TAR_CUST CRM_DB

Billing

3

BILL_TYPE BILL BILLING_DB TAR_BILL BILLING_DB

Note that is it also possible to populate the list of target's PK fields in trnRefList. These fields can be later used to customized the load flow to run an Upsert on the target reference table.

Step 2 - Creating Cassandra Tables for Reference Tables

Run the fnValidateAndRebuildRefTables job to create an empty Cassandra table for each Reference table in trnRefList. The tables are created in the k2view_tdm keyspace.

Step 3 - Create and Execute TDM Extract Tasks

TDM extract tasks store the selected Reference data in the related Cassandra DB table.

Click for more information on how the TDM stores Reference tables in the Cassandra DB.

Step 4 - Create and Execute TDM Load Tasks

The TDM GUI enables creating TDM load tasks to copy Reference tables that have been successfully extracted into the Cassandra DB. The execution of a load task runs the TDMReferenceLoader Broadway flow to copy the task's reference tables to the target environment.

Click for more information about generic TDM Broadway flows.

Previous