TDM Implementation - Supporting Non-JDBC Data Sources

How Do I Define the Entity List?

Extract All Entities Tasks

Create a Broadway flow to get the list of entities from the data source and populate it in the MigrateList MTable object.

For more information on how to populate the entity list based on a non-JDBC data source, read Step 6 - Get the Entity List for an Extract All Task Using a Broadway Flow section of the Broadway Flows Implementation article.

Using Custom Logic to Select Entities

Create Broadway flows to be executed by the task execution and get the entity list for the task.

For more information on how to build Custom Logic flows, read Step 7 - Optional - Build Broadway Flows for the Custom Logic Selection Method.

Building the LU Tables Populations

Step 1 - Main Source LU Table

The population of the main source LU table must be generated based on populationRootTable.pop.flow template imported from the TDM Library:

• Right-click the table name > New Table Population Flow From Template > populationRootTable.pop.flow. A pop-up window opens.
• Populate the pop-up window's settings as follows:
  • File Name: populate the file name by [LU Table Name].[flow name]
• Do not populate the TABLE_NAME , KEY, and the SOURCE_INTERFACE parameters when populating the main source LU table based on a non-JDBC data source.
• The generated flow is created with an empty source query in the Source stage. Replace the empty sourceDbQuery in the Source stage with the updated Actor to extract the data from the data source and link it to the main source LU table in the LU Table stage.

Step 2 - Other LU Tables

Create Broadway flows to populate the other LU tables.

Building the Broadway TDM Flows

Step 1 - Build Load and Delete Flows

The createLoadTableFlows and createDeleteTableFlow flows call the getDbTables function to get the target JDBC structure and therefore cannot be used for a non-JDBC target. Therefore, the load and delete flows need to be created manually:

• A separate load flow for each target object. The data is taken from the related LU table. Add sequence and masking handling if needed.

• A separate delete flow of each target object. The target keys are taken from the TAR_ LU tables.

• LoadAllTables flow: This flow executes the inner load flows on each target object in the correct order.

  View the below example:

  

​

​

The Get Entity Status Actors check the ENTITY_STATUS value and the flow runs the load flows of the next stage if the ENTITY_STATUS is not failed.

• DeleteAllTables flow: This flow executes the inner delete flows on each target object in the correct order.

  View the below example:

Step 2 - Generate Build Load and Delete Flows

Once all load and delete flows are ready, create an orchestrator using the TDMOrchestrator.flow template.