The task execution process executes the task's LUs from parent to child.
Click for more information about the execution order of hierarchical LUs.
The task execution process builds an entity list for each LU, as described below. Root LUs and Children LUs require different steps:
The entity list of the root LUs is based on the task's entity selection method:
Entity Selection Method |
Where the Entities are Taken From |
Task Actions |
Entity list |
Run the task on the list of entities that are set in either the task itself or the task overridden execution parameters. |
All task actions |
Custom logic |
Run the selected Broadway flow to get the entity list. |
All task actions |
Select a predefined entity list |
Run either the SQL query or the Broadway flow that are defined in the MigrateList MTable for the LU. |
Extract tasks |
Select all entities of the selected version |
Select all entities that are successfully extracted by the selected data version. The entities are selected from the TASK_EXECUTION_ENTITIES TDM DB table based on the task_execution_id of the selected data version. |
Load data versioning tasks |
Parameters |
Select the entities based on the task's parameters from the LU parameters tables. These tables are created in the TDM DB for each BE and source environment combination. |
Regular load tasks (Data Versioning is cleared) |
Entity clone |
Create duplications on the target environment of the first Entity ID returned by the task's subset. Only one LUI is created on the entity ID. Note that from TDM 9.0 onwards, entity clone is defined as an attribute on the target and it can be added to regardless of the subset's selection method. |
Regular load tasks (Data Versioning is cleared) |
Random selection |
Randomly select the entities from the <LU Name>_<params> table. |
Regular load tasks (Data Versioning is cleared) |
Rule-based Synthetic Entities' Generation |
Create a list of dummy entities for generating synthetic entities tasks. |
Rule-based entites' generation. |
AI-based Synthetic Entities' Generation |
The AI generates the required entities with a dummy ID for each generated entity. TDM creates a unique LUI on each generated entity before importing the entity to Fabric. The mapping between the AI-generated ID and its Fabric LUI (imported LUI) is kept in tdm_ai_gen_iid_mapping TDM DB table. |
AI-based entites' generation. |
The task's entity list is generated by a Broadway flow in the following scenarios:
when using the Custom Logic selection method.
Click here for more information about the custom logic modes.
when using the Select a predefined entity list selection method on an extract task when the MigrateList object is populated with a Broadway flow.
when generating a list of dummy entities for a synthetic data generation task. The prefix Synthetic is concatenated to each generated ID to create a dummy LUI, e.g., Synthetic_84.
Click here for more information about synthetic data generation tasks.
The entity list of a child LU must include all IDs relating to parent IDs that have been successfully processed by the task execution.
Click for an execution of hierarchical BEs example.
The child LU's entity list is based on a JOIN of the task_execution_entities and the TDM relationship tables:
The task execution process executes the task's LUs from parent to child.
Click for more information about the execution order of hierarchical LUs.
The task execution process builds an entity list for each LU, as described below. Root LUs and Children LUs require different steps:
The entity list of the root LUs is based on the task's entity selection method:
Entity Selection Method |
Where the Entities are Taken From |
Task Actions |
Entity list |
Run the task on the list of entities that are set in either the task itself or the task overridden execution parameters. |
All task actions |
Custom logic |
Run the selected Broadway flow to get the entity list. |
All task actions |
Select a predefined entity list |
Run either the SQL query or the Broadway flow that are defined in the MigrateList MTable for the LU. |
Extract tasks |
Select all entities of the selected version |
Select all entities that are successfully extracted by the selected data version. The entities are selected from the TASK_EXECUTION_ENTITIES TDM DB table based on the task_execution_id of the selected data version. |
Load data versioning tasks |
Parameters |
Select the entities based on the task's parameters from the LU parameters tables. These tables are created in the TDM DB for each BE and source environment combination. |
Regular load tasks (Data Versioning is cleared) |
Entity clone |
Create duplications on the target environment of the first Entity ID returned by the task's subset. Only one LUI is created on the entity ID. Note that from TDM 9.0 onwards, entity clone is defined as an attribute on the target and it can be added to regardless of the subset's selection method. |
Regular load tasks (Data Versioning is cleared) |
Random selection |
Randomly select the entities from the <LU Name>_<params> table. |
Regular load tasks (Data Versioning is cleared) |
Rule-based Synthetic Entities' Generation |
Create a list of dummy entities for generating synthetic entities tasks. |
Rule-based entites' generation. |
AI-based Synthetic Entities' Generation |
The AI generates the required entities with a dummy ID for each generated entity. TDM creates a unique LUI on each generated entity before importing the entity to Fabric. The mapping between the AI-generated ID and its Fabric LUI (imported LUI) is kept in tdm_ai_gen_iid_mapping TDM DB table. |
AI-based entites' generation. |
The task's entity list is generated by a Broadway flow in the following scenarios:
when using the Custom Logic selection method.
Click here for more information about the custom logic modes.
when using the Select a predefined entity list selection method on an extract task when the MigrateList object is populated with a Broadway flow.
when generating a list of dummy entities for a synthetic data generation task. The prefix Synthetic is concatenated to each generated ID to create a dummy LUI, e.g., Synthetic_84.
Click here for more information about synthetic data generation tasks.
The entity list of a child LU must include all IDs relating to parent IDs that have been successfully processed by the task execution.
Click for an execution of hierarchical BEs example.
The child LU's entity list is based on a JOIN of the task_execution_entities and the TDM relationship tables: