Knowledge Base Academy Community My Tickets
Fabric 7.2
Fabric 7.2 Fabric 7.1 Fabric 7.0 Fabric 6.5 Fabric 6.4

Building Entity Lists on a Task's LUs

The task execution process executes the task's LUs from parent to child.

Click for more information about the execution order of hierarchical LUs.

The task execution process builds an entity list for each LU, as described below. Root LUs and Children LUs require different steps:

Root LUs

The entity list of the root LUs is based on the task's entity selection method:

Entity Selection Method

Where the Entities are Taken From

Task Actions

Entity list

Run the task on the list of entities that are set in the task itself or in the task overridden execution parameters.

All task actions

Custom logic

Run the selected Broadway flow to get the entity list.

All task actions

Select a predefined entity list

Run the SQL query or the Broadway flow defined in the MigrateList MTable for the LU.

Extract tasks

Select all entities of the selected version

Select all entities that are successfully extracted by the selected data version. The entities are selected from TASK_EXECUTION_ENTITIES TDM DB table based on the task_execution_id of the selected data version.

Load data versioning tasks

Parameters

Select the entities based on the task's parameters from the LU parameters tables. These tables are created in the TDM DB for each BE and source environment combination.

Regular load tasks (Data Versioning is cleared)

Entity clone

Create duplications on the target environment of the Entity ID set in the task. Note that only one LUI is created on the entity ID.

Regular load tasks (Data Versioning is cleared)

Random selection

Randomly select the entities from the <LU Name>_<params> table.

Regular load tasks (Data Versioning is cleared)

Generate Synthetic Entities

Create a list of dummy entities for generating synthetic entities tasks.

Generate tasks (Generate task action is selected).

Running a Broadway Flow to Get the Task's Entity List

The task's entity list is generated by a Broadway flow in the following scenarios:

  • when using the Custom Logic selection method.

    Click here for more information about the custom logic modes.

  • when using the Select a predefined entity list selection method on an extract task when the MigrateList object is populated with a Broadway flow.

  • when generating a list of dummy entities for a synthetic data generation task. The prefix Synthetic is concatenated to each generated ID to create a dummy LUI, e.g., Synthetic_84.

    Click here for more information about synthetic data generation tasks.

Children LUs

The entity list of a child LU must include all IDs relating to parent IDs that have been successfully processed by the task execution.

Click for an execution of hierarchical BE example.

The generated entity list is based on a JOIN of the task_execution_entities and the TDM relationship tables:

 

Task Actions

 SQL Query
  • Extract
  • Extract + Load
  • Extract + Load + Reserve
  • Load
  • Load + Reserve
  • Generate
  • Generate + Load
  • Generate + Load + Reserve

SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_relation_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and t.source_env = rel.source_env
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>
and rel.version_name = <empty string on a regular task
and the selected version name on a Data Versioning task>;
  • Extract + Load + Delete
  • Extract + Load + Delete+ Reserve
  • Load + Delete
  • Load + Delete + Reserve

SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_relation_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and t.source_env = rel.source_env
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>
and rel.version_name = <empty string on a regular task
and the selected version name on a Data Versioning task>
UNION
SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_rel_tar_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and rel.target_env = <target environment name>
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>;
  • Delete only

SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_rel_tar_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and rel.target_env = <target environment name>
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>;

Previous

Building Entity Lists on a Task's LUs

The task execution process executes the task's LUs from parent to child.

Click for more information about the execution order of hierarchical LUs.

The task execution process builds an entity list for each LU, as described below. Root LUs and Children LUs require different steps:

Root LUs

The entity list of the root LUs is based on the task's entity selection method:

Entity Selection Method

Where the Entities are Taken From

Task Actions

Entity list

Run the task on the list of entities that are set in the task itself or in the task overridden execution parameters.

All task actions

Custom logic

Run the selected Broadway flow to get the entity list.

All task actions

Select a predefined entity list

Run the SQL query or the Broadway flow defined in the MigrateList MTable for the LU.

Extract tasks

Select all entities of the selected version

Select all entities that are successfully extracted by the selected data version. The entities are selected from TASK_EXECUTION_ENTITIES TDM DB table based on the task_execution_id of the selected data version.

Load data versioning tasks

Parameters

Select the entities based on the task's parameters from the LU parameters tables. These tables are created in the TDM DB for each BE and source environment combination.

Regular load tasks (Data Versioning is cleared)

Entity clone

Create duplications on the target environment of the Entity ID set in the task. Note that only one LUI is created on the entity ID.

Regular load tasks (Data Versioning is cleared)

Random selection

Randomly select the entities from the <LU Name>_<params> table.

Regular load tasks (Data Versioning is cleared)

Generate Synthetic Entities

Create a list of dummy entities for generating synthetic entities tasks.

Generate tasks (Generate task action is selected).

Running a Broadway Flow to Get the Task's Entity List

The task's entity list is generated by a Broadway flow in the following scenarios:

  • when using the Custom Logic selection method.

    Click here for more information about the custom logic modes.

  • when using the Select a predefined entity list selection method on an extract task when the MigrateList object is populated with a Broadway flow.

  • when generating a list of dummy entities for a synthetic data generation task. The prefix Synthetic is concatenated to each generated ID to create a dummy LUI, e.g., Synthetic_84.

    Click here for more information about synthetic data generation tasks.

Children LUs

The entity list of a child LU must include all IDs relating to parent IDs that have been successfully processed by the task execution.

Click for an execution of hierarchical BE example.

The generated entity list is based on a JOIN of the task_execution_entities and the TDM relationship tables:

 

Task Actions

 SQL Query
  • Extract
  • Extract + Load
  • Extract + Load + Reserve
  • Load
  • Load + Reserve
  • Generate
  • Generate + Load
  • Generate + Load + Reserve

SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_relation_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and t.source_env = rel.source_env
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>
and rel.version_name = <empty string on a regular task
and the selected version name on a Data Versioning task>;
  • Extract + Load + Delete
  • Extract + Load + Delete+ Reserve
  • Load + Delete
  • Load + Delete + Reserve

SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_relation_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and t.source_env = rel.source_env
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>
and rel.version_name = <empty string on a regular task
and the selected version name on a Data Versioning task>
UNION
SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_rel_tar_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and rel.target_env = <target environment name>
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>;
  • Delete only

SELECT rel.lu_type2_eid as child_entity_id
FROM task_execution_entities t, tdm_lu_type_rel_tar_eid rel
where t.task_execution_id= <task execution id>
and t.execution_status = 'completed'
and t.lu_name = <parent lu name>
and rel.target_env = <target environment name>
and t.lu_name = rel.lu_type_1
and t.iid = rel. lu_type1_eid
and rel.lu_type_2= <child lu name>;

Previous

© Copyright 2024 K2VIEW