TDM Extract Task
An Extract task extracts the selected entities and/or Reference tables from the selected source environment. The data can be saved in Fabric (TDM warehouse) for a later use.
An Extract task contains the following tabs:
- General
- Additional Execution Parameters
- Requested Entities - opened if the Data Type contains entities.
- Task Scheduling
When checking the Set Task Variables setting, a new Task Variables tab opens.
When setting the Data Type by checking the Reference setting, a Reference tab opens.
Additional Execution Parameters Tab
The following execution parameters are set on Extract tasks:
Data Versioning
- Check to set the task mode to Data Versioning.
- Leave the Data Versioning unchecked to create a regular mode task.
Data Type
Check Entities to extract the selected entities and/or Reference to extract the selected reference tables into the TDM warehouse (Fabric).
Click here for more information about the reference handling.
Set Sync Policy
This setting enables the user to change the default LUI sync mode (Sync ON) and extract the LUI from the data source whenever the task is executed (Sync FORCE).
Note that this setting is only available when the Data Versioning checkbox is cleared (regular task). If Data Versioning is checked, each task execution extracts the data from the data source and creates a new LUI.
Retention Period
The retention period set on the extracted entities and/or Reference tables. When this period ends, the task's entities and/or Reference tables are automatically deleted from Fabric and are no longer available.
Retention Period Values
Do not Delete - do not delete from Fabric.
Do not Retain - avoid saving the task's entities and/or Reference tables in Fabric (instead of saving and deleting). This option can be used, for example, to run an Extract task on a large subset of entities in order to populate the TDM parameter’s tables on each entity without saving the entities into Fabric.
Set unit of measure (Minutes, Hours, Days …) and value. For example, save the data in Fabric for 2 days. After 2 days the data are automatically deleted from Fabric.
Note that the retention period can be set in minutes, hours, days, weeks or years, depending on the maximum retention period set in the TDM DB. Both parameters - default retention period and maximum retention period - are set in the TDM DB.
Default Retention Period
- Data versioning is cleared - Do not Delete.
- Data versioning is checked - 5 days. A retention period must be set on a Data Versioning Extract task, i.e. the retention period must be set to a value greater than zero when the Data Versioning is checked.
Additional Execution Parameters
Set Task Variables
Check to open the Task Variables tab and set the variable value on a task level.
Post Execution Processes
Select all, partial, or one post execution process of the selected BE.
Requested Entities Tab
This tab opens when the task's Data Type includes entities. This tab defines the subset of entities for the task:
The following selection methods are available on extract tasks:
Select a predefined entity list: run the SQL query or the Broadway flow defined in the trnMigrateList translation object for the LU. This option is available only for the Admin and Environment owner users.
Entity list: this is the default option. Populate the list of entities for the task. The entities populated should be separated with a comma.
Custom logic: select a Broadway flow in order to both get the entity list for the task and set the maximum number of entities for the task.
Note that the TDM 8.0 added integration of Broadway editors into the TDM portal when populating the Custom logic parameters in the task’s tabs:
Notes:
The maximum number of entities populated by the tester user is limited by their environment's permission set. This is the maximum number of entities of the task.
The maximum number of entities in the task is limited to the number of entities returned from the Custom Logic flow or by the Max Number of Entities task parameter. For example: if the maximum number of entities in the task is 50, but the custom logic only returns 30 entities, the task will process 30 entities.
Populate the Entity ID as populated in the source environment. For example, populate the Entities List with 1, 2 in order to extract Customers 1 and 2. The TDM execution process concatenates the required components to each Entity ID when building its LUI.
TDM Extract Task
An Extract task extracts the selected entities and/or Reference tables from the selected source environment. The data can be saved in Fabric (TDM warehouse) for a later use.
An Extract task contains the following tabs:
- General
- Additional Execution Parameters
- Requested Entities - opened if the Data Type contains entities.
- Task Scheduling
When checking the Set Task Variables setting, a new Task Variables tab opens.
When setting the Data Type by checking the Reference setting, a Reference tab opens.
Additional Execution Parameters Tab
The following execution parameters are set on Extract tasks:
Data Versioning
- Check to set the task mode to Data Versioning.
- Leave the Data Versioning unchecked to create a regular mode task.
Data Type
Check Entities to extract the selected entities and/or Reference to extract the selected reference tables into the TDM warehouse (Fabric).
Click here for more information about the reference handling.
Set Sync Policy
This setting enables the user to change the default LUI sync mode (Sync ON) and extract the LUI from the data source whenever the task is executed (Sync FORCE).
Note that this setting is only available when the Data Versioning checkbox is cleared (regular task). If Data Versioning is checked, each task execution extracts the data from the data source and creates a new LUI.
Retention Period
The retention period set on the extracted entities and/or Reference tables. When this period ends, the task's entities and/or Reference tables are automatically deleted from Fabric and are no longer available.
Retention Period Values
Do not Delete - do not delete from Fabric.
Do not Retain - avoid saving the task's entities and/or Reference tables in Fabric (instead of saving and deleting). This option can be used, for example, to run an Extract task on a large subset of entities in order to populate the TDM parameter’s tables on each entity without saving the entities into Fabric.
Set unit of measure (Minutes, Hours, Days …) and value. For example, save the data in Fabric for 2 days. After 2 days the data are automatically deleted from Fabric.
Note that the retention period can be set in minutes, hours, days, weeks or years, depending on the maximum retention period set in the TDM DB. Both parameters - default retention period and maximum retention period - are set in the TDM DB.
Default Retention Period
- Data versioning is cleared - Do not Delete.
- Data versioning is checked - 5 days. A retention period must be set on a Data Versioning Extract task, i.e. the retention period must be set to a value greater than zero when the Data Versioning is checked.
Additional Execution Parameters
Set Task Variables
Check to open the Task Variables tab and set the variable value on a task level.
Post Execution Processes
Select all, partial, or one post execution process of the selected BE.
Requested Entities Tab
This tab opens when the task's Data Type includes entities. This tab defines the subset of entities for the task:
The following selection methods are available on extract tasks:
Select a predefined entity list: run the SQL query or the Broadway flow defined in the trnMigrateList translation object for the LU. This option is available only for the Admin and Environment owner users.
Entity list: this is the default option. Populate the list of entities for the task. The entities populated should be separated with a comma.
Custom logic: select a Broadway flow in order to both get the entity list for the task and set the maximum number of entities for the task.
Note that the TDM 8.0 added integration of Broadway editors into the TDM portal when populating the Custom logic parameters in the task’s tabs:
Notes:
The maximum number of entities populated by the tester user is limited by their environment's permission set. This is the maximum number of entities of the task.
The maximum number of entities in the task is limited to the number of entities returned from the Custom Logic flow or by the Max Number of Entities task parameter. For example: if the maximum number of entities in the task is 50, but the custom logic only returns 30 entities, the task will process 30 entities.
Populate the Entity ID as populated in the source environment. For example, populate the Entities List with 1, 2 in order to extract Customers 1 and 2. The TDM execution process concatenates the required components to each Entity ID when building its LUI.
