TDM Extract Task

An Extract task extracts the selected entities and/or Reference tables from the selected source environment and saves this data in Fabric for later use.

An Extract task contains the following tabs:

• General
• Additional Execution Parameters
• Requested Entities - opened if the Data Type contains entities.
• Task Scheduling

When checking the Set Task Variables setting, a new Task Variables tab opens.

When setting the Data Type by checking the Reference setting, a Reference tab opens.

Additional Execution Parameters Tab

The following execution parameters are set on Extract tasks:

Data Versioning

• Check to set the task mode to Data Versioning.
• Leave the Data Versioning unchecked to create a regular mode task.

Data Type

Check Entities to extract the selected entities and/or Reference to extract the selected reference tables into the TDM warehouse (Fabric).

Click here for more information about the reference handling.

Set Sync Policy

This setting enables the user to change the default LUI sync mode (Sync ON) and extract the LUI from the data source whenever the task is executed (Sync FORCE).

Note that this setting is only available when the Data Versioning checkbox is cleared (regular task). If Data Versioning is checked, each task execution extracts the data from the data source and creates a new LUI.

Retention Period

This is the retention period set on the extracted LUIs. When this period ends, the LUIs are automatically deleted from Fabric and are no longer available.

Note that when the Retention Period is set to zero, no retention period is set on the extracted LUIs.

A retention period must be set on a Data Versioning extract task, i.e. the retention period must be set to a value greater than zero when the Data Versioning is checked.

The retention period is optional on a regular extract task.

The start date of the retention period is the task's execution time. The retention period can be set in minutes, hours, days, weeks, or years, depends on the maximum retention period set in the TDM DB. Both parameters - default retention period and maximum retention period - are set in the TDM DB.

Example:

The defaultPeriod parameter is set to 5 days and the maxRetentionPeriod parameter is set to 90 days.

The Retention Period window displays the following options:

• When Data Versioning is checked, the period is set by default to five days.
• When Data Versioning is cleared, the period is set by default to zero, i.e. no retention period is set for the extracted data.
• As a result of the above, the retention period can be set in the task to a maximum period of 90 days or 12 weeks. The Years option is not available since the maximum retention period is 90 days.

Additional Execution Parameters

Set Task Variables

Check to open the Task Variables tab and set the variable value on a task level.

Post Execution Processes

Select all, partial, or one post execution process of the selected BE.

Requested Entities Tab

This tab opens when the task's Data Type includes entities. This tab defines the subset of entities for the task:

The following selection methods are available on extract tasks:

• Select a predefined entity list: run the SQL query or the Broadway flow defined in the trnMigrateList translation object for the LU. This option is available only for the Admin and Environment owner users.
• Entity list: this is the default option. Populate the list of entities for the task. The entities populated should be separated with a comma.
• Custom logic: select a Broadway flow in order to both get the entity list for the task and to set the maximum number of entities for the task:

TDM 7.5 supports the creation of external input parameters on a Custom Logic Flow. In this case, the TDM GUI displays the input parameters in the task window, enabling the user to send values for these parameters. See example:

Notes:

• The maximum number of entities populated by the tester user is limited by their environment's permission set. This is the maximum number of entities of the task.

• The maximum number of entities in the task is limited to the number of entities returned from the Custom Logic flow or by the Max Number of Entities task parameter. For example: if the maximum number of entities in the task is 50, but the custom logic only returns 30 entities, the task will process 30 entities.

• It is possible to set an array value in a Custom Logic's parameter. The values are populated as a String with the delimiter, which is set in the Custom Logic Broadway flow. For example: 1,2,3 or CA,NY.

• Populate the Entity ID as populated in the source environment. For example, populate the Entities List with 1, 2 in order to extract Customers 1 and 2. The TDM execution process concatenates the required components to each Entity ID when building its LUI.