TDM Load Task

A Load task contains the Load task type and loads (provisions) the task's entities and/or Reference tables to the target environment. A load task can also delete entities from target environments before loading them and/or reserve the target entities in the TDM DB. Note that the Reserve task type is also checked by default when the Load task type is checked.

A Load task contains the following tabs:

• General
• Additional Execution Parameters
• Requested Entities - opens if the Data Type contains entities.
• Task Scheduling

When checking the Set Task Variables setting, a new Task Variables tab opens.

When checking the Reference Data Type setting, a Reference tab opens.

Additional Execution Parameters Tab

Loading Entities from a Physical Environment

The following execution parameters are set on Load tasks:

Data Versioning

• Check to set the task mode to Data Versioning.
• Leave the Data Versioning unchecked to create a regular mode task.

Data Type

Check Entities and/or Reference to load them to the target environment selected in the Provision to Environment setting.

Note that the Entities must be checked in the task if it also includes an entity reservation (the Reserve task type is checked together with the Load task type).

Click here for more information about the reference handling.

Set Sync Policy

This setting enables the user to change the default LUI sync mode (Sync ON). The following sync policies are available:

• Sync New Data - this is the default option (Sync ON).
• Do Not Sync From Source Data - get the data from the TDM warehouse (Fabric) and do not sync it from the data source (Sync OFF).
• Refresh all Data from Source - extract the data source whenever the task is executed (Sync FORCE).

Notes:

• This setting is only available when the Data Versioning checkbox is cleared (regular task). A Load Data Versioning task gets the data version from the TDM warehouse (Fabric) and reloads it to the target environment.
• The Do Not Sync From Source Data option is not displayed if the user selects a combination of Extract and Load task types.

Click for more information on how overriding the Sync mode and the Task Operation mode impacts the task execution process.

Retention Period

The Retention Period setting is displayed if the Sync Policy is not set to Do Not Sync From Source Data to enable setting a retention period on the task's entities if they are synced by the task execution. The default Retention Period is Do not Delete.

Reservation Period

The reservation period settings are displayed if the load task also reserves the loaded entities on the target environment (the Reserve task type is checked together with the Load task type).

Note that when the Reservation Period is set to zero, the entities are reserved for an unlimited period.

Only Admin and Environment owner users can reserve entities for an unlimited period. Users that are attached to the target environment as testers must set a reservation period. The maximum number of days of a reservation period is set in the tdm_general_parameters TDB DB table in the MAX_RESERVATION_DAYS_FOR_TESTER parameter.

The start date of the reservation period is the task's execution time. The reservation period can be set in minutes, hours, days or weeks.

Reservation Note

This setting has been added by TDM 7.5.2. The user can populate it with a free text. The reservation note is displayed in the Reserved Entities window (Notes fields) and can be used for filtering the reserved entities display. The reservation note can be edited using the Reserved Entities window.

Additional Execution Parameters

Set Task Variables

Check to open the Task Variables tab and set the variable value on a task level.

Replace Sequence

When checked, the task execution process replaces the IDs of all selected entities before loading them into the target. This option is required for avoiding key duplications if the testing environment is not empty and contains entities.

Notes:

• The Replace Sequence is not displayed on Data Versioning task, since the data version is loaded 'as is'.
• The Replace Sequence setting is disabled if the tester user does not have permissions to replace the sequence in the Write TDM Environment permission setting on the task's target environment.
• The Replace Sequence must be implemented in the Fabric implementation.

Post Execution Processes

Select all, partial, or one post execution process of the selected BE.

Loading Previously Generated Entities

If the Extract from Environment is Synthetic, the Load task gets from the TDM warehouse (Fabric) the synthetic entities previously generated by Generate tasks.

The following execution parameters are set on Load tasks when the Extract from Environment is Synthetic:

Main Differences Between Loading Synthetic Entities and Loading from a Physical Environment

The following settings are not displayed in the task when loading synthetic entities:

• The Data Versioning is not displayed. There is no Data Versioning on generated synthetic entities.

• The Data Type is Entities since the task loads generated synthetic entities. The Reference value is not displayed.

• The Set Sync Policy and Retention Period settings are not displayed since the task gets the synthetic entities from the TDM warehouse (Fabric).

Requested Entities Tab

This tab opens when the task's Data Type includes entities. This tab defines the subset of entities for the task:

The following selection methods are available on load tasks:

Entity list

This is the default option. Populate the list of entities for the task with entities separated with a comma between them. Note that a warning is given if the entity list has entities that are reserved for another user.

Custom Logic

select a Broadway flow in order to both get the entity list for the task and set the maximum number of entities for the task.

Notes:

• TDM 8.0 added integration of Broadway editors into the TDM portal when populating the Custom logic parameters in the task’s tabs.
• The Filter out Reserved Entities checkbox indicates if entities that are reserved for other users must be filtered out from the task's entity list. If checked, these entities are filtered out from the task's entity list.

Parameters

Select entities from a predefined list of parameters. You should set the parameters and their values and the maximum number of entities for the task.

Note that entities reserved for another user in the environment are excluded from the task. You can select one or several parameters. Additionally, you can add the same parameter multiple times with different values:

The list of parameters should be defined for each LU in the task BE in the Fabric project.

Use Parameters with Random Selection Checkbox

The Parameters selection has 2 modes:

• When checked (default), TDM randomly selects the entities from the list of all entities, those that match the selected parameters whereby each task execution gets different lists of entities that match the selected parameters. The Selection Method displayed in the Tasks List window is Parameters - selection based on parameters with random selection.

  Example:

  • Creating a task to load 5 customers using selected parameters. There are 800 customers that match the selected parameters. The task execution gets a random list of 5 customers from the list of 800 customers that match the selected parameters.

• When unchecked, it gets the first entities that match the selected parameters whereby each task execution gets the same list of entities that match the selected parameters. The Selection Method displayed in the Tasks List window is Parameters - selection based only on Parameters.

  Example:

  • Create a task to load 5 customers with selected parameters. There are 800 customers that match the selected parameters. The task execution gets the first 5 customers that match the selected parameters.

Filter out Reserved Entities Checkbox

The Filter out Reserved Entities checkbox has been added in TDM 7.6 and it indicates if entities that are reserved for other users must be filtered out from the task's entity list. If checked, these entities are filtered out from the task's entity list and from the Entities Matched setting.

For example:

• There are 500 customers with VIP Status 'Gold' and 10 of them are reserved for other users on the task's environments.
• If the Filter out Reserved Entities is checked, the Entities Matched is 490.
• If the Filter out Reserved Entities is cleared, the Entities Matched is 500.

How Do I Add a Condition?

To add a parameter:

• Click Add Condition.
• Select the parameter and the operator from the drop-down lists and populate their values.
• Add AND/OR operator to connect the parameter to the previous parameters or group. The TDM Portal displays the SQL query, which is built based on the selected parameters.

How Do I Populate a Parameter's Value?

There are several types of parameters:

• Combo - parameters with a limited number of values. The Task window displays a drop-down list of the parameters' values. Select a value from the drop-down list.

  Click for more information about setting a parameter as a combo parameter.

• Number - the TDM Portal displays the minimum and maximum values of this parameter. If the populated values exceed a parameter's range, an error message is displayed.

• Date - populate the value using the following format: YYYY-MM-DD.

  • Notes:

  • The date format is defined in DATETIME_FORMAT parameter of the config.ini Fabric file. The default format is 'yyyy-MM-dd HH:mm:ss.SSS'.

  • The parameter's search may be based on either a Date (removing the time from the Date format) or a Datetime.

    Examples:

    • 2021-12-28
    • 2022-04-03 10:00:00

  • The date value is set without quotation marks.

• Text - populate the value using a free text.

How Do I Add a Group of Parameters?

Click Add Group.

Note that nested groups of parameters can be added, for example, in order to define an inner group in an outer group.

How Do I Remove A Parameter or a Parameters Group?

1. Click next to the parameter in order to delete it.
2. Click Remove Group to remove a parameter's group.

Getting the Number of Matching Entities

Click Refresh next to the Entities Matched in order to calculate the number of entities that match the selected parameters. The Parameters selection supports the parent-child hierarchy relationship between the LUs of the selected BE. It can both cross-check the matching entities of a selected combination of parameters and take into consideration parameters from different LUs in the same BE hierarchy. For example, if customers living in New York with scheduled visits (see the screenshot below) are selected, TDM checks the visits of each patient living in New York and then checks their status.

• Click to display the number of matching entities according to the parameters’ conditions.

Click for more information about the TDM parameters tables and View created by TDM in the TDM DB to display a hierarchical view of TDM parameters.

Entity Clone

Create X clones of the selected entity in the target environment. When selecting this method, the real entity ID must be populated in order to be replicated.

For example, if the Number of Clones setting is populated with 10 and the Entity ID is set to 645, the task creates 10 clones (replicas) of Entity ID 645 in the target environment.

The task replaces the sequences of each replica in order to avoid duplicated sequences in the target environment.

Testers may select this option only if they are permitted to clone an entity in the task's target environment.

Random Selection

Get a random list of entities from the LU PARAMS table created in the TDM DB for the root LU of the task's BE. Entities that are reserved for other users are excluded from the selected entities.

Testers can select this option only if they are permitted to do so in the task's target environment.

Filter out Reserved Entities Checkbox

The Filter out Reserved Entities checkbox indicates if entities that are reserved for other users must be filtered out from the task's entity list. If checked, these entities are filtered out from the task's entity list.