Task - TDM DB Tables

TDM settings are saved in the TDM PostgreSQL DB.

TDM tasks update the following TDM DB tables:

TASKS

This table holds all TDM tasks defined in the TDM Portal.

  • A new record is created for each task.

  • Each task record gets a unique task_id sequence, which is the table's PK.

    Task General Information

    • task_title - the task name. To prevent creating several active tasks with the same name, the task_title column has a unique index when the status is Active.
    • task_description
    • task_type - Extract , Load, Generate, AI-Generated, Training, Delete, or Reserve.
    • be_id - the task's BE. The be_id can be linked to the product_logical_units TDM DB table.
    • number_of_entities - populated by the number of entities in the task.
    • load_entities - populated with true for Load tasks. Otherwise, populated with false.
    • delete_before_load - populated with true for Delete or Load and Delete tasks. Otherwise, populated with false.
    • task_created_by, and task_last_updated_by - populated by the name of the user who creates the task. From TDM 9.0 onwards, the user's Fabric role is concatenated to the user name. This is required in order to identify the task creator's Fabric role when the users are managed and kept by the organization’s identity provider (IDP), for example: SAML. The task creator's Fabric role needs to be identified in order to decide if a tester user can execute a task created by another tester.
    • task_creation_date and task_last_updated_date - populated by the task's creation datetime.

    Reservation Information

    • reserve_ind - indicates whether the task has reserved entities on the environment.
    • reserve retention period fields - defines the reservation period on the task's entities.

    Task Status

    • task_status: each task is created in Active task_status. Deleted tasks have an Inactive task_status and are not physically deleted from this table.
    • task_execution_status:
    • Active - the task can be executed.
    • onHold - pause the task and set it to On Hold.
    • Inactive - a deleted task.

    Requested Entities Columns

These columns are used by the TDM task execution process for creating the entities list of the root LUs for each task:

  • selection method: populated based on the task's data type (entities or tables) and the subset selection method.

  • clone_ind: this field is set to true if the task needs to clone an entity.

  • num_of_entities: number of provisioned entities.

  • selection_param_value: populated when the task selection method is either Entity List, Business Parameters, or Custom Logic selection methods:

Task selection method

selection_method field (letter symbol)

Selection_param_value field

Entity List

L

Populated by the list of entities separated by a comma.

Examples:

  • 1,2,3,4
  • 66

Business Parameters, with or without a random selection

P and PR

Populated by the SQL WHERE statement, generated by the selected parameters.

Example:

  • (( 'New York' = ANY("Customer.CITY") ))

Custom Logic

C Populated with the name of the selected Broadway flow that runs and gets the entity list for the task execution.
  • parameters - populated when the task's subset is based on Business parameters selection method.

    Data Versioning Columns

    • version_ind - populated with true in a Data Snapshot (Version) task.
    • selected_version_task_exe_id - the task execution id of the selected data snapshot (version) loading a selected data snapshot (version).

AI-based Generation Columns

  • selected_subset_task_exe_id - the task execution of the selected training model (data generator) for the AI-based generation.

    Environments Columns

    Source Environment

  • source_env_name

  • source_environment_id

Target Environment

  • environment_id

Task Execution Parameters

Scheduling Parameters

  • Scheduler - set based on the task's Scheduler:
    • If the Scheduler is not set (that is, the task is executed by request), the scheduler is populated by immediate.
    • If the Scheduler is set (a scheduled task), the scheduler is populated by a crontab based on the selected scheduling parameters of the task.
  • scheduling_end_date - the scheduler is populated by the End Date if set on scheduled tasks.

Extract Tasks - Retention Period

  • retention_period_type and retention_period_value.

Other Parameters

  • replace_sequences
  • sync_mode -
    • If the task does not override the sync mode, this column remains empty.
    • If the task overrides the sync mode, this parameter is populated by FORCE or OFF.

TASKS_LOGICAL_UNITS

This table holds the LUs list of each task. A separate record is created for each LU.

TASKS_EXE_PROCESS


This table holds a task's pre and post execution processes. A new record is created for each pre or post execution process.

TASK_GLOBALS

This table holds all variables that are overridden by the task. A separate record is created for each variable.

TDM_GENERATE_TASK_FIELD_MAPPINGS

This table is populated with data generation parameters for Generate tasks.

TASK_REF_TABLES

This table holds a list of the task's tables. A separate record is created for each table.

Previous

Task - TDM DB Tables

TDM settings are saved in the TDM PostgreSQL DB.

TDM tasks update the following TDM DB tables:

TASKS

This table holds all TDM tasks defined in the TDM Portal.

  • A new record is created for each task.

  • Each task record gets a unique task_id sequence, which is the table's PK.

    Task General Information

    • task_title - the task name. To prevent creating several active tasks with the same name, the task_title column has a unique index when the status is Active.
    • task_description
    • task_type - Extract , Load, Generate, AI-Generated, Training, Delete, or Reserve.
    • be_id - the task's BE. The be_id can be linked to the product_logical_units TDM DB table.
    • number_of_entities - populated by the number of entities in the task.
    • load_entities - populated with true for Load tasks. Otherwise, populated with false.
    • delete_before_load - populated with true for Delete or Load and Delete tasks. Otherwise, populated with false.
    • task_created_by, and task_last_updated_by - populated by the name of the user who creates the task. From TDM 9.0 onwards, the user's Fabric role is concatenated to the user name. This is required in order to identify the task creator's Fabric role when the users are managed and kept by the organization’s identity provider (IDP), for example: SAML. The task creator's Fabric role needs to be identified in order to decide if a tester user can execute a task created by another tester.
    • task_creation_date and task_last_updated_date - populated by the task's creation datetime.

    Reservation Information

    • reserve_ind - indicates whether the task has reserved entities on the environment.
    • reserve retention period fields - defines the reservation period on the task's entities.

    Task Status

    • task_status: each task is created in Active task_status. Deleted tasks have an Inactive task_status and are not physically deleted from this table.
    • task_execution_status:
    • Active - the task can be executed.
    • onHold - pause the task and set it to On Hold.
    • Inactive - a deleted task.

    Requested Entities Columns

These columns are used by the TDM task execution process for creating the entities list of the root LUs for each task:

  • selection method: populated based on the task's data type (entities or tables) and the subset selection method.

  • clone_ind: this field is set to true if the task needs to clone an entity.

  • num_of_entities: number of provisioned entities.

  • selection_param_value: populated when the task selection method is either Entity List, Business Parameters, or Custom Logic selection methods:

Task selection method

selection_method field (letter symbol)

Selection_param_value field

Entity List

L

Populated by the list of entities separated by a comma.

Examples:

  • 1,2,3,4
  • 66

Business Parameters, with or without a random selection

P and PR

Populated by the SQL WHERE statement, generated by the selected parameters.

Example:

  • (( 'New York' = ANY("Customer.CITY") ))

Custom Logic

C Populated with the name of the selected Broadway flow that runs and gets the entity list for the task execution.
  • parameters - populated when the task's subset is based on Business parameters selection method.

    Data Versioning Columns

    • version_ind - populated with true in a Data Snapshot (Version) task.
    • selected_version_task_exe_id - the task execution id of the selected data snapshot (version) loading a selected data snapshot (version).

AI-based Generation Columns

  • selected_subset_task_exe_id - the task execution of the selected training model (data generator) for the AI-based generation.

    Environments Columns

    Source Environment

  • source_env_name

  • source_environment_id

Target Environment

  • environment_id

Task Execution Parameters

Scheduling Parameters

  • Scheduler - set based on the task's Scheduler:
    • If the Scheduler is not set (that is, the task is executed by request), the scheduler is populated by immediate.
    • If the Scheduler is set (a scheduled task), the scheduler is populated by a crontab based on the selected scheduling parameters of the task.
  • scheduling_end_date - the scheduler is populated by the End Date if set on scheduled tasks.

Extract Tasks - Retention Period

  • retention_period_type and retention_period_value.

Other Parameters

  • replace_sequences
  • sync_mode -
    • If the task does not override the sync mode, this column remains empty.
    • If the task overrides the sync mode, this parameter is populated by FORCE or OFF.

TASKS_LOGICAL_UNITS

This table holds the LUs list of each task. A separate record is created for each LU.

TASKS_EXE_PROCESS


This table holds a task's pre and post execution processes. A new record is created for each pre or post execution process.

TASK_GLOBALS

This table holds all variables that are overridden by the task. A separate record is created for each variable.

TDM_GENERATE_TASK_FIELD_MAPPINGS

This table is populated with data generation parameters for Generate tasks.

TASK_REF_TABLES

This table holds a list of the task's tables. A separate record is created for each table.

Previous