Task - TDM DB Tables

TDM settings are saved in the TDM PostgreSQL DB.

TDM tasks update the following TDM DB tables:

TASKS

This table holds all TDM tasks defined in the TDM Portal.

  • A new record is created for each task.

  • Each task record gets a unique task_id sequence, which is the table's PK.

    Task General Information

    • task_title - the task name. To prevent creating several active tasks with the same name, the task_title column has a unique index when the status is Active.
    • task_type - Extract , Load or Reserve.
    • be_id - the task's BE. The be_id can be linked to the product_logical_units TDM DB table.
    • number_of_entities - populated by the number of entities in the task.
    • load_entities - populated with true for Load tasks. Otherwise, populated with false.
    • delete_before_load - populated with true for Delete or Load and Delete tasks. Otherwise populated with false.
    • task_created_by, and task_last_updated_by - populated by the name of the user who creates the task.
    • task_creation_date and task_last_updated_date - populated by the task's creation datetime.

    Reservation Information

    • reserve_ind - indicates if the task reserved entities on the environment.
    • reserve retention period fields - define the reservation period on the task's entities.

    Map the TDM Portal Task Actions (types) to Tasks TDM Table

Task Actions Combination

TDM Tasks Table Population

Extract

  • task_type = Extract
  • load_entity = false
  • delete_before_load = false
  • reserve_ind = false

Extract + Load

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = false

Extract + Load + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = true

Extract + Load + Delete

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = false

Extract + Load + Delete + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = true

Generate

  • task_type = Generate
  • load_entity = false
  • delete_before_load = false
  • reserve_ind = false
  • selection_method = Generate

Generate + Load

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = false
  • selection_method = Generate

Generate + Load + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = true
  • selection_method = Generate

Load

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = false

Load + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = true

Load + Delete

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = false

Load + Delete + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = true

Delete

  • task_type = Load
  • load_entity = false
  • delete_before_load = true
  • reserve_ind = false

Reserve

  • task_type = Reserve
  • load_entity = false
  • delete_before_load = true
  • reserve_ind = false

### Task Status

Task selection method

selection_method

Selection_param_value

Entity List

L

Populated by the list of entities separated by a comma.

Examples:

  • 1,2,3,4
  • 66

Parameters, with or without a random selection

P, PR

Populated by the SQL where statement, generated by the selected parameters.

Example:

  • (( 'New York' = ANY("Customer.CITY") ))

Entity Clone

CLONE

Populated with the cloned entity ID.

Example:

  • 7889

Custom Logic

C Populated with the name of the selected Broadway flow that runs and gets the entity list for the task execution.

This column is used by the TDM task execution process to create the entities list of the root LUs for each task.

### Data Versioning Parameters

  • version_ind - populated by true in a Data Versioning task.

  • selected_version_task_name, selected_version_datetime, and selected_version_task_exe_id - the selected entities when creating a load Data Versioning task to reload a selected version of entities into the target environment.

  • selected_ref_version_task_name, selected_ref_version_datetime, and selected_ref_version_task_exe_id - the selected Reference's version when creating a Data Versioning task to copy a selected version of Reference tables into the target environment.

    Environments Columns

    Source Environment

    • source_env_name

    • source_environment_id

    Target Environment

    • environment_id

Task Execution Parameters

Scheduling Parameters

  • Scheduler - set based on the task's Execution Timing:
    • Execution by Request, populated by immediate.
    • Scheduled Execution, populated by a crontab based on the selected scheduling parameters of the task.
  • scheduling_end_date - populated by the End Date if set on scheduled tasks.

Extract Tasks - Retention Period

  • retention_period_type and retention_period_value.

Load Tasks - Request Parameters

Other Parameters

  • replace_sequences
  • sync_mode -
    • If the task does not override the sync mode, this column remains empty.
    • If the task overrides the sync mode, populated by FORCE or OFF.

TASKS_LOGICAL_UNITS

This table holds the LUs list of each task. A separate record is created for each LU.

TASKS_POST_EXE_PROCESS


This table holds a task's post execution processes. A new record is created for each post execution process.

This table holds the following columns:

  • task_id - a unique identifier of the task, which links to the Tasks TDM DB table.
  • process_id - a unique identifier of the process, which links tdm_be_post_exe_process TDM DB table.
  • execution_order - the execution_order of the post execution process as defined in the tdm_be_post_exe_process TDM DB table.

TASK_GLOBALS

This table holds all variables that are overridden by the task. A separate record is created for each variable.

TDM_GENERATE_TASK_FIELD_MAPPINGS

This table is populated with data generation parameters for Generate tasks.

TASK_REF_TABLES

This table holds a list of the task's Reference tables. A separate record is created for each Reference table.

Previous

Task - TDM DB Tables

TDM settings are saved in the TDM PostgreSQL DB.

TDM tasks update the following TDM DB tables:

TASKS

This table holds all TDM tasks defined in the TDM Portal.

  • A new record is created for each task.

  • Each task record gets a unique task_id sequence, which is the table's PK.

    Task General Information

    • task_title - the task name. To prevent creating several active tasks with the same name, the task_title column has a unique index when the status is Active.
    • task_type - Extract , Load or Reserve.
    • be_id - the task's BE. The be_id can be linked to the product_logical_units TDM DB table.
    • number_of_entities - populated by the number of entities in the task.
    • load_entities - populated with true for Load tasks. Otherwise, populated with false.
    • delete_before_load - populated with true for Delete or Load and Delete tasks. Otherwise populated with false.
    • task_created_by, and task_last_updated_by - populated by the name of the user who creates the task.
    • task_creation_date and task_last_updated_date - populated by the task's creation datetime.

    Reservation Information

    • reserve_ind - indicates if the task reserved entities on the environment.
    • reserve retention period fields - define the reservation period on the task's entities.

    Map the TDM Portal Task Actions (types) to Tasks TDM Table

Task Actions Combination

TDM Tasks Table Population

Extract

  • task_type = Extract
  • load_entity = false
  • delete_before_load = false
  • reserve_ind = false

Extract + Load

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = false

Extract + Load + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = true

Extract + Load + Delete

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = false

Extract + Load + Delete + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = true

Generate

  • task_type = Generate
  • load_entity = false
  • delete_before_load = false
  • reserve_ind = false
  • selection_method = Generate

Generate + Load

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = false
  • selection_method = Generate

Generate + Load + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = true
  • selection_method = Generate

Load

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = false

Load + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = false
  • reserve_ind = true

Load + Delete

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = false

Load + Delete + Reserve

  • task_type = Load
  • load_entity = true
  • delete_before_load = true
  • reserve_ind = true

Delete

  • task_type = Load
  • load_entity = false
  • delete_before_load = true
  • reserve_ind = false

Reserve

  • task_type = Reserve
  • load_entity = false
  • delete_before_load = true
  • reserve_ind = false

### Task Status

Task selection method

selection_method

Selection_param_value

Entity List

L

Populated by the list of entities separated by a comma.

Examples:

  • 1,2,3,4
  • 66

Parameters, with or without a random selection

P, PR

Populated by the SQL where statement, generated by the selected parameters.

Example:

  • (( 'New York' = ANY("Customer.CITY") ))

Entity Clone

CLONE

Populated with the cloned entity ID.

Example:

  • 7889

Custom Logic

C Populated with the name of the selected Broadway flow that runs and gets the entity list for the task execution.

This column is used by the TDM task execution process to create the entities list of the root LUs for each task.

### Data Versioning Parameters

  • version_ind - populated by true in a Data Versioning task.

  • selected_version_task_name, selected_version_datetime, and selected_version_task_exe_id - the selected entities when creating a load Data Versioning task to reload a selected version of entities into the target environment.

  • selected_ref_version_task_name, selected_ref_version_datetime, and selected_ref_version_task_exe_id - the selected Reference's version when creating a Data Versioning task to copy a selected version of Reference tables into the target environment.

    Environments Columns

    Source Environment

    • source_env_name

    • source_environment_id

    Target Environment

    • environment_id

Task Execution Parameters

Scheduling Parameters

  • Scheduler - set based on the task's Execution Timing:
    • Execution by Request, populated by immediate.
    • Scheduled Execution, populated by a crontab based on the selected scheduling parameters of the task.
  • scheduling_end_date - populated by the End Date if set on scheduled tasks.

Extract Tasks - Retention Period

  • retention_period_type and retention_period_value.

Load Tasks - Request Parameters

Other Parameters

  • replace_sequences
  • sync_mode -
    • If the task does not override the sync mode, this column remains empty.
    • If the task overrides the sync mode, populated by FORCE or OFF.

TASKS_LOGICAL_UNITS

This table holds the LUs list of each task. A separate record is created for each LU.

TASKS_POST_EXE_PROCESS


This table holds a task's post execution processes. A new record is created for each post execution process.

This table holds the following columns:

  • task_id - a unique identifier of the task, which links to the Tasks TDM DB table.
  • process_id - a unique identifier of the process, which links tdm_be_post_exe_process TDM DB table.
  • execution_order - the execution_order of the post execution process as defined in the tdm_be_post_exe_process TDM DB table.

TASK_GLOBALS

This table holds all variables that are overridden by the task. A separate record is created for each variable.

TDM_GENERATE_TASK_FIELD_MAPPINGS

This table is populated with data generation parameters for Generate tasks.

TASK_REF_TABLES

This table holds a list of the task's Reference tables. A separate record is created for each Reference table.

Previous