TDM Sequence Implementation Steps
The TDM task provides an option to replace the sequences (IDs) of all selected entities before loading them into the target environment. This is required in order to prevent key duplication when the testing environment is not empty (i.e., already contains entities). In addition, the task execution replaces an entity's sequences when generating clones to ensure uniqueness in the target environment.
Notes:
- Sequence replacement must be set up in advance.
- Sequence implementation is also required for rule-based data generation flows to maintain referential integrity of synthetic entity IDs.
Starting with TDM V9.3, the following two sequence handling methods are supported:
I. Sequence handling based on Catalog.
II. Sequence handling without Catalog.
How to Set the Sequence Method for Each LU
A new shared Global — TDM_USING_CATALOG_SEQUENCES — was introduced in TDM V9.3. It sets the default sequence handling behavior of TDM to either Catalog-based or without Catalog. This Global (set to either true or false) can be added to an LU for controlling the LU's behavior.
Example:
- A TDM project contains CRM, Billing and Ordering LUs.
- By default, the sequences are handled without Catalog, except for the Billing LU, in which they are Catalog-based.
- The TDM_USING_CATALOG_SEQUENCES Global must be set as follows:
- Shared Global — set to false.
- Billing LU — set to true.
Generating Sequence Flows and Actors
Both sequence handling methods require execution of the TDMLuInitBasedOnFabric flow to create:
Sequence flows and Actors — for each record in the TDMSeqList Actor, a pair consisting of a sequence flow and an Actor is created under the Shared Broadway flows.
Load flows —
Each load flow invokes the HandleMaskAndSeqFields flow, which applies masking and sequence replacements to every record prior to loading it into the target environment. The TDM_USING_CATALOG_SEQUENCES Global specifies which of the two sequence handling methods is used.
For tables with records in the TDMSeqSrc2TrgMapping Actor, a new sequence flow, [table name]_sequences, is created under the Broadway/SequencesFlows directory of each LU. This flow invokes the sequence Actors for the table's sequence fields. Example:
Rule-based data generation flows — data generation flows with sequence generation added for ensuring consistency and referential integrity.
Note that the input parameter OVERRIDING_EXISTING_FLOWS of the TDMLuInitBasedOnFabric flow must be set to true as a way to properly generate all sequence flows and Actors.
Data Consistency of Generated Sequences
A sequence can be shared across tables and multiple LUs. For example, the subscriber_id sequence is shared between the CRM and Billing LUs. To maintain consistenct and referential integrity, the same sequence name must be used across all shared LUs, regardless of the applied sequence handling method.
The two sequence handling methods — Catalog-based or without Catalog — require creation of the k2masking schema, which is done by the TDM deploy flow. Alternatively, creating the k2masking schema can be done by running the masking-create-cache-table.flow from the Broadway Examples (found in the Broadway Flow window, Main Menu > Actions > Examples… and select this flow). Before deploying the TDM LU, verify that the SEQ_CACHE_INTREFACE shared Global is set with the correct interface name.
Click here for more information about the TDM installation and the k2masking schema creation.
Setting the Sequence Report Global
A new Global — TDM_SEQ_REPORT — has been added to the Shared Globals in TDM V8.1. When set to true (the default value), the task execution populates the TDM_SEQ_MAPPING table and adds the Replace Sequence Summary Report to the task execution report.
For better performance, set the TDM_SEQ_REPORT Global to false to prevent populating the TDM_SEQ_MAPPING table and generating the Replace Sequence Summary Report. Note that the Replace Sequence Summary Report would not be available for tasks that are executed with the TDM_SEQ_REPORT Global set to false.
TDM Sequence Implementation Steps
The TDM task provides an option to replace the sequences (IDs) of all selected entities before loading them into the target environment. This is required in order to prevent key duplication when the testing environment is not empty (i.e., already contains entities). In addition, the task execution replaces an entity's sequences when generating clones to ensure uniqueness in the target environment.
Notes:
- Sequence replacement must be set up in advance.
- Sequence implementation is also required for rule-based data generation flows to maintain referential integrity of synthetic entity IDs.
Starting with TDM V9.3, the following two sequence handling methods are supported:
I. Sequence handling based on Catalog.
II. Sequence handling without Catalog.
How to Set the Sequence Method for Each LU
A new shared Global — TDM_USING_CATALOG_SEQUENCES — was introduced in TDM V9.3. It sets the default sequence handling behavior of TDM to either Catalog-based or without Catalog. This Global (set to either true or false) can be added to an LU for controlling the LU's behavior.
Example:
- A TDM project contains CRM, Billing and Ordering LUs.
- By default, the sequences are handled without Catalog, except for the Billing LU, in which they are Catalog-based.
- The TDM_USING_CATALOG_SEQUENCES Global must be set as follows:
- Shared Global — set to false.
- Billing LU — set to true.
Generating Sequence Flows and Actors
Both sequence handling methods require execution of the TDMLuInitBasedOnFabric flow to create:
Sequence flows and Actors — for each record in the TDMSeqList Actor, a pair consisting of a sequence flow and an Actor is created under the Shared Broadway flows.
Load flows —
Each load flow invokes the HandleMaskAndSeqFields flow, which applies masking and sequence replacements to every record prior to loading it into the target environment. The TDM_USING_CATALOG_SEQUENCES Global specifies which of the two sequence handling methods is used.
For tables with records in the TDMSeqSrc2TrgMapping Actor, a new sequence flow, [table name]_sequences, is created under the Broadway/SequencesFlows directory of each LU. This flow invokes the sequence Actors for the table's sequence fields. Example:
Rule-based data generation flows — data generation flows with sequence generation added for ensuring consistency and referential integrity.
Note that the input parameter OVERRIDING_EXISTING_FLOWS of the TDMLuInitBasedOnFabric flow must be set to true as a way to properly generate all sequence flows and Actors.
Data Consistency of Generated Sequences
A sequence can be shared across tables and multiple LUs. For example, the subscriber_id sequence is shared between the CRM and Billing LUs. To maintain consistenct and referential integrity, the same sequence name must be used across all shared LUs, regardless of the applied sequence handling method.
The two sequence handling methods — Catalog-based or without Catalog — require creation of the k2masking schema, which is done by the TDM deploy flow. Alternatively, creating the k2masking schema can be done by running the masking-create-cache-table.flow from the Broadway Examples (found in the Broadway Flow window, Main Menu > Actions > Examples… and select this flow). Before deploying the TDM LU, verify that the SEQ_CACHE_INTREFACE shared Global is set with the correct interface name.
Click here for more information about the TDM installation and the k2masking schema creation.
Setting the Sequence Report Global
A new Global — TDM_SEQ_REPORT — has been added to the Shared Globals in TDM V8.1. When set to true (the default value), the task execution populates the TDM_SEQ_MAPPING table and adds the Replace Sequence Summary Report to the task execution report.
For better performance, set the TDM_SEQ_REPORT Global to false to prevent populating the TDM_SEQ_MAPPING table and generating the Replace Sequence Summary Report. Note that the Replace Sequence Summary Report would not be available for tasks that are executed with the TDM_SEQ_REPORT Global set to false.
