Catalog Artifacts
Overview
Catalog artifacts are extracts of Catalog metadata, generated upon request and uploaded to Fabric memory. Additionally, the artifacts are saved as CSV files within the Project tree.
There are two types of Catalog artifacts: fields artifacts and relations artifacts.
Artifacts are used in the masking and synthetic data generation mechanisms, as explained here.
Building Artifacts
To create Catalog artifacts, naviaget to Actions > Build artifacts in the Menu bar.
- You can build artifacts for a single data platform (either selected or expanded) or for the entire Catalog.
- Once initiated, a progress popup appears in the bottom-right corner of the screen. Details about the progress bar can be found further in this article.
- Note that building artifacts for a selected data platform and the progress bar feature are available starting from Fabric V8.4.
Artifacts are uploaded to Fabric memory as catalog_field_info and catalog_relations_info MTables.
The following CSV files are created and saved under Implementation/SharedObjects/Interfaces/Discovery/MTable in the Project tree:
- Field artifact files, named
catalog_field_info___<dataPlatform>_<schema>.csv, (prefixed with 3 underscores before the data platform name). These fields include all fields with their properties. - Relation artifact files, named
catalog_relations_info___<dataPlatform>_<schema>.csv, (prefixed with 3 underscores before the data platform name). These files include refersTo relations with their properties (parent info, child info, origin). Note that the relation artifact files are generated from Fabric V8.3.1.
The below image is an example of a catalog_field_info___DB2_sakila.csv file:
The below image is an example of catalog_relations_info___CRM_DB_main.csv:
The heading of the last column indicates the version number (V14 in the above examples), though the column itself remains empty.
Artifacts can be created for any Catalog version. Each new build overrides the existing artifact in the Project tree.
Splitting and Combining Artifacts
Catalog artifacts are split into separate files for each data platform and schema of a given Catalog version. While saved separately in the Project tree, their content is combined into a single MTable in Fabric's memory.
Starting from Fabric V8.3, all artifacts are split by default into separate files. The splitting or combining of artifacts is controlled by the SPLIT_CATALOG_ARTIFACTS parameter in the config.ini file.
This feature allows you to combine separate artifacts created in different projects or spaces into a single artifact. Hence, the artifact files can be copied from one project to another, and upon deployment, they will be combined into one MTable.
Note that if either the catalog_field_info.csv or catalog_relations_info.csv combined file exists in the Project tree, it should be manually deleted.
Build Artifacts Progress Bar
The progress popup provides the current status and estimated remaining time to completion.
The progress bar can be minimized or closed.
To reopen the progress bar while artifact building is still underway, select Actions > Build artifacts > Show progress.
Completed executions are displayed by selecting Actions > Build artifacts > Show completed. This view displays only the last completed execution for each data platform or for the entire Catalog.
Catalog Artifacts
Overview
Catalog artifacts are extracts of Catalog metadata, generated upon request and uploaded to Fabric memory. Additionally, the artifacts are saved as CSV files within the Project tree.
There are two types of Catalog artifacts: fields artifacts and relations artifacts.
Artifacts are used in the masking and synthetic data generation mechanisms, as explained here.
Building Artifacts
To create Catalog artifacts, naviaget to Actions > Build artifacts in the Menu bar.
- You can build artifacts for a single data platform (either selected or expanded) or for the entire Catalog.
- Once initiated, a progress popup appears in the bottom-right corner of the screen. Details about the progress bar can be found further in this article.
- Note that building artifacts for a selected data platform and the progress bar feature are available starting from Fabric V8.4.
Artifacts are uploaded to Fabric memory as catalog_field_info and catalog_relations_info MTables.
The following CSV files are created and saved under Implementation/SharedObjects/Interfaces/Discovery/MTable in the Project tree:
- Field artifact files, named
catalog_field_info___<dataPlatform>_<schema>.csv, (prefixed with 3 underscores before the data platform name). These fields include all fields with their properties. - Relation artifact files, named
catalog_relations_info___<dataPlatform>_<schema>.csv, (prefixed with 3 underscores before the data platform name). These files include refersTo relations with their properties (parent info, child info, origin). Note that the relation artifact files are generated from Fabric V8.3.1.
The below image is an example of a catalog_field_info___DB2_sakila.csv file:
The below image is an example of catalog_relations_info___CRM_DB_main.csv:
The heading of the last column indicates the version number (V14 in the above examples), though the column itself remains empty.
Artifacts can be created for any Catalog version. Each new build overrides the existing artifact in the Project tree.
Splitting and Combining Artifacts
Catalog artifacts are split into separate files for each data platform and schema of a given Catalog version. While saved separately in the Project tree, their content is combined into a single MTable in Fabric's memory.
Starting from Fabric V8.3, all artifacts are split by default into separate files. The splitting or combining of artifacts is controlled by the SPLIT_CATALOG_ARTIFACTS parameter in the config.ini file.
This feature allows you to combine separate artifacts created in different projects or spaces into a single artifact. Hence, the artifact files can be copied from one project to another, and upon deployment, they will be combined into one MTable.
Note that if either the catalog_field_info.csv or catalog_relations_info.csv combined file exists in the Project tree, it should be manually deleted.
Build Artifacts Progress Bar
The progress popup provides the current status and estimated remaining time to completion.
The progress bar can be minimized or closed.
To reopen the progress bar while artifact building is still underway, select Actions > Build artifacts > Show progress.
Completed executions are displayed by selecting Actions > Build artifacts > Show completed. This view displays only the last completed execution for each data platform or for the entire Catalog.
