Catalog APIs

Fabric provides out-of-the-box Web Service APIs for querying a project's Catalog.

Any web service call passes authentication and authorization processes prior to being executed. For more information, read here.

All APIs are accessed over HTTPS, from the Fabric URL endpoint https://<Domain Name>:<PORT>

Start Crawler Job

GET /api/catalog/start-crawler-job

The API invokes the DISCOVERY_CRAWLER job for a given interface.

Example of an API call:

https://localhost:3213/api/catalog/start-crawler-job?dataPlatform=CRM_DB

Get Catalog

GET /api/catalog

The API retrieves a list of Catalog versions along with their creation timestamps.

Example of an API call:

https://localhost:3213/api/catalog

Get Details of Catalog Version

GET /api/catalog/{version}

The API retrieves a list of data platforms that belong to a given Catalog version. Each data platform is retrieved along with its contains relations to the respective schema nodes.

The API has two modes — view and compare — that are interchangeably invoked based on the version input parameter, as explained below:

Examples of an API call in view mode:

https://localhost:3213/api/catalog/latest

https://localhost:3213/api/catalog/latest

Examples of an API call in compare mode:

https://localhost:3213/api/catalog/2...5

https://localhost:3213/api/catalog/1...latest

Get Details of Catalog Data Platform

GET /api/catalog/{version}/{dataPlatform}

The API retrieves a list of schemas that belong to the specified Catalog version and data platform. Each schema is retrieved along with its contains relations to the respective dataset nodes.

The API has two modes — view and compare — that are interchangeably invoked based on the version input parameter, as explained below:

Examples of an API call in view mode:

https://localhost:3213/api/catalog/latest/CRM_DB

https://localhost:3213/api/catalog/latest/CRM_DB?shortMode=true

https://localhost:3213/api/catalog/1/CRM_DB

Examples of an API call in compare mode:

https://localhost:3213/api/catalog/1...5/CRM_DB

https://localhost:3213/api/catalog/1...latest/CRM_DB

Get Details of Catalog Schema

GET /api/catalog/{version}/{dataPlatform}/{schema}

The API retrieves all elements that belong to the specified Catalog version, data platform and schema. The elements include datasets, fields and their properties, and the refersTo relations between dataset nodes.

The API has two modes — view and compare — that are interchangeably invoked based on the version input parameter, as explained below:

Examples of an API call in view mode:

https://localhost:3213/api/catalog/latest/CRM_DB/main

https://localhost:3213/api/catalog/1/CRM_DB/main

Examples of an API call in compare mode:

https://localhost:3213/api/catalog/1...5/CRM_DB/main

https://localhost:3213/api/catalog/1...latest/CRM_DB/main

GET /api/catalog/{dataPlatform}/{schema}/datasets

The API retrieves a list of datasets that belong to the latest version of the specified data platform and schema. This API is available starting from Fabric V8.3.

GET /api/catalog/{dataPlatform}/{schema}/{dataset}/fields

The API retrieves a list of fields that belong to the latest version of the specified data platform, schema and dataset. This API is available starting from Fabric V8.3.

Building Catalog Artifacts

GET /api/catalog/{version}/build-catalog-artifacts

The API builds the Catalog artifacts based on a given version. The artifacts include details of all Catalog fields and their properties, such as Classification and PII. The artifact is created in a CSV format, saved in the Implementation/SharedObjects/Interfaces/Discovery/MTable folder of the Project tree, and uploaded to the Fabric memory as an MTable.

Starting from Fabric V8.3, the relations artifact can also be extracted by the API. It can be done when setting refersTo=true in the API input. The relations artifact is created in a CSV format, saved in the Implementation/SharedObjects/Interfaces/Discovery/MTable folder of the Project tree, and uploaded to the Fabric memory as an MTable.

Refer to the Catalog Artifacts article for more details about the structure and naming format of relations artifacts.

Example of an API call:

https://localhost:3213/api/catalog/4/build-catalog-artifacts

https://localhost:3213/api/catalog/latest/build-catalog-artifacts?refersTo=true

Catalog Search

POST /api/catalog/{version}/search-graph

The API retrieves all elements that belong to the specified Catalog version, based on the search criteria defined in the request body.

The syntax of the request body definition is as follows:

• input is an array of keywords

  • Keyword is a string included in the node (or relation) name

• type specifies which object types will be searched

  • The valid values are: dataPlatform, schema, dataset, field, relation
  • Send an empty array in case of no limitation on object type

• advanced includes a list of additional search parameters, such as:

  • pii is a PII property with a value of either true or false
  • classification is a Classification property with one of its valid values
  • score represents the maximum score among the searched object types (nodes or relations)

At least one of the search parameters must be provided in the request body.

Example of an API call:

https://localhost:3213/api/catalog/4/search-graph

Examples of the request body:

Example 1: When searching for Data Platform and Schema nodes, whose names include the keyword customer, the request body is as follows:

{
    "input": [
        "customer"
    ],
    "type": [
        "dataPlatform",
        "schema"
    ],
    "advanced": {
    }
}

Example 2: When searching for node types with PII = true and Classification = EMAIL properties, the request body is as follows:

{
    "input": [],
    "type": [],
    "advanced": {
        "pii": "true",
        "classification": "EMAIL"
    }
}