Logical Unit / Data Product Overview
What Is a Logical Unit?
A Logical Unit (LU or Logical Unit type - LUT), also known as a Data Product, is a blueprint data asset, engineered to deliver a trusted dataset for a specific business purpose. It holds a set of definitions and instructions that are used for integrating data from source systems, processes the data and stores it. The LU is the prototype from which LU Instances (LUIs) are created. An LU is defined and configured in the Fabric Studio. These definitions are comprised of 3 main objects:
LU Table: The definition of an LU table upon its columns, PKs, indexes and triggers.
-
- Feed data into LU tables from a variety of data sources and keep it up to date.
- Ability to manipulate the fed data i.e. performance of enrichment, cleansing, masking, transforming, etc.
LU Schema: The relationship between the LU tables (similar to foreign keys). An LU schema has one LU table defined as its Root Table. The Root Table holds the LU’s unique key.
Let’s use an example of a Customer 360 implementation for Company ABC:
- LU / Data Product: Customer.
- Data sources: CRM, Ordering, Billing and Collection, Billing and Assets.
- LU tables: The tables that will hold the data you want to keep about a customer from the 4 data sources.
- LU Table Populations: The set of definitions that will be used to extract, transform, clean, aggregate, validate (etc.) the data from the 4 data sources into the LU tables.
- LU schema: The definition of the Root Table and the relationship between all LU tables.
What Is a Logical Unit Instance (LUI)?
A Logical Unit Instance is one instance of a Logical Unit Type – it is a single physical database, which holds the data of one single Digital Entity in the LUT structure definition. Using our example from above (Customer 360), assume that Company ABC has 35 million customers:
- LU/LUT = Customer.
- LUI = one single customer database.
Fabric will hold 35 million instances (LUIs) of the Customer LUT. That is, one physical database for each customer.
Things to Consider Before Designing an LU
Every Fabric project starts by defining its LUs. Analyze the business requirements and understand how the consuming application will use the data. Use this information to define the different Digital Entities to implement and build an LU for each Digital Entity.
General Recommendations for Designing an LU
Digital Entity data is often split between different data sources. In some cases, it is preferable to create one LU that contains all data sources. In other cases, it is more advantageous to split the LUs and create a separate LU for each data source.
In general, an LU should be based on the smallest number of data sources, as long as it represents a full 360 view of a Digital Entity.
For example, if you have a Digital Entity called Customer, but different Customer Types (e.g. consumer and business) have different data sources, the recommended approach will be to create an LU for each Digital Entity subtype (in our example, different Customer Types).
Below is a table of pros and cons of each alternative:
Note:
The file name ambiguity is not supported within the same Logical Unit. This is not restricted by the Fabric Studio on purpose, to allow the implementor continue his work and update the names later. For example, if 2 Java function files with identical names were exported from other projects or libraries, they can be saved in the project in the Fabric Studio.
However, at run-time there should be no ambiguity within the LU, otherwise the server will run the first file that it finds (no commitment to what is considered the first one).
Logical Unit / Data Product Overview
What Is a Logical Unit?
A Logical Unit (LU or Logical Unit type - LUT), also known as a Data Product, is a blueprint data asset, engineered to deliver a trusted dataset for a specific business purpose. It holds a set of definitions and instructions that are used for integrating data from source systems, processes the data and stores it. The LU is the prototype from which LU Instances (LUIs) are created. An LU is defined and configured in the Fabric Studio. These definitions are comprised of 3 main objects:
LU Table: The definition of an LU table upon its columns, PKs, indexes and triggers.
-
- Feed data into LU tables from a variety of data sources and keep it up to date.
- Ability to manipulate the fed data i.e. performance of enrichment, cleansing, masking, transforming, etc.
LU Schema: The relationship between the LU tables (similar to foreign keys). An LU schema has one LU table defined as its Root Table. The Root Table holds the LU’s unique key.
Let’s use an example of a Customer 360 implementation for Company ABC:
- LU / Data Product: Customer.
- Data sources: CRM, Ordering, Billing and Collection, Billing and Assets.
- LU tables: The tables that will hold the data you want to keep about a customer from the 4 data sources.
- LU Table Populations: The set of definitions that will be used to extract, transform, clean, aggregate, validate (etc.) the data from the 4 data sources into the LU tables.
- LU schema: The definition of the Root Table and the relationship between all LU tables.
What Is a Logical Unit Instance (LUI)?
A Logical Unit Instance is one instance of a Logical Unit Type – it is a single physical database, which holds the data of one single Digital Entity in the LUT structure definition. Using our example from above (Customer 360), assume that Company ABC has 35 million customers:
- LU/LUT = Customer.
- LUI = one single customer database.
Fabric will hold 35 million instances (LUIs) of the Customer LUT. That is, one physical database for each customer.
Things to Consider Before Designing an LU
Every Fabric project starts by defining its LUs. Analyze the business requirements and understand how the consuming application will use the data. Use this information to define the different Digital Entities to implement and build an LU for each Digital Entity.
General Recommendations for Designing an LU
Digital Entity data is often split between different data sources. In some cases, it is preferable to create one LU that contains all data sources. In other cases, it is more advantageous to split the LUs and create a separate LU for each data source.
In general, an LU should be based on the smallest number of data sources, as long as it represents a full 360 view of a Digital Entity.
For example, if you have a Digital Entity called Customer, but different Customer Types (e.g. consumer and business) have different data sources, the recommended approach will be to create an LU for each Digital Entity subtype (in our example, different Customer Types).
Below is a table of pros and cons of each alternative:
Note:
The file name ambiguity is not supported within the same Logical Unit. This is not restricted by the Fabric Studio on purpose, to allow the implementor continue his work and update the names later. For example, if 2 Java function files with identical names were exported from other projects or libraries, they can be saved in the project in the Fabric Studio.
However, at run-time there should be no ambiguity within the LU, otherwise the server will run the first file that it finds (no commitment to what is considered the first one).
