This article describes the source data metrics plugins of the Catalog solution; these metrics are calculated based on the data snapshot.
The plugins are:
The purpose of this plugin is to calculate the percentage of NULL values per column, based on the data snapshot. This percentage is calculated on each column of non-empty tables. The default size of the data snapshot is configured in the plugins.discovery file as explained here.
As a result, when the calculated value is above the threshold, the Null Percentage property is added to the field's properties.
For example, when 30% of the values in a certain field are null, the Null Percentage property will be added to this field with the value = 0.3. However, if 20% or less of the values in this field are null, then this property would not be added.
This plugin exists until the Fabric V8.1. In V8.2 it has been combined with the Data Quality Metrics.
This plugin scans the data of the data sample in order to calculate the following 4 parameters:
This article describes the source data metrics plugins of the Catalog solution; these metrics are calculated based on the data snapshot.
The plugins are:
The purpose of this plugin is to calculate the percentage of NULL values per column, based on the data snapshot. This percentage is calculated on each column of non-empty tables. The default size of the data snapshot is configured in the plugins.discovery file as explained here.
As a result, when the calculated value is above the threshold, the Null Percentage property is added to the field's properties.
For example, when 30% of the values in a certain field are null, the Null Percentage property will be added to this field with the value = 0.3. However, if 20% or less of the values in this field are null, then this property would not be added.
This plugin exists until the Fabric V8.1. In V8.2 it has been combined with the Data Quality Metrics.
This plugin scans the data of the data sample in order to calculate the following 4 parameters: