Data profiling attributes can be used in the SELECT list, WHERE clause and ORDER BY clause of a MQL Query.
The following Data Profiling attributes are supported on file fields or table/view columns.
|
Data Profiling attribute name |
Description |
Value examples |
Value data type |
Supported operators in WHERE |
|
"Data Profiling".Blank |
Number of blank value rows |
20 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Distinct" |
Number of distinct rows |
1001 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Duplicate |
Number of duplicate rows |
5 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Valid |
Number of valid rows |
5 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Empty (Deprecated) |
Number of empty rows: null in database or empty in files. This attribute is split into 3 attributes: "Data Profiling".Null, "Data Profiling".Blank and "Data Profiling".Zero. |
5 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Invalid |
Number of invalid rows |
5 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Min |
Lowest value |
1.0 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Max |
Highest value |
3.123456E30 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Mean |
Mean value |
3.123456E30 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Null |
Number of null value rows |
10 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Variance |
Variance value |
3.123456E30 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling".Median |
Median value |
3.123456E30 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Lower Quantile" |
Lower quantile |
3.123456E30 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Upper Quantile" |
Upper quantile |
3.123456E30 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Avg Length" |
Average length of strings |
6.5 |
Double |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Min Length" |
Minimum length of strings |
3 |
Integer |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Max Length" |
Maximum length of strings |
20 |
Integer |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Inferred Data Types" |
List of data types detected by the profiler |
["integer", "string"] |
Array of String |
=, !=, = ANY (...), != ANY (...), = ALL (...), != ALL (...), [NOT] EXISTS |
|
"Data Profiling".Zero |
Number of zero value rows |
40 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
The following Data Profiling attributes are supported on files, tables and views.
|
Data Profiling attribute name |
Description |
Value examples |
Value data type |
Supported operators in WHERE |
|
"Data Profiling"."Profiled Date" |
Last profiled date |
2022-09-24 13:25:17 |
Date |
=, BEFORE, AFTER, BETWEEN ... AND ... , [NOT] EXISTS |
|
"Data Profiling"."Processed Records" |
Processed row count |
10 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
|
"Data Profiling"."Total Records" |
Total row count |
100 |
Long |
=, !=, = ANY (...), != ANY (...), [NOT] EXISTS, <, <=, >, >= and BETWEEN |
When somebody uses a data profiling attribute in a filter, MIMM removes all objects from the results which are marked as hide data for that user. The hide data is controlled by multiple factors: whether the user has the Data Viewer capability object role assignment, whether a harvested object has a proposed or assigned data class resulting in a Sensitivity Label that has the HIDE DATA flag, or whether the object has a Sensitivity Label that has the HIDE DATA flag.