tDataUnmasking properties for Apache Spark Batch
These properties are used to configure tDataUnmasking running
in the Spark Batch Job framework.
The Spark Batch
tDataUnmasking component belongs to the Data Quality family.
Basic settings
Schema and |
A schema is a row description. It defines the number of fields Click Sync Click Edit
The output schema of this component |
 |
Built-In: You create and store the schema locally for this component |
 |
Repository: You have already created the schema and stored it in the |
Modifications |
Define in the table what fields to unmask and how to unmask them: Input Column: Select the column from the input These modifications are based on the function you select in the Function Category: select a category of unmasking functions Function: Select the function that will unmask The functions you can select from the Function list depend on the data Method: From this list, select the The FF1 with AES method is based on the Advanced Java 8u161 is the minimum required version to use the FF1 with To unmask data, the FF1 with AES and When using the Replace all, Replace characters between two positions, Select the alphabet used to mask data with the
Extra Parameter: This field is used Keep |
Advanced settings
Password for FF1 methods |
To unmask data, the FF1 with AES and FF1 with SHA-2 methods require the |
Use tweaks |
If tweaks have been generated while encrypting |
Column containing tweaks |
Available when the Use tweaks check box is selected. Select the column |
Output the original row |
Select this check box to output masked data rows in addition to the original data. Having both data rows can be useful in debug or test processes. |
Should null input return null |
This check box is selected by default. When selected, the component |
Should empty input return empty |
When this check box is selected, the component returns the input values |
Send invalid data to “Invalid” output flow |
This check box is selected by default.
The data are considered invalid when:
|
tStat |
Select this check box to gather the Job processing metadata at the Job |
Usage
Usage rule |
This component is used as an intermediate step. This component, along with the Spark Batch component Palette it belongs to, Note that in this documentation, unless otherwise explicitly stated, a |
Spark Connection |
In the Spark
Configuration tab in the Run view, define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |