August 15, 2023

tBlockedFuzzyJoin (deprecated) – Docs for ESB 6.x

  2. Talend for ESB Docs
  3. Docs 6.x
  4. tBlockedFuzzyJoin (deprecated) – Docs for ESB 6.x

tBlockedFuzzyJoin (deprecated)

Helps ensuring the data quality of any
source data against a reference data
source.

tBlockedFuzzyJoin joins two tables by doing a fuzzy match on several columns. It
compares columns from the main flow with reference columns from the lookup flow and outputs
the match data, the possible match data and the rejected data.

tBlockedFuzzyJoin Standard properties

These properties are used to configure tBlockedFuzzyJoin running in the Standard Job framework.

The Standard
tBlockedFuzzyJoin component belongs to the Data Quality family.

This component is available in the Palette of the Studio only if you have subscribed to one of the Talend Platform products.

Basic
settings

Schema and
Edit Schema

A schema is a row description, it defines
the number of fields to be processed and passed on
to the next component. The schema is either
Built-in or
stored remotely in the Repository.

 

Built-in:
You create and store the schema locally for this
component only. Related topic: see
Talend Studio User Guide .

 

Repository:
You have already created and stored the schema in
the Repository. You can reuse it in other projects
and Job designs. Related topic: see
Talend Studio User Guide .

Replace output column with lookup
column if matches or possible
matches

Select this check box to replace the output
column with the lookup column in case of match or
possible match values.

Input key
attribute

Select the column(s) from the main flow that
needs to be checked against the reference (lookup)
key column.

Lookup key
attribute

Select the lookup key columns that you will
use as a reference against which to compare the
columns from the input flow.

Matching type

Select the relevant matching algorithm from
the list:

Exact Match:
matches each processed entry to all possible
reference entries that have exactly the same
value.

Levenshtein:
Based on the edit distance theory. It calculates
the number of insertion, deletion or substitution
required for an entry to match the reference
entry.

Metaphone: Based
on a phonetic algorithm for indexing entries by
their pronunciation. It first loads the phonetics
of all entries of the lookup reference and checks
all entries of the main flow against the entries
of the reference flow.

Double Metaphone:
a new version of the Metaphone phonetic algorithm,
that produces more accurate results than the
original algorithm. It can return both a primary
and a secondary code for a string. This accounts
for some ambiguous cases as well as for multiple
variants of surnames with common ancestry.

Case sensitive

Select this check box to consider the letter
case.

Min. distance

Only for Levenshtein. Set the minimum number of
changes allowed to match the reference. If set to
0, only perfect matches(Exact Match) are returned.

Note:

You can create and store context variables
for the minimum and maximum distances and then
have your Job to loop on these values in order to
start from a low max number to match rows and go
up to higher max number to match more possible
rows. You can press Ctrl+Space to access the variable list
and select the new context variables. For more
information about context variables, see
Talend Studio User Guide.

Max. distance

Only for Levenshtein. Set the maximum number of
changes allowed to match the reference.

Advanced
settings

tStat
Catcher
Statistics

Select this check box to collect log data at
the component level.

Global
Variables

Global
Variables

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule

This component is deprecated, use the tRecordMatching
component instead.

This component is not startable (green
background) and it requires two input components
and one or more output components.

Scenario: Doing a fuzzy match on two columns and outputting the match,
possible match and non match values (deprecated)

This scenario applies only to a subscription-based Talend Platform solution or Talend Data Fabric.

This scenario describes a six-component Job that aims at:

  • matching each processed group number in the grp column against the entries that have exactly the same values in the
    reference input file,

  • checking the edit distance between the entries in the firstname column of an input file against those of the
    reference input file.

The outputs of these two matching types are written in three output files:
the first for match values, the second for possible match values and the third for the
values for which there are no matches in the lookup file.

In this scenario, we have already stored the main and reference input
schemas in the Repository. For more information about storing schema metadata in the
Repository, see
Talend Studio User Guide.

The main input file contains four columns: grp, gender, firstname and count. The data in this input file have problems such as duplication, first names
spelled differently or wrongly, different information for the same customer.

Use_Case_tMergeRow.png

Setting up the Job

  1. In the Repository tree view,
    expand Metadata and the
    FileExcel node
    where you have stored the main input schemas and then drop
    it onto the design workspace.

    A tFileInputExcel component holding your schema
    displays on the workspace.

  2. Do the same with the reference input schema to display the
    reference tFileInputExcel
    on the workspace.
  3. Drop a tBlockedFuzzyJoin and
    tLogRow (x3) from
    the Palette onto the design
    workspace.

    Use_Case_tBlockedFuzzyJoin.png

  4. Connect the main and reference input Excel files to tBlockedFuzzyJoin using
    Main links. The
    link between the reference input Excel file and tBlockedFuzzyJoin displays as
    a Lookup link on the design
    workspace.
  5. Connect tBlockedFuzzyJoin to
    the three tLogrow
    components using the Matches, Possible
    Matches     and Non
    Matches     links.

Configuring the input components

Double-click the main and reference input Excel file components to display
their Basic settings views.

The capture below shows the properties of the main input
file.

Use_Case_tBlockedFuzzyJoin1.png

The capture below shows the properties of the reference input
file.

Use_Case_tBlockedFuzzyJoin2.png

The property fields for both tFileInputExcel components are automatically filled in. If you
do not define your input schemas locally in the Repository, fill in the
details manually after selecting Built-in in the Schema
and Property Type fields.

Configuring the tBlockedFuzzyJoin component

  1. Double-click tBlockedFuzzyJoin to display its
    Basic settings view
    and define its properties.

    Use_Case_tBlockedFuzzyJoin3.png

  2. Click the Edit schema button
    to open a dialog box. Here you can define the data you want
    to pass to the output components.

    In this example we want to pass the four input columns to the output
    components in addition to the new column ref_firstname.

    Use_Case_tBlockedFuzzyJoin4.png

  3. Click OK to close the dialog
    box and proceed to the next step.
  4. In the Key definition area of
    the Basic settings view of
    tBlockedFuzzyJoin,
    click the plus button to add two columns to the list.
  5. Select the input columns and the output columns you want to do
    the fuzzy matching on from the Input
    key attribute     and Lookup key attribute lists respectively,
    grp and
    firstname in this
    example.
  6. Click in the first cell of the Matching
    type     column and select from the list the
    method to be used to check the incoming data against the
    reference data, Exact match in this
    example. There is no minimum nor maximum distance to
    set.
  7. Set the matching type for the second column,
    Levenshtein in this
    example.
  8. Then set the minimum and maximum distances. In this method,
    the distance is the number of character changes (insertion,
    deletion or substitution) that needs to be carried out in
    order for the entry to fully match the reference. In this
    example, we want the min. distance to be 0 and the max.
    distance to be 2. This will output all entries in the
    firstname column that exactly
    match or that have maximum two character changes.

Executing the Job

  1. Double-click the first tLogRow
    component to display its Basic settings view and define its
    properties.

    Use_Case_tBlockedFuzzyJoin5.png

  2. In the Mode area, select
    Table to display
    the source file and the tBlockedFuzzyJoin results together to be
    able to compare them.
  3. Do the same for the other two
    tLogRow     components.
  4. Save your Job and press F6 to
    execute it.

Three output tables are written on the console. The first shows the match entries,
the second show the possible match entries and the third shows the non match entries
according to the used matching method in the defined columns.

The figure below illustrates extractions of the three output tables.

use_case-tblockedfuzzyjoin.png

Document get from Talend https://help.talend.com
Thank you for watching.
Docs 6.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x