• the Configuration view, where you can set the
  match rules and the blocking column(s).

• the matching chart, which presents the graphic matching result,

• the matching table, which presents the details of the matching result.

From this view, you can edit the configuration of the tMatchGroup component or define different configurations in which to execute
the Job.

You can use these different configurations for testing purposes for example, but
you can only save one configuration from the wizard, the open configuration.

In each configuration, you can define the parameters to generate match rules with the VSR
or the T-Swoosh algorithm. The settings of the Configuration view differ slightly depending if you select Simple VSR or T-Swoosh in the
basic settings of the tMatchGroup component.

You can define survivorship rules, blocking key(s) and multiple conditions using several
match rules. You can also set different match intervals for each rule. The match results on
multiple conditions will list data records that meet any of the defined rules. When a
configuration has multiple conditions, the Job conducts an OR match operation. It evaluates
data records against the first rule and the records that match are not evaluated against the
other rules.

For an example of a match rule with the T-Swoosh algorithm, see Using survivorship functions to merge two records and create a master record.

• 48 items are analyzed and classified into 18 groups according to a given match
  rule and after excluding items that are unique, by setting the Hide groups less than parameter to 2.

• 11 groups have 2 items each. In each group, the 2 items are duplicates of each
  other.

• 3 groups have 3 items each. In each group, these items are duplicates of one
  another.

• 3 groups have 4 items each. In each group, these items are duplicates of one
  another.

• One single group has 5 duplicate items.

Two records match if at least one of the match rules is satisfied. As soon as two records
match according to a given rule, the other rules are not checked.

In each block, the blocking keys must have the same value. Then, each block is processed
independently.

Using blocking keys reduces the time needed by the Simple VSR Matcher and the T-Swoosh
algorithms to process data. For example, if 100,000 records are split into 100 blocks of
1,000 records each, the number of comparisons are reduced by a factor 100. This means
the algorithm runs around 100 times faster.

It is recommended to use the tGenKey component to generate blocking
keys and to visualize the statistics regarding the number of blocks. In a Job,
right-click the tGenKey component and select View Key
Profile in the contextual menu to visualize the distribution of the
number of blocks according to their size.

In this example, the average block size is around 40.

For the 13 blocks with 38 rows, there are 18,772 comparisons in these 13 blocks
(13 × 382). If records are compared with four columns, this means there will be 75,088
string comparisons in these 13 blocks (18,772 × 4).

If a record does not match any of the previous master records, it is considered as a new
master record and added to the lookup table. This means that the first record of the
dataset is necessarily a master record. So, the order of the records is important and
can have an impact on the creation process of the master records.

When a record matches a master record, the Simple VSR Matcher algorithm does not further
attempt to match with other master records because all the master records in the lookup
table are not similar. So, once a record matches a master record, the chance of matching
another master record is low.

This means a record can only exist in one group of records and be linked to one master
record.

For example, take the following set of records as input:

The algorithm processes the input records as follows:

1. The algorithm takes record 1 and compares it with an empty set of records. Since
   record 1 does not match any record, it is added to the lookup table.
2. The algorithm takes record 2 and compares it with record 1. Since it is not a match,
   record 2 is added to the lookup table.
3. The algorithm takes record 3 and compares it with record 1 and record 2. Record 3
   matches record 1. So, record 3 is added to the group of record 1.
4. The algorithm takes record 4 and compares it with record 1 and record 2 but not with
   record 3, which is not a master record. Since it is not a match, record 4 is added
   to the lookup table.

The output will look like this:

To create master records, you can design survivorship rules to decide which attribute
will survive.

There are two types of survivorship rules:

• The rules related to matching keys: each attribute used as a matching key can have a
  specific survivorship rule.
• The default rules: they are applied to all the attributes of the same data type
  (Boolean, String, Date, Number).

If a column is a matching key, the rule related to matching keys specific to this column
is applied.

If the column is not a matching key, the default survivorship rule for this data type is
applied. If the default survivorship rule is not defined for the data type, the
Most common survivorship function is used.

Each time two records are merged to create a new master record, this new master record is
added to the queue of records to be examined. The two records that are merged are
removed from the lookup table.

For example, take the following set of records as input:

The survivorship rule uses the Concatenate function with
“,” as a parameter to separate values.

At the beginning of the process, the queue contains all the input records and the lookup
is empty. To process the input records, the algorithm iterates until the queue is
empty:

1. The algorithm takes record 1 and compares it with an empty set of records. Since
   record 1 does not match any record, it is added to the set of master records. The
   queue contains now record 2, record 3 and record 4. The lookup contains record
   1.
2. The algorithm takes record 2 and compares it with record 1. Since record 2 does not
   match any record, it is added to the set of master records. The queue contains now
   record 3 and record 4. The lookup contains record 1 and record 2.
3. The algorithm takes record 3 and compares it with record 1. Record 3 matches record
   1. So, record 1 and record 3 are merged to create a new master record called record
   1,3. The queue contains now record 4 and record 1,3. The lookup contains record
   2.
4. The algorithm takes record 4 and compares it with record 2. Since it is not a match,
   record 4 is added to the set of master records. The queue contains now record 1,3.
   The lookup table contains record 2 and record 4.
5. The algorithm takes record 1,3 and compares it with record 2 and record 4. Record
   1,3 matches record 4. So, record 1,3 and record 4 are merged to create a new master
   record called record 1,3,4. Record 4 is removed from the lookup table. Since record
   1,3 was the result of a previous merge, it is removed from the table. The queue now
   contains record 1,3,4. The lookup contains record 2.
6. The algorithm takes record 1,3,4 and compares it with record 2. Since it is not a
   match, record 1,3,4 is added to the set of master records. The queue is now empty.
   The lookup contains records 1,3,4 and record 2.

The output will look like this:

As you can see in this example, the value in the GRP_QUALITY column can be less than the
Match Threshold parameter. That is
because a group is created from record pairs with a matching score greater than
or equal to the Match Threshold but the
records are not all compared to each other; whereas GRP_QUALITY takes into account all record pairs in the
group.

You can design a Job with consecutive tMatchGroup components to
create data partitions based on different blocking keys.

For example, you want to find duplicates having either the same city or the same zip code
in a customer database. In this particular case, you can use two consecutive
tMatchGroup to process the data partitions:

• One tMatchGroup component in which the column “city” is defined
  as a blocking key.
• One tMatchGroup component in which the column “ZipCode” is
  defined as a blocking key.

What is multi-pass matching?

The idea behind multi-pass matching is to reuse the master records defined in the
previous pass as the input of the current tMatchGroup component.
Multi-pass matching is more effective if the blocking keys are almost not
correlated. For example, it is not relevant to define the column “country” as a
blocking key and the column “city” as another blocking key because all the
comparisons made with the blocking key “city” will also be done with blocking key
“country”.

When using multi-pass matching with the Simple VSR matcher algorithm, only master
records of size 1 – records that did not match any record – are compared with master
records of any size. There are no comparisons between two master records that are
derived from at least two children each.

An example of multi-pass matching

In the following example, the dataset contains four records. It is assumed that the
first tMatchGroup component has a blocking key on the column
“ZipCode”, and the second tMatchGroup component has a blocking
key on the column “city”. The attribute “name” is used as a matching key.

After the first pass, records 1 and 3 are grouped, and records 2 and 4 are grouped.
In these groups, record 1 and record 2 are master records.

In the second tMatchGroup, only the master records from the first
pass, record 1 and record 2, are compared. Since their group size is strictly
greater than 1, they are not compared. Then, the order in which the input records
are sorted is very important.

The following results are returned:

When running the T-Swoosh algorithm with the same parameters and the Most
common survivorship function, the following results are
returned:

This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.

1. Drop the following components from the Palette onto the design workspace: tFileInputExcel, tMatchGroup and three tLogRow components.
2. Connect tFileInputExcel to tMatchGroup using the Main
   link.
3. Click the tMatchGroup component and select the Separate output check box in the Advanced settings view.
4. Connect tMatchGroup to
   the three tLogRow components using the
   Uniques, Matches and Suspects
   links.

1. In the Basic settings view of tFileInputExcel, fill in the File Name field by browsing to the input file
   and set other properties in case they are not stored in the Repository.
   

2. Create the schema through the Edit Schema
   button, if the schema is not already stored in the Repository. Remember to set the data type in the Type column.
   

1. Double-click tMatchGroup to display the
   Basic settings view and define the
   component properties.
2. From the Matching Algorithm list, select
   Simple VSR.
   In this scenario, the match rule is based on the VSR algorithm.
3. Click Sync columns to retrieve the schema
   from the preceding component.
4. Click the Edit schema button to view the
   input and output schema and do any modifications in the output schema, if
   necessary.
   

   In the output schema of this component there are few output standard columns
   that are read-only. For more information, see tMatchGroup Standard properties.
5. Click OK to close the dialog box.
6. Click the […] button next to
   Configure match rules to open the configuration
   wizard and define the component configuration and the match rule(s).
   

   You can use the configuration wizard to import match rules
   created and tested in the studio and stored in the repository, and use them in
   your match Jobs. For further information, see Importing match rules from the studio repository.
   It is important to import or define the same type of the rule
   selected in the basic settings of the component, otherwise the Job runs with
   default values for the parameters which are not compatible between the two
   algorithms.
7. Define the first match rule as the following:

   • In the Key definition table, click
     the [+] button to add to the list the
     column(s) on which you want to do the matching operation,
     lname and fname.

     Note: When you select a date column on which to apply an algorithm or a matching
     algorithm, you can decide what to compare in the date format.

     For example,
     if you want to only compare the year in the date, in the component schema set the type
     of the date column to Date and then enter “yyyy” in the Date Pattern field.
     The component then converts the date format to a string according to the pattern defined
     in the schema before starting a string comparison.

   • Click in the Matching Function column
     and select from the list Jaro-Winkler
     as the method to be used for the matching operation.

     If you select custom as a matching
     type, you must set in the Custom Matcher
     column the path pointing to the custom class (external
     matching algorithm) you need to use. This path is defined by yourself in
     the library file (.jar file).

   • From the Tokenized measure list,
     select not to use a tokenized distance for the selected
     algorithms.

   • Click in the cell of the Weight
     column to set the numerical weights for the two columns used as key
     attributes.

   • Click in the cell of the Handle Null
     column and select the null operator you want to use to handle null
     attributes in the columns. In this example, select Null Match None in order to have matching results where
     null values have minimal effect.

   • Set the match probability in the Match
     Interval field.

8. Follow the same procedure in the above step to define the second match rule.
   Set the address1 column as an input attribute and select
   Jaro as the matching type. Select Null Match None as the null operator. And finally set
   the match probability which can be different from the one set for the first
   rule.
9. Set the Hide groups of less than parameter in
   order to decide what groups to show in the result chart and matching table. This
   parameter enables you to hide groups of small group size.
10. Click the Advanced settings tab and set the
    advanced parameters for the tMatchGroup
    component as the following:
    

    • Select the Separate output check box.

      The component will have three separate output flows: Unique rows, Confident
      groups and Uncertain
      groups.

      If this check box is not selected, the tMatchGroup component will have only one output flow
      where it groups all output data. For an example scenario, see Comparing columns and grouping in the output flow duplicate records that have the same functional key.

    • Select the Sort the output data by
      GID check box to sort the output data by their group
      identifier.

    • Select the Output distance details
      and Display detailed labels check
      boxes.

      The component will output the MATCHING_DISTANCES
      column. This column provides the distance between the input and the
      master columns giving also the names of the columns against which the
      records are matched.

    • Select the Deactivate matching computation when
      opening the wizard check box if you do not want to run
      the match rules the next time you open the wizard.

11. Click the Chart button in the wizard to
    execute the Job in the defined configuration and have the matching results
    directly in the wizard.
    

    The matching chart gives a global picture about the duplicates in the analyzed
    data. The matching table indicates the details of items in each group and colors
    the groups in accordance with their color in the matching chart.
    The Job conducts an OR match operation on the
    records. It evaluates the records against the first rule and the records that
    match are not evaluated against the second rule. The MATCHING_DISTANCES column allows you to
    understand which rule has been used on what records. Some records are matched
    according to the second rule that uses address1 as a key attribute, whereas the other records in the group are
    matched according to the first rule which uses the lname and fname
    as key attributes.
    You can set the Hide groups of less than
    parameter in order to decide what groups to show in the matching chart and
    table

1. Double-click each of the tLogRow
   components to display the Basic
   settings view and define the component properties.
2. Save your Job and press F6 to execute
   it.

You can see that records are grouped together in three different groups.
Each record is listed in one of the three groups according to the value of
the group score which is the minimal distance computed in the group.

The identifier for each group, which is of String
data type, is listed in the GID column next to the
corresponding record. This identifier will be of the data type
Long for Jobs that are migrated from older
releases. To have the group identifier as String, you
must replace the tMatchGroup component in
the imported Job with tMatchGroup from the
studio Palette.

The number of records in each of the three output blocks is listed
in the GRP_SIZE column and computed only on the master record.
The MASTER column indicates with true or false whether the
corresponding record is a master record or not. The SCORE
column lists the calculated distance between the input record and the master record
according to the Jaro-Winkler and Jaro matching algorithms.

The Job evaluates the records against the first rule and the records that
match are not evaluated against the second rule.

All records with a group score between the match
interval, 0.95 or 0.85 depending on the applied rule, and the confidence
threshold defined in the advanced settings of tMatchGroup are listed in the Suspects output flow.

All records with a group score greater than or
equal to one of the match probabilities are listed in the Matches output flow.

All records with a group size equal to 1 is listed in the
Uniques output flow.

For another scenario that groups the output records in one single output flow
based on a generated functional key, see Comparing columns and grouping in the output flow duplicate records that have the same functional key.

This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.

1. Drop the following components from the Palette onto the design workspace: tFixedFlowInput, tMatchGroup and three tLogRow components.
2. Connect tFixedFlowInput
   to tMatchGroup using the Main link.
3. Click the tMatchGroup component and select the Separate output check box in the Advanced settings view.
4. Connect tMatchGroup to
   the three tLogRow components using the
   Uniques, Matches and Suspects
   links.

1. Double-click the tFixedFlowInput
   component to display its Basic settings view.
2. Select Built-in and click the […]
   button next to Edit Schema.
3. Define the seven columns and click OK.
4. Select Use Inline Content(delimited file).
5. Fill in the Row Separator and Field
   Separator fields.
6. Enter the input data in the Content field.

1. Click the tMatchGroup
   component to display its Basic settings
   view.
2. From the Matching
   Algorithm list, select t-Swoosh.
3. Click Sync columns to retrieve the schema
   from the preceding component.
4. Click the Edit schema button to view the
   input and output schema and do any modifications in the output schema, if
   necessary.
   In the output schema of this component there are output standard columns
   that are read-only. For more information, see the tMatchGroup
   properties.
5. Click OK to close the dialog box.
6. Click the Advanced
   settings tab and select the following check boxes:
   1. Separate output

      The component will have three separate output flows:
      Uniques,
      Matches
      and Suspects.

      If this check box is not selected, the tMatchGroup component will have only
      one output flow where it groups all output data. For an example
      scenario, see Comparing columns and grouping in the output flow duplicate records that have the same functional key.

   2. Sort the output data by
      GID
   3. Output distance details
   4. Display detailed labels
   5. Deactivate matching
      computation when opening the wizard
7. Click the […] button next to Configure match
   rules to define the component configuration and the match
   rule(s).
   You can use the configuration wizard to import
   match rules created and tested in the studio and stored in the repository, and
   use them in your match Jobs. For further information, see Importing match rules from the studio repository.
   It is important to import or define the same
   type of the rule selected in the basic settings of the component. Otherwise the
   Job runs with default values for the parameters which are not compatible between
   the two algorithms.
8. Define the match rule as the following:
   1. In the Key definition table,
      click the [+] button to add to
      the list the column(s) on which you want to do the matching operation,
      fname and lname.
      Note: When you select a date column on which to apply an algorithm or a matching
      algorithm, you can decide what to compare in the date format.

      For example,
      if you want to only compare the year in the date, in the component schema set the type
      of the date column to Date and then enter “yyyy” in the Date Pattern field.
      The component then converts the date format to a string according to the pattern defined
      in the schema before starting a string comparison.

   2. Click in the Matching Function
      column and select from the list Jaro-Winkler as the method to be used for the matching
      operation.

      If you select custom
      as a matching type, you must set in the Custom
      Matcher column the path pointing to the custom class
      (external matching algorithm) you need to use. This path is defined by
      yourself in the library file (.jar file).

   3. From the Tokenized measure list,
      select No.
   4. Click in the cell of the Threshold column and
      enter 0.7 for fname and 0.4 for lname.
   5. Click in the cell of the Confidence
      Weight column to set the numerical weights for the two
      columns used as key attributes: 1 for fname and 4 for lname.
   6. Click in the cell of the Handle Null column and select the null
      operator you want to use to handle null attributes in the columns. In
      this example, select Null
      Match NONE in order to have matching results where null
      values have minimal effect.
   7. Select Most common in the
      Survivorship Function.
9. Follow the same procedure in the above step to define the second match rule and
   set the parameters as follows:
   1. Click the [+] button (Duplicate
      Rule).
   2. Input Key Attribute: address
   3. Matching Function:
      Jaro
   4. Tokenized Measure:
      No
   5. Threshold: 0.8
   6. Confidence Weight: 1
   7. Handle Null: Null Match
      NONE
   8. Survivorship Function:
      Most common
10. Set the Match Threshold
    parameter of each Match Rule to 0.8.
11. Set the Hide groups of less than parameter to 2. This
    parameter enables you to hide groups of small size.
12. Click the Chart button
    to execute the Job in the defined configuration and have the matching results
    directly in the wizard.
    

    The matching chart gives a global picture about the duplicates
    in the analyzed data. The matching table indicates the details of items in each
    group and colors the groups in accordance with their color in the matching
    chart.
    The Job conducts an OR match operation on the records. It
    evaluates the records against the rule. The MATCHING_DISTANCES column allows you to
    understand which rule has been used on what records. 

    For example, in the
    second data group (brick red), the last Amic record is matched according to the second rule that uses address1 as a key attribute,
    whereas the other records in the group are matched according to the first
    rule which uses the lname and fname as key
    attributes.

    As you can see in this example, the value in the GRP_QUALITY column can be less than the
    Match Threshold parameter. That is
    because a group is created from record pairs with a matching score greater than
    or equal to the Match Threshold but the
    records are not all compared to each other; whereas GRP_QUALITY takes into account all record pairs in the
    group.

1. Double-click each tLogRow
   component to display the Basic
   settings view.
2. Select Table (print values in cells of a table).
3. Save your Job and press F6 to execute
   it.

You can see that records are grouped together in three different groups.
Each record is listed in one of the three groups according to the value of
the group score which is the minimal distance computed in the group.

The identifier for each group, which is of String
data type, is listed in the GID column next to the
corresponding record. This identifier is of the data type Long for Jobs that are migrated from older releases. To have the group
identifier as String, replace the tMatchGroup component in the imported Job with
tMatchGroup from the studio
Palette.

The number of records in each of the three output blocks
is listed in the GRP_SIZE column and
computed only on the master record. The MASTER column indicates with true or false whether the corresponding
record is a master record or not. The SCORE column lists the calculated distance between the input record and
the master record according to the Jaro-Winkler and Jaro matching algorithms.

The Job evaluates the records against the first rule and the records that
match are not evaluated against the second rule.

All records with a group score between the match interval,
0.95 or 0.85 depending on the applied rule, and the confidence
threshold defined in the advanced settings of tMatchGroup are listed in the Suspects output flow.

All records with a group score greater than or equal to
one of the match probabilities are listed in the Matches output flow.

All records with a group size equal to 1 is listed in the
Uniques output
flow.

For another scenario that groups the output records in one single output flow
based on a generated functional key, see Comparing columns and grouping in the output flow duplicate records that have the same functional key.

In this scenario, the main input schema is already stored in the Repository. For more information about storing schema
metadata in the repository, see the
Talend Studio User
Guide.

1. Double-click tMysqlInput to open its
   Component view.
   

   The property fields for tMysqlInput are
   automatically filled in. If you do not define your input schema locally in
   the repository, fill in the details manually after selecting Built-in in the Schema and Property Type
   lists.
   The input table used in this scenario is called
   customer.
2. Modify the query in the Query box to
   select only the columns you want to match:
   account_name, lname,
   fname, mi,
   address1, city,
   state_province and
   postal_code.

1. Double-click the first tGenKey to open
   the Component view.
   

2. Click

   

   and import blocking keys from match rules created and
   tested in the
   Profiling
   perspective
   of
   Talend Studio
   and use them in your Job. Otherwise,
   define the blocking key parameters as described in the below steps.

3. Under the Algorithm table, click the
   [+] button to add two rows in the
   table.
4. In the column column, click the newly added row and
   select from the list the column you want to process using an algorithm. In this
   example, select lname.
5. Do the same on the second row to select
   postal_code.
6. In the pre-algorithm column, click the newly
   added row and select from the list the pre-algorithm you want to apply to the
   corresponding column.
   In this example, select remove diacritical marks and
   convert to upper case to remove any diacritical mark and converts
   the fields of the lname column to upper
   case.
   This conversion does not change your raw data.
7. In the algorithm column, click the newly
   added row and select from the list the algorithm you want to apply to the
   corresponding column. In this example, select N first
   characters of each word.
   If you select the Show help check box, you
   can display instructions on how to set algorithms/options parameters.
8. Do the same for the second row on the algorithm column to select first N
   characters of the string.
9. Click in the Value column next to the
   algorithm column and enter the value for
   the selected algorithm, when needed.
   In this scenario, enter 1 for both rows. The first letter
   of each field in the corresponding columns will be used to generate the
   key.
   Make sure to set a value for the algorithm which need one, otherwise you may
   have a compilation error when you run the Job.

1. Double-click the second tGenKey to open
   the Component view.
   

2. In the Algorithm table, define the column
   you want to use to partition data, account_num in this
   component. Select the first N characters of the
   string algorithm and set the value to 1
   in the Value column.
   Each of the two tGenKey components will
   generate a read_only T_GEN_KEY column in
   the output schema. You must rename one of theT_GEN_KEY columns to stop them from overwriting each
   other.
3. Double-click the tMap component to open
   its editor.
   

4. In the Schema editor, copy the columns
   from the first table onto the second table and rename T_GEN_KEY to T_GEN_KEY1, for example.
5. In the top part of the editor, drop all columns from the input table to
   the output table.
6. Click Ok to save data transformation and
   close the editor.
7. In the tGenKey basic settings, click the

   

   button to verify that the two generated keys are named
   differently in the output schema.

   

1. In the basic settings of the tMatchGroup
   labelled pass1, select Simple VSR from the Matching
   Algorithm list.
   In this scenario, the match rule is based on the VSR algorithm.
2. Click the Preview button to display the
   Configuration Wizard.
   

3. Click

   

   and import matching keys from the match rules created and
   tested in the
   Profiling
   perspective of
   
   Talend Studio
   and use them in your Job. Otherwise, define
   the matching key parameters as described in the below steps.

   It is important to import or define the same type of the rule selected in the
   basic settings of the component, otherwise the Job runs with default values for
   the parameters which are not compatible between the two algorithms.
4. In the Key definition table, click the
   [+] button to add the column(s) on which
   you want to do the matching operation, lname
   in this scenario.
   Note: When you select a date column on which to apply an algorithm or a matching
   algorithm, you can decide what to compare in the date format.

   For example,
   if you want to only compare the year in the date, in the component schema set the type
   of the date column to Date and then enter “yyyy” in the Date Pattern field.
   The component then converts the date format to a string according to the pattern defined
   in the schema before starting a string comparison.

5. Select the Jaro-Winkler algorithm in the Matching Function column.
6. From the Tokenized measure list, select
   Any order.
7. Set Weight to 1 and in
   the Handle Null column, select the null
   operator you want to use to handle null attributes in the column, Null Match Null in this scenario.
8. Click the [+] button below the Blocking Selection table to add one row in the table
   then click in the line and select from the list the column you want to use as a
   blocking value, T_GEN_KEY in this example.
   Using a blocking value reduces the number of pairs of records that needs to be
   examined. The input data is partitioned into exhaustive blocks based on the
   functional key. This will decrease the number of pairs to compare, as comparison
   is restricted to record pairs within each block.
9. If required, click Edit schema to open the
   schema editor and see the schema retrieved from the previous component in the
   Job.
   

10. Click the Advanced settings tab and select
    the Sort the output data by GID check box to
    arrange the output data by their group IDs.
11. Select the Deactivate matching computation when opening
    the wizard check box if you do not want to run the match rules
    the next time you open the wizard.

1. In the basic settings of the tMatchGroup
   labelled pass2, select Simple VSR from the Matching
   Algorithm list.
   In this scenario, the match rule is based on the VSR algorithm.
2. Click the Preview button to display the
   Configuration Wizard.
   If this component does not have the same schema of the preceding component, a
   warning icon appears. If so, click the Sync
   columns button to retrieve the schema from the preceding one and
   once done, the warning icon disappears.
3. In the Key Definition table, click
   the [+] button to add the column(s) on
   which you want to do the matching operation, lname in this scenario.
   Note: When you select a date column on which to apply an algorithm or a matching
   algorithm, you can decide what to compare in the date format.

   For example,
   if you want to only compare the year in the date, in the component schema set the type
   of the date column to Date and then enter “yyyy” in the Date Pattern field.
   The component then converts the date format to a string according to the pattern defined
   in the schema before starting a string comparison.

4. Select the Jaro-Winkler algorithm in the Matching Function column.
5. Set Weight to 1 and in
   the Handle Null column, select the null
   operator you want to use to handle null attributes in the column, Null Match Null in this scenario.
6. Click the [+] button below the Blocking Selection table to add one row in the table
   then click in the line and select from the list the column you want to use as a
   blocking value, T_GEN_KEY1 in this example.
   
7. Click the Advanced settings tab and select
   the Multi-pass check box. This option enables
   tMatchGroup to receive data sets from the
   tMatchGroup that precedes it in the Job.
   
8. In the Advanced settings view, select the
   Sort the output data by GID check box to
   arrange the output data by their group IDs.
9. Select the Deactivate matching computation when opening
   the wizard check box if you do not want to run the match rules
   the next time you open the wizard.

1. Double-click each of the tLogRow
   components to open the Component view and
   in the Mode area, select the Table (print values in cells of a table)
   option.
2. Save your Job and press F6 to execute
   it.
   The results after the first pass read as follows:
   

   The results after the second pass read as follows:
   

   When you compare, for example, the customer name Wilson from the results of the two passes, you will find
   that more customers using the last name Wilson are grouped together after the second pass.

In this scenario, the main input schema is already stored in the Repository. For more information about storing schema metadata in the
repository, see the
Talend Studio User Guide.

The main input file contains the columns: id,
first_name and age. The data in this input
file has problems such as duplication, names spelled differently or wrongly, different
information for the same customer.

1. Double-click tMatchGroup to display the
   Basic settings view and define the
   component properties.
2. From the Matching Algorithm list, select
   T-Swoosh.
   In this scenario, the match rule is based on the T-Swoosh algorithm.
3. Click Sync columns to retrieve the schema
   from the preceding component.
4. Click the Edit schema button to view the
   input and output schema and do any modifications in the output schema, if
   necessary.
   

   In the output schema of this component there are few output standard columns
   that are read-only. For more information, see tMatchGroup Standard properties.
5. In the Advanced settings view, leave the
   parameters selected by default.

1. In the tMatchGroup basic settings, click
   Preview to open the configuration wizard
   and define the matching key and the survivorship function.
   

   You can use the configuration wizard to import match rules created and tested
   in the studio and stored in the repository, and use them in your match Jobs. For
   further information, see Importing match rules from the studio repository.
   It is important to have the same type of the matching algorithm selected in
   the basic settings of the component and defined in the configuration wizard.
   Otherwise the Job runs with default values for the parameters which are not
   compatible between the two algorithms.
2. Define the match rule as the following:

   • In the Key definition table, click
     the [+] button to add a line in the
     table. Click in the Input Key Attribute
     column and select the column on which you want to do the matching
     operation, first_name in this scenario.

   • Click in the Matching Function column
     and select Soundex from the list. This
     method matches processed entries according to a standard English
     phonetic algorithm which indexes strings by sound, as pronounced in
     English.

   • From the Tokenized measure list,
     select not to use a tokenized distance for the selected
     algorithm.

   • Set the Threshold to 0.8 and the Confidence Weight to 1.

   • Select Null Match None in the
     Handle Null column in order to have
     matching results where null values have minimal effect.

   • Select Most common
     in the Survivorship Function
     column. This method validates the most frequent name value in each group
     of duplicates.

3. Define the default survivorship rule as the following:

   • In the Default Survivorship
     Rules table, click the [+] button to add a line in the table. Click in the
     Data Type column and select
     Number.

   • Click in the Survivorship
     Function column and select Largest (for numbers) from the list. This method
     validates the largest numerical value in each group.

4. Set the Hide groups of less than parameter in
   order to decide what groups to show in the result chart and matching table. This
   parameter enables you to hide groups of small group size.
5. Click the Chart button in the wizard to
   execute the Job in the defined configuration and have the results directly in
   the wizard.
   

   The matching chart gives a global picture about the duplicates in the analyzed
   data. The matching table indicates the details of the items in each group,
   colors the groups in accordance with their color in the matching chart and
   indicates with true the records which are
   master records. The master record in each group is the result of merging two
   similar records according to the phonetic algorithm and survivorship rule. The
   master record is a new record that does not exist in the input data.
6. Click OK to close the wizard.

• define multiple conditions using several match rules to group
  data,

• set different match intervals for each rule,

• import match rules created and tested in the studio and stored in
  the repository, and use them in your match Jobs. You can only import rules
  configured with the VSR algorithm. For further information, see Importing match rules from the studio repository.

• select a blocking key to partition data.