tExtractRegexFields

Extracts data and generates multiple columns from a formatted string using regex
matching.

Depending on the Talend solution you
are using, this component can be used in one, some or all of the following Job
frameworks:

Standard: see tExtractRegexFields Standard properties.

The component in this framework is generally available.
MapReduce: see tExtractRegexFields MapReduce properties.

The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.
Spark Batch:
see tExtractRegexFields properties for Apache Spark Batch.

The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.
Spark Streaming:
see tExtractRegexFields properties for Apache Spark Streaming.

The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.

tExtractRegexFields Standard properties

These properties are used to configure tExtractRegexFields running in the Standard Job framework.

The Standard
tExtractRegexFields component belongs to the Data Quality and the Processing families.

The component in this framework is generally available.

Basic settings

Field to split	Select an incoming field from the Field to split list to split.
Regex	Enter a regular expression according to the programming language you are using.
Schema and Edit Schema	A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository. Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window. Click Sync columns to retrieve the schema from the previous component connected in the Job. Warning: Make sure that the output schema does not contain any column with the same name as the input column to be split. Otherwise, the regular expression will not work as expected.
	Built-In: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.
	Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Related topic: see Talend Studio User Guide.

Advanced settings

Die on error	Select the check box to stop the execution of the Job when an error occurs. Clear the check box to skip any rows on error and complete the process for error-free rows. When errors are skipped, you can collect the rows on error using a Row > Reject link.
Check each row structure against schema	Select this check box to check whether the total number of columns in each row is consistent with the schema. If not consistent, an error message will be displayed on the console.
tStatCatcher Statistics	Select this check box to gather the processing metadata at the Job level as well as at each component level.

Die on error

Select the check box to stop the execution of the Job when an error
occurs.

Clear the check box to skip any rows on error and complete the process for
error-free rows. When errors are skipped, you can collect the rows on error using a Row > Reject link.

Check each row structure against
schema

Select this check box to check whether the total number of columns
in each row is consistent with the schema. If not consistent, an
error message will be displayed on the console.

tStatCatcher Statistics

Select this check box to gather the processing metadata at the Job
level as well as at each component level.

Global Variables

Global Variables	ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box. A Flow variable functions during the execution of a component while an After variable functions after the execution of the component. To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it. For further information about variables, see Talend Studio User Guide.

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule	This component handles flow of data therefore it requires input and output components. It allows you to extract data from a delimited field, using a Row > Main link, and enables you to create a reject flow filtering data which type does not match the defined type.

Scenario: Extracting name, domain and TLD from e-mail addresses

This scenario describes a three-component Job where tExtractRegexFields is used to specify a regular expression that
corresponds to one column in the input data, email. The tExtractRegexFields component is used to perform the actual
regular expression matching. This regular expression includes field identifiers for user
name, domain name and Top-Level Domain (TLD) name portions in each e-mail address. If
the given e-mail address is valid, the name, domain and TLD are extracted and displayed
on the console in three separate columns. Data in the other two input columns,
id and age is extracted and routed to
destination as well.

Setting up the Job

Drop the following components from the Palette onto the design workspace: tFileInputDelimited, tExtractRegexFields, and tLogRow.
Connect tFileInputDelimited to tExtractRegexFields using a Row > Main link, and do
the same to connect tExtractRegexFields to
tLogRow.

Configuring the components

Double-click the tFileInputDelimited
component to open its Basic settings view
in the Component tab.
Click the […] button next to the
File name/Stream field to browse to the
file where you want to extract information from.

The input file used in this scenario is called test4.
It is a text file that holds three columns: id,
email, and
age.

id;email;age 1;anna@yahoo.net;24 2;diana@sohu.com;31 3;fiona@gmail.org;20

1
2
3
4

id;email;age
1;anna@yahoo.net;24
2;diana@sohu.com;31
3;fiona@gmail.org;20

For more information, see tFileInputDelimited.
Click Edit schema to define the data
structure of this input file.
Double-click the tExtractRegexFields
component to open its Basic settings
view.
Select the column to split from the Field to
split list: email in this
scenario.
Enter the regular expression you want to use to perform data matching in
the Regex panel. In this scenario, the
regular expression "([a-z]*)@([a-z]*).([a-z]*)" is used to
match the three parts of an email address: user name, domain name and TLD
name.

For more information about the regular expression, see http://en.wikipedia.org/wiki/Regular_expression.
Click Edit schema to open the [Schema of tExtractRegexFields] dialog box, and
click the plus button to add five columns for the output schema.

In this scenario, we want to split the input email
column into three columns in the output flow, name,
domain, and tld. The two other input
columns will be extracted as they are.
Double-click the tLogRow component to
open its Component view.
In the Mode area, select Table (print values in cells of a table).

Saving and executing the Job

Press Ctrl+S to save your Job.
Execute the Job by pressing F6 or
clicking Run on the Run tab.

The tExtractRegexFields component matches all
given e-mail addresses with the defined regular expression and extracts the name,
domain, and TLD names and displays them on the console in three separate columns.
The two other columns, id and age, are
extracted as they are.

tExtractRegexFields MapReduce properties

These properties are used to configure tExtractRegexFields running in the MapReduce Job framework.

The MapReduce
tExtractRegexFields component belongs to the Processing family.

The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.

Basic settings

Field to split	Select an incoming field from the Field to split list to split.
Regex	Enter a regular expression according to the programming language you are using.
Property type	Either Built-In or Repository.
	Built-In: No property data stored centrally.
	Repository: Select the repository file where the properties are stored. The properties are stored centrally under the Hadoop Cluster node of the Repository tree. The fields that come after are pre-filled in using the fetched data. For further information about the Hadoop Cluster node, see the Getting Started Guide.
Schema and Edit Schema	A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository. Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window. Click Sync columns to retrieve the schema from the previous component connected in the Job. Warning: Make sure that the output schema does not contain any column with the same name as the input column to be split. Otherwise, the regular expression will not work as expected.
	Built-In: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.
	Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Related topic: see Talend Studio User Guide.

Advanced settings

Die on error	Select the check box to stop the execution of the Job when an error occurs. Clear the check box to skip any rows on error and complete the process for error-free rows. When errors are skipped, you can collect the rows on error using a Row > Reject link.
Encoding	Select the encoding from the list or select Custom and define it manually. This field is compulsory for database data handling.

Die on error

Select the check box to stop the execution of the Job when an error
occurs.

Clear the check box to skip any rows on error and complete the process for
error-free rows. When errors are skipped, you can collect the rows on error using a Row > Reject link.

Encoding

Select the encoding from the list or select Custom and define it manually. This field is compulsory for database
data handling.

Global Variables

Global Variables	ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box. A Flow variable functions during the execution of a component while an After variable functions after the execution of the component. To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it. For further information about variables, see Talend Studio User Guide.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule	In a Talend Map/Reduce Job, this component is used as an intermediate step and other components used along with it must be Map/Reduce components, too. They generate native Map/Reduce code that can be executed directly in Hadoop. You need to use the Hadoop Configuration tab in the Run view to define the connection to a given Hadoop distribution for the whole Job. This connection is effective on a per-Job basis. For further information about a Talend Map/Reduce Job, see the sections describing how to create, convert and configure a Talend Map/Reduce Job of the Talend Open Studio for Big Data Getting Started Guide . Note that in this documentation, unless otherwise explicitly stated, a scenario presents only Standard Jobs, that is to say traditional Talend data integration Jobs, and non Map/Reduce Jobs.

Usage rule

In a
Talend
Map/Reduce Job, this component is used as an intermediate
step and other components used along with it must be Map/Reduce components, too. They
generate native Map/Reduce code that can be executed directly in Hadoop.

You need to use the Hadoop Configuration tab in the
Run view to define the connection to a given Hadoop
distribution for the whole Job.

This connection is effective on a per-Job basis.

For further information about a
Talend
Map/Reduce Job, see the sections
describing how to create, convert and configure a
Talend
Map/Reduce Job of the

Talend Open Studio for Big Data Getting Started
Guide
.

Note that in this documentation, unless otherwise
explicitly stated, a scenario presents only Standard Jobs,
that is to say traditional
Talend
data integration Jobs, and non Map/Reduce Jobs.

Related scenarios

No scenario is available for the Map/Reduce version of this component yet.

tExtractRegexFields properties for Apache Spark Batch

These properties are used to configure tExtractRegexFields running in the Spark Batch Job framework.

The Spark Batch
tExtractRegexFields component belongs to the Processing family.

The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.

Basic settings

Prev.Comp.Column list	Select the column you need to extract data from.
Regex	Enter a regular expression according to the programming language you are using.
Schema and Edit Schema	A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository. Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window. Click Sync columns to retrieve the schema from the previous component connected in the Job. Warning: Make sure that the output schema does not contain any column with the same name as the input column to be split. Otherwise, the regular expression will not work as expected.
	Built-In: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.
	Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Related topic: see Talend Studio User Guide.
Die on error	Select the check box to stop the execution of the Job when an error occurs.

Advanced settings

Encoding	Select the encoding from the list or select Custom and define it manually. This field is compulsory for database data handling.

Usage

Usage rule	This component is used as an intermediate step. This component, along with the Spark Batch component Palette it belongs to, appears only when you are creating a Spark Batch Job. Note that in this documentation, unless otherwise explicitly stated, a scenario presents only Standard Jobs, that is to say traditional Talend data integration Jobs.
Spark Connection	You need to use the Spark Configuration tab in the Run view to define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files: Yarn mode: when using Google Dataproc, specify a bucket in the Google Storage staging bucket field in the Spark configuration tab; when using other distributions, use a tHDFSConfiguration component to specify the directory. Standalone mode: you need to choose the configuration component depending on the file system you are using, such as tHDFSConfiguration or tS3Configuration. This connection is effective on a per-Job basis.

Usage rule

This component is used as an intermediate step.

This component, along with the Spark Batch component Palette it belongs to, appears only
when you are creating a Spark Batch Job.

Note that in this documentation, unless otherwise
explicitly stated, a scenario presents only Standard Jobs,
that is to say traditional
Talend
data integration Jobs.

Spark Connection

You need to use the Spark Configuration tab in
the Run view to define the connection to a given
Spark cluster for the whole Job. In addition, since the Job expects its dependent jar
files for execution, you must specify the directory in the file system to which these
jar files are transferred so that Spark can access these files:

Yarn mode: when using Google
Dataproc, specify a bucket in the Google Storage staging
bucket field in the Spark
configuration tab; when using other distributions, use a
tHDFSConfiguration
component to specify the directory.
Standalone mode: you need to choose
the configuration component depending on the file system you are using, such
as tHDFSConfiguration
or tS3Configuration.

This connection is effective on a per-Job basis.

Related scenarios

No scenario is available for the Spark Batch version of this component
yet.

tExtractRegexFields properties for Apache Spark Streaming

These properties are used to configure tExtractRegexFields running in the Spark Streaming Job framework.

The Spark Streaming
tExtractRegexFields component belongs to the Processing family.

The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.

Basic settings

Prev.Comp.Column list	Select the column you need to extract data from.
Regex	Enter a regular expression according to the programming language you are using.
Schema and Edit Schema	A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository. Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window. Click Sync columns to retrieve the schema from the previous component connected in the Job. Warning: Make sure that the output schema does not contain any column with the same name as the input column to be split. Otherwise, the regular expression will not work as expected.
	Built-In: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.
	Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Related topic: see Talend Studio User Guide.
Die on error	Select the check box to stop the execution of the Job when an error occurs.

Advanced settings

Encoding	Select the encoding from the list or select Custom and define it manually. This field is compulsory for database data handling.

Usage

Usage rule	This component is used as an intermediate step. This component, along with the Spark Streaming component Palette it belongs to, appears only when you are creating a Spark Streaming Job. Note that in this documentation, unless otherwise explicitly stated, a scenario presents only Standard Jobs, that is to say traditional Talend data integration Jobs.
Spark Connection	You need to use the Spark Configuration tab in the Run view to define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files: Yarn mode: when using Google Dataproc, specify a bucket in the Google Storage staging bucket field in the Spark configuration tab; when using other distributions, use a tHDFSConfiguration component to specify the directory. Standalone mode: you need to choose the configuration component depending on the file system you are using, such as tHDFSConfiguration or tS3Configuration. This connection is effective on a per-Job basis.

Usage rule

This component is used as an intermediate step.

This component, along with the Spark Streaming component Palette it belongs to, appears
only when you are creating a Spark Streaming Job.

Note that in this documentation, unless otherwise explicitly stated, a scenario presents
only Standard Jobs, that is to say traditional
Talend
data
integration Jobs.

Spark Connection

Yarn mode: when using Google
Dataproc, specify a bucket in the Google Storage staging
bucket field in the Spark
configuration tab; when using other distributions, use a
tHDFSConfiguration
component to specify the directory.
Standalone mode: you need to choose
the configuration component depending on the file system you are using, such
as tHDFSConfiguration
or tS3Configuration.

This connection is effective on a per-Job basis.

Related scenarios

No scenario is available for the Spark Streaming version of this component
yet.

Document get from Talend https://help.talend.com

Thank you for watching.

Docs 6.x

0 Comments

Inline Feedbacks

View all comments

tExtractRegexFields – Docs for ESB 6.x

tExtractRegexFields

tExtractRegexFields Standard properties

Basic settings

Advanced settings

Global Variables

Usage

Scenario: Extracting name, domain and TLD from e-mail addresses

Setting up the Job

Configuring the components

Saving and executing the Job

tExtractRegexFields MapReduce properties

Basic settings

Advanced settings

Global Variables

Usage

Related scenarios

tExtractRegexFields properties for Apache Spark Batch

Basic settings

Advanced settings

Usage

Related scenarios

tExtractRegexFields properties for Apache Spark Streaming

Basic settings

Advanced settings

Usage

Related scenarios

My Website Links

Tags