tFileInputRegex

Reads a file row by row to split them up into fields using regular expressions and
sends the fields as defined in the schema to the next component.

Powerful feature which can replace number of other
components of the File family. Requires some advanced knowledge on regular expression
syntax.

Depending on the Talend solution you
are using, this component can be used in one, some or all of the following Job
frameworks:

Standard: see tFileInputRegex Standard properties.

The component in this framework is generally available.
MapReduce: see tFileInputRegex MapReduce properties.

The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.
Spark Batch: see tFileInputRegex properties for Apache Spark Batch.

The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.
Spark Streaming: see tFileInputRegex properties for Apache Spark Streaming.

The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.

tFileInputRegex Standard properties

These properties are used to configure tFileInputRegex running
in the Standard Job framework.

The Standard
tFileInputRegex component belongs to the File
family.

The component in this framework is generally available.

Basic settings

Property type	Either Built-In or Repository.
	Built-In: No property data stored centrally.
	Repository: Select the repository file where the properties are stored.
File Name/Stream	File name: Name of the file and/or the variable to be processed Stream: Data flow to be processed. The data must be added to the flow so that it can be collected by the tFileInputRegex via the INPUT_STREAM variable in the autocompletion list (Ctrl+Space) For further information about how to define and use a variable in a Job, see Talend Studio User Guide.
Row separator	The separator used to identify the end of a row.
Regex	This field can contain multiple lines. Type in your regular expressions including the subpattern matching the fields to be extracted. Note: Antislashes need to be doubled in regexp Warning: Regex syntax requires double quotes.
Header	Enter the number of rows to be skipped in the beginning of file.
Footer	Number of rows to be skipped at the end of the file.
Limit	Maximum number of rows to be processed. If Limit = 0, no row is read or processed.
Ignore error message for the unmatched record	Select this check box to avoid outputing error messages for records that do not match the specified regex. This check box is cleared by default.
Schema and Edit Schema	A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository. Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available: View schema: choose this option to view the schema only. Change to built-in property: choose this option to change the schema to Built-in for local changes. Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.
	Built-In: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.
	Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Related topic: see Talend Studio User Guide.
Skip empty rows	Select this check box to skip the empty rows.
Die on error	Select the check box to stop the execution of the Job when an error occurs. Clear the check box to skip any rows on error and complete the process for error-free rows. When errors are skipped, you can collect the rows on error using a Row > Reject link.

Advanced settings

Encoding	Select the encoding from the list or select Custom and define it manually. This field is compulsory for database data handling. In the Map/Reduce version of tFileInputRegex, you need to select the Custom encoding check box to display this list.
tStatCatcher Statistics	Select this check box to gather the Job processing metadata at a Job level as well as at each component level.

Encoding

Select the encoding from the list or select Custom and define it manually. This field is compulsory for database
data handling.

In the Map/Reduce version of tFileInputRegex, you need to select the Custom encoding check box to display
this list.

tStatCatcher Statistics

Select this check box to gather the Job processing
metadata at a Job level as well as at each component level.

Global Variables

Global Variables	NB_LINE: the number of rows processed. This is an After variable and it returns an integer. ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box. A Flow variable functions during the execution of a component while an After variable functions after the execution of the component. To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it. For further information about variables, see Talend Studio User Guide.

NB_LINE: the number of rows processed. This is an After
variable and it returns an integer.

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule	Use this component to read a file and separate fields contained in this file according to the defined Regex. You can also create a rejection flow using a Row > Reject link to filter the data which doesn’t correspond to the type defined. For an example of how to use these two links, see Procedure.

Scenario: Regex to Positional file

The following scenario creates a two-component Job, reading data from an Input
file using regular expression and outputting delimited data into a positional file.

Dropping and linking the components

Drop a tFileInputRegex component from the
Palette to the design workspace.
Drop a tFileOutputPositional component the
same way.
Right-click on the tFileInputRegex component
and select Row
>

Main. Drag this main row link
onto the tFileOutputPositional component and
release when the plug symbol displays.

Configuring the components

Select the tFileInputRegex again so the
Component view shows up, and define the
properties:
The Job is built-in for this scenario. Hence, the Properties are set for this
station only.
Fill in a path to the file in File Name
field. This field is mandatory.
Define the Row separator identifying the end
of a row.
Then define the Regular expression in order
to delimit fields of a row, which are to be passed on to the next component. You
can type in a regular expression using Java code, and on mutiple lines if
needed.

Warning:

Regex syntax requires double quotes.
In this expression, make sure you include all subpatterns matching the fields
to be extracted.
In this scenario, ignore the header, footer and limit fields.
Select a local (Built-in) Schema to define the data to pass on to the tFileOutputPositional component.
You can load or create the schema through the Edit
Schema function.
Then define the second component properties:
Enter the Positional file output path.
Enter the Encoding standard, the output file
is encoded in. Note that, for the time being, the encoding consistency
verification is not supported.
Select the Schema type. Click on Sync columns to automatically synchronize the schema
with the Input file schema.

Saving and executing the Job

Press Ctrl+S to save your Job.
Now go to the Run tab, and click on Run to execute the Job.

The file is read row by row and split up into fields based on the Regular Expression definition. You can open it using
any standard file editor.