tFSAggregate – Docs for ESB 5.x

Component family

FileScale

Note that this component is deprecated.

Function

tFSAggregate performs an
aggregation based on one or more columns. For each output line, an
aggregation key is provided, as well as the result of the
corresponding aggregation operation (count, avg or sum).

This component has high speed capabilities for aggregating large
scale files.

Purpose

tFSAggregate helps setting
metrics and statistics based on values or calculations from one or
several large scale files.

Basic settings

Schema type and Edit Schema

A schema is a row description, it defines the number of fields to
be processed and passed on to the next component. The schema is
either Built-in or stored remotely
in the Repository.

Click Edit schema to make changes to the schema. If the
current schema is of the Repository type, three options are
available:

Repository: You have already
created the schema and stored it in the Repository. You can reuse it
in various projects and job flowcharts. Related topic: see
Talend Studio
User Guide.

Built-in: You create and store
the schema locally for this component only. Related topic: see
Talend Studio
User Guide.

Property type

Either Built-in or Repository.

Since version 5.6, both the Built-In mode and the Repository mode are
available in any of the Talend solutions.

Built-in: No property data stored
centrally.

Repository: Select the repository
file where Properties are stored. The fields that follow are
pre-filled in using the fetched data.

Input File Name

Name of the file holding the data you want to collect.

Output File Name

Name of the file where you want to write the collected
data.

Record separator (char)

Character, string or regular expression to separate records
(lines).

Field separator (char)

Character, string or regular expression to separate fields in a
record.

Header

Number of records to be skipped in the beginning of the
file.

Footer

Number of records to be skipped at the end of the file.

Group by

Column: List of the columns of
the input file.

Key Attribute: Select the check box
next to the name of the column(s) you want to use to regroup
data.

Operations

Additional Output Column: Enter
the name you want to use for the column containing the results of
the aggregation operation.

Function: Select the type of the
aggregation operation to perform:

count: calculates the number of
lines,

avg: calculates the average,

sum: calculates the sum.

Input Column: Select the input
column from which the values are collected for the aggregation
operation.

Advanced settings

Generate FSLang File

Select this check box to generate the FSLang file corresponding to
your Job and click the three-dot button next to the FSLang File Name field to specify its
path and its name.

Assign FileScale Path

Select this check box and then click the three-dot button next to
the FileScale Path field to select
the FileScale program executive file required to execute the
component.

Specify Number of Process Child

Select this check box and enter the number of child processes to
use for carrying out the aggregation.

Sort results

Select this check box to sort the results.

Custom FileScale Parameter (separated
by,)

Enter the parameters for any specific operation you want to add to
the FileScale executable call.

tStatCatcher Statistics

Select this check box to gather the job processing metadata at a
job level as well as at each component level.

Global Variables

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio
User Guide.

Usage

This component handles files therefore it does not require input
and output data flows. It is used to aggregate data from large scale
files.

Limitation

Limitation is imposed by the limits of physical memory and Central
Processing Unit architectures. For example, total length of
processed files cannot exceed file system limit for LargeFile
support (maximum value of 64 signed bits).