tSparkLog – Docs for ESB 5.x

Component family

Big Data / Spark

Function

tSparkLog terminates a Spark
process and outputs the execution log. This component does not
require any configuration in its Basic
settings view.

Purpose

tSparkLog outputs the execution
log into the Job console of the Studio.

Advanced settings

tStatCatcher Statistics

Select this check box to collect log data at the component
level.

Global Variables

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio
User Guide.

Usage

This component is placed at the end of a Spark process when you
need to read the execution log of that Spark process in the Job
console.

Limitations

It is strongly recommended to use this component in a Spark-only Job, that is to say, to
design and run a Spark Job separately from the non Spark components or Jobs. For example, it
is not recommended to use the tRunJob component to
coordinate a Spark Job and a non Spark Job, or to use the tHDFSPut component along with the Spark components in the same Job.