tCollectAndCheck
Shows and validates the result of a component test.
tCollectAndCheck is available only in a test case
about a given component and is added automatically to the test case you are using. For
further information about test cases, see Talend Studio User Guide.
tCollectAndCheck
receives output from a component being tested, loads given reference files to be compare
with this output and returns whether the output matches the expected result.
Depending on the Talend solution you
are using, this component can be used in one, some or all of the following Job
frameworks:
-
Spark Batch:
see tCollectAndCheck properties for Apache Spark Batch.The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data. -
Spark Streaming: see tCollectAndCheck properties for Apache Spark Streaming.
The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.
tCollectAndCheck properties for Apache Spark Batch
These properties are used to configure tCollectAndCheck running in the Spark Batch Job framework.
The Spark Batch
tCollectAndCheck component belongs to the Technical family.
The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.
Basic settings
|
Separator |
Enter character, string or regular expression to separate fields for the transferred |
|
Line separator |
The separator used to identify the end of a row. |
| Use context variable |
If you have already created the context variable representing the reference file to be Then syntax to call a variable is context.VariableName. For further information about variables, see |
|
Reference data |
If you do not want to use context variables to represent the reference data to be used, |
|
Keep the order from the reference |
If the RDDs to be checked are sorted, select this check box to keep your reference data |
Advanced settings
|
When the reference is empty, expect no incoming value |
By default, this check box is clear, meaning that when an field in the reference data is If you want the test to expect no value when the reference is empty, select this check |
Usage
|
Usage rule |
This component is used as an end component and requires an input link. This component is added automatically to a test case being created to show the test result |
|
Spark Connection |
You need to use the Spark Configuration tab in
the Run view to define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |
Related scenarios
No scenario is available for the Spark Batch version of this component
yet.
tCollectAndCheck properties for Apache Spark Streaming
These properties are used to configure tCollectAndCheck running in the Spark Streaming Job framework.
The Spark Streaming
tCollectAndCheck component belongs to the Technical family.
The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.
Basic settings
|
Separator |
Enter character, string or regular expression to separate fields for the transferred |
|
Line separator |
The separator used to identify the end of a row. |
|
Micro batch separator |
Enter the separator used to identify the end of a micro batch in the |
| Use context variable |
If you have already created the context variable representing the reference file to be The syntax to call a variable is context.VariableName. For further information about variables, see |
|
Reference data |
If you do not want to use context variables to represent the reference data to be used, |
|
Keep the order from the reference |
If the RDDs to be checked are sorted, select this check box to keep your reference data |
Advanced settings
|
When the reference is empty, expect no incoming |
By default, this check box is clear, meaning that when an field in the reference data is If you want the test to expect no value when the reference is empty, select this check |
Usage
|
Usage rule |
This component is used as an end component and requires an input link. This component is added automatically to a test case being created to show the test result |
|
Spark Connection |
You need to use the Spark Configuration tab in
the Run view to define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |
Related scenarios
No scenario is available for the Spark Streaming version of this component
yet.