tCollectAndCheck
Shows and validates the result of a component test.
tCollectAndCheck is available only in a test case
about a given component and is added automatically to the test case you are using. For
further information about test cases, see
Talend Studio User Guide.
tCollectAndCheck
receives output from a component being tested, loads given reference files to be compare
with this output and returns whether the output matches the expected result.
Depending on the Talend
product you are using, this component can be used in one, some or all of the following
Job frameworks:
-
Spark Batch:
see tCollectAndCheck properties for Apache Spark Batch.The component in this framework is available in all subscription-based Talend products with Big Data
and Talend Data Fabric. -
Spark Streaming: see tCollectAndCheck properties for Apache Spark Streaming.
This component is available in Talend Real Time Big Data Platform and Talend Data Fabric.
tCollectAndCheck properties for Apache Spark Batch
These properties are used to configure tCollectAndCheck running in the Spark Batch Job framework.
The Spark Batch
tCollectAndCheck component belongs to the Technical family.
The component in this framework is available in all subscription-based Talend products with Big Data
and Talend Data Fabric.
Basic settings
Separator |
Enter character, string or regular expression to separate fields for the transferred |
Line separator |
The separator used to identify the end of a row. |
Use context variable |
If you have already created the context variable representing the reference file to be Then syntax to call a variable is context.VariableName. For further information about variables, see |
Reference data |
If you do not want to use context variables to represent the reference data to be used, |
Keep the order from the reference |
If the RDDs to be checked are sorted, select this check box to keep your reference data |
Advanced settings
When the reference is empty, expect no incoming value |
By default, this check box is clear, meaning that when an field in the reference data is If you want the test to expect no value when the reference is empty, select this check |
Usage
Usage rule |
This component is used as an end component and requires an input link. This component is added automatically to a test case being created to show the test result |
Spark Connection |
In the Spark
Configuration tab in the Run view, define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |
Related scenarios
No scenario is available for the Spark Batch version of this component
yet.
tCollectAndCheck properties for Apache Spark Streaming
These properties are used to configure tCollectAndCheck running in the Spark Streaming Job framework.
The Spark Streaming
tCollectAndCheck component belongs to the Technical family.
This component is available in Talend Real Time Big Data Platform and Talend Data Fabric.
Basic settings
Separator |
Enter character, string or regular expression to separate fields for the transferred |
Line separator |
The separator used to identify the end of a row. |
Micro batch separator |
Enter the separator used to identify the end of a micro batch in the |
Use context variable |
If you have already created the context variable representing the reference file to be The syntax to call a variable is context.VariableName. For further information about variables, see |
Reference data |
If you do not want to use context variables to represent the reference data to be used, |
Keep the order from the reference |
If the RDDs to be checked are sorted, select this check box to keep your reference data |
Advanced settings
When the reference is empty, expect no incoming |
By default, this check box is clear, meaning that when an field in the reference data is If you want the test to expect no value when the reference is empty, select this check |
Usage
Usage rule |
This component is used as an end component and requires an input link. This component is added automatically to a test case being created to show the test result |
Spark Connection |
In the Spark
Configuration tab in the Run view, define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |
Related scenarios
No scenario is available for the Spark Streaming version of this component
yet.