tDynamoDBLookupInput
Executes a database query with a strictly defined order which must correspond to
the schema definition.
It passes on the extracted data to tMap in order to
provide the lookup data to the main flow. It must be directly connected to a tMap component and requires this tMap to use Reload at each row or Reload at each row (cache) for the lookup flow.
tDynamoDBLookupInput properties for Apache Spark Streaming
These properties are used to configure tDynamoDBLookupInput running in the Spark Streaming Job framework.
The Spark Streaming
tDynamoDBLookupInput component belongs to the Databases family.
The component in this framework is available in Talend Real Time Big Data Platform and in Talend Data Fabric.
Basic settings
Use an existing connection |
Select this check box and in the Component List click the relevant connection component to |
Access |
Enter the access key ID that uniquely identifies an AWS |
Secret |
Enter the secret access key, constituting the security To enter the secret key, click the […] button next to |
Region |
Specify the AWS region by selecting a region name from the |
Use End Point |
Select this check box and in the Server Url field |
Schema and Edit |
A schema is a row description. It defines the number of fields
|
 |
Click Edit
|
Table Name |
Specify the name of the table from which the lookup data is extracted. |
Advanced key condition expression |
Enter the key condition expressions used to determine the items to be read from the The result of the query must contain only records that match join key you need to use in This approach ensures that no redundant records are loaded into memory and outputted to |
Value mapping |
Specify the placeholders for the expression attribute values.
For more information, see Expression Attribute Values. |
Die on error |
Select the check box to stop the execution of the Job when an error |
Advanced settings
Advanced properties |
Add properties to define extra operations you need tDynamoDBInput to perform when reading data. This table is present for future evolution of the component and using it requires the |
Usage
Usage rule |
This component is used as a start component and requires an output link. This component should use a tDynamoDBConfiguration This component, along with the Spark Streaming component Palette it belongs to, appears Note that in this documentation, unless otherwise explicitly stated, a scenario presents |
Spark Connection |
In the Spark
Configuration tab in the Run view, define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |
Related scenarios
For a scenario about how to use the same type of component in a Spark Streaming Job, see
Reading and writing data in MongoDB using a Spark Streaming Job.