tDynamoDBLookupInput
Executes a database query with a strictly defined order which must correspond to
the schema definition.
tDynamoDBLookupInput reads a database
and extracts fields based on a query.
tDynamoDBLookupInput properties for Apache Spark Streaming
These properties are used to configure tDynamoDBLookupInput running in the Spark Streaming Job framework.
The Spark Streaming
tDynamoDBLookupInput component belongs to the Databases family.
The component in this framework is available only if you have
subscribed to Talend Real-Time Big Data Platform or Talend Data Fabric .
Basic settings
Use an existing connection |
Select this check box and in the Component |
Access |
Enter the access key ID that uniquely identifies an AWS Account. For |
Secret |
Enter the secret access key, constituting the security credentials in To enter the secret key, click the […] button next to |
Region |
Specify the AWS region by selecting a region name from the list or entering |
Use End Point |
Select this check box and in the Server Url field |
Schema and Edit |
A schema is a row description. It defines the number of fields (columns) to
|
|
Click Edit schema to make changes to the schema.
|
Table Name |
Specify the name of the table from which the lookup data is extracted. |
Advanced key condition expression |
Enter the key condition expressions used to determine the items to be read from the The result of the query must contain only records that match join key you need to use in This approach ensures that no redundant records are loaded into memory and outputted to |
Value mapping |
Specify the placeholders for the expression attribute values.
For more information, see Expression Attribute Values. |
Die on error |
Select the check box to stop the execution of the Job when an error |
Advanced settings
Advanced properties |
Add properties to define extra operations you need tDynamoDBInput to perform when reading data. This table is present for future evolution of the component and using it requires the |
Usage
Usage rule |
This component is used as a start component and requires an output link. This component should use a tDynamoDBConfiguration This component, along with the Spark Streaming component Palette it belongs to, appears Note that in this documentation, unless otherwise explicitly stated, a scenario presents |
Spark Connection |
You need to use the Spark Configuration tab in
the Run view to define the connection to a given Spark cluster for the whole Job. In addition, since the Job expects its dependent jar files for execution, you must specify the directory in the file system to which these jar files are transferred so that Spark can access these files:
This connection is effective on a per-Job basis. |
Related scenarios
For a scenario about how to use the same type of component in a Spark Streaming Job, see
Reading and writing data in MongoDB using a Spark Streaming Job.