tDynamoDBConfiguration
Stores connection information and credentials to be reused by other DynamoDB
components.
You define the connection to a DynamoDB database in tDynamoDBConfiguration and configure the
other DynamoDB components to reuse this configuration. At runtime, the
Spark executors read this configuration in order to connect to
DynamoDB.
Depending on the Talend
product you are using, this component can be used in one, some or all of the following
Job frameworks:
-
Spark Batch: see tDynamoDBConfiguration properties for Apache Spark Batch.
The component in this framework is available in all subscription-based Talend products with Big Data
and Talend Data Fabric. -
Spark Streaming: see tDynamoDBConfiguration properties for Apache Spark Streaming.
This component is available in Talend Real Time Big Data Platform and Talend Data Fabric.
tDynamoDBConfiguration properties for Apache Spark Batch
These properties are used to configure tDynamoDBConfiguration running in the Spark Batch Job framework.
The Spark Batch
tDynamoDBConfiguration component belongs to the Storage and the Databases families.
The component in this framework is available in all subscription-based Talend products with Big Data
and Talend Data Fabric.
Basic settings
Access key |
Enter the access key ID that uniquely identifies an AWS |
Secret key |
Enter the secret access key, constituting the security To enter the secret key, click the […] button next to |
Region |
Specify the AWS region by selecting a region name from the list. For more information |
Use End Point |
Select this check box and in the field displayed, specify the Web service URL of the |
Usage
Usage rule |
This component is used with no need to be connected to other The configuration in a tDynamoDBConfiguration component This component, along with the Spark Batch component Palette it belongs to, Note that in this documentation, unless otherwise explicitly stated, a |
Related scenarios
For a scenario about how to use the same type of component in a Spark Batch Job, see Writing and reading data from MongoDB using a Spark Batch Job.
tDynamoDBConfiguration properties for Apache Spark Streaming
These properties are used to configure tDynamoDBConfiguration running in the Spark Streaming Job framework.
The Spark Streaming
tDynamoDBConfiguration component belongs to the Storage and the Databases families.
This component is available in Talend Real Time Big Data Platform and Talend Data Fabric.
Basic settings
Access key |
Enter the access key ID that uniquely identifies an AWS |
Secret key |
Enter the secret access key, constituting the security To enter the secret key, click the […] button next to |
Region |
Specify the AWS region by selecting a region name from the list. For more information |
Use End Point |
Select this check box and in the field displayed, specify the Web service URL of the |
Advanced settings
Connection pool |
In this area, you configure, for each Spark executor, the connection pool used to control
|
Evict connections |
Select this check box to define criteria to destroy connections in the connection pool. The
|
Usage
Usage rule |
This component is used with no need to be connected to other components. The configuration in a tDynamoDBConfiguration component This component, along with the Spark Streaming component Palette it belongs to, appears Note that in this documentation, unless otherwise explicitly stated, a scenario presents |
Related scenarios
For a scenario about how to use the same type of component in a Spark Streaming Job, see
Reading and writing data in MongoDB using a Spark Streaming Job.