tDynamoDBConfiguration
Stores connection information and credentials to be reused by other DynamoDB
components.
You define the connection to a DynamoDB database in tDynamoDBConfiguration and configure the
other DynamoDB components to reuse this configuration. At runtime, the
Spark executors read this configuration in order to connect to
DynamoDB.
Depending on the Talend solution you
are using, this component can be used in one, some or all of the following Job
frameworks:
-
Spark Batch: see tDynamoDBConfiguration properties for Apache Spark Batch.
The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data. -
Spark Streaming: see tDynamoDBConfiguration properties for Apache Spark Streaming.
The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.
tDynamoDBConfiguration properties for Apache Spark Batch
These properties are used to configure tDynamoDBConfiguration running in the Spark Batch Job framework.
The Spark Batch
tDynamoDBConfiguration component belongs to the Storage and the Databases families.
The component in this framework is available only if you have subscribed to one
of the
Talend
solutions with Big Data.
Basic settings
|
Access key |
Enter the access key ID that uniquely identifies an AWS Account. For |
|
Secret key |
Enter the secret access key, constituting the security credentials in To enter the secret key, click the […] button next to |
|
Region |
Specify the AWS region by selecting a region name from the list. For more information |
|
Use End Point |
Select this check box and in the field displayed, specify the Web service URL of the |
Usage
|
Usage rule |
This component is used with no need to be connected to other components. The configuration in a tDynamoDBConfiguration component This component, along with the Spark Batch component Palette it belongs to, appears only Note that in this documentation, unless otherwise |
Related scenarios
For a scenario about how to use the same type of component in a Spark Batch Job, see Writing and reading data from MongoDB using a Spark Batch Job.
tDynamoDBConfiguration properties for Apache Spark Streaming
These properties are used to configure tDynamoDBConfiguration running in the Spark Streaming Job framework.
The Spark Streaming
tDynamoDBConfiguration component belongs to the Storage and the Databases families.
The component in this framework is available only if you have subscribed to Talend Real-time Big Data Platform or Talend Data
Fabric.
Basic settings
|
Access key |
Enter the access key ID that uniquely identifies an AWS Account. For |
|
Secret key |
Enter the secret access key, constituting the security credentials in To enter the secret key, click the […] button next to |
|
Region |
Specify the AWS region by selecting a region name from the list. For more information |
|
Use End Point |
Select this check box and in the field displayed, specify the Web service URL of the |
Advanced settings
|
Connection pool |
In this area, you configure, for each Spark executor, the connection pool used to control
|
|
Evict connections |
Select this check box to define criteria to destroy connections in the connection pool. The
|
Usage
|
Usage rule |
This component is used with no need to be connected to other components. The configuration in a tDynamoDBConfiguration component This component, along with the Spark Streaming component Palette it belongs to, appears Note that in this documentation, unless otherwise explicitly stated, a scenario presents |
Related scenarios
For a scenario about how to use the same type of component in a Spark Streaming Job, see
Reading and writing data in MongoDB using a Spark Streaming Job.