
Component family |
Cloud/AmazonS3 |
|
Function |
Lists the files on Amazon S3 based on the bucket/file prefix |
|
Purpose |
tS3List is designed to list the |
|
Basic settings |
Use existing connection |
Select this check box and in the Component List click the |
Access Key |
The Access Key ID that uniquely identifies an AWS Account. For how |
|
|
Access Secret |
The Secret Access Key, constituting the security credentials in To enter the secret key, click the […] button next to |
|
Region |
Specify the AWS region by selecting a region name from the list or entering a region |
|
List all bucket objects |
Select this check box to list all the files on the S3 Key prefix: enter the prefix of |
|
Bucket |
Click the [+] button to add one Bucket name: name of the bucket Key prefix: prefix of files to be Not available when List all bucket |
|
Die on error |
This check box is cleared by default, meaning to skip the row on |
Advanced settings |
Config client |
Select this check box to configure client parameters. Client parameter: select client Value: enter the parameter Not available when Use existing |
tStatCatcher Statistics |
Select this check box to collect log data at the component |
|
Dynamic settings |
Click the [+] button to add a row in the table and fill Once a dynamic parameter is defined, the Component List For more information on Dynamic settings and context |
|
Global Variables |
CURRENT_BUCKET: the current bucket name. This is a Flow CURRENT_KEY: the current key. This is a Flow variable and NB_BUCKET: the number of buckets. This is an After NB_BUCKET_OBJECT: the number of objects in all the ERROR_MESSAGE: the error message generated by the A Flow variable functions during the execution of a component while an After variable To fill up a field or expression with a variable, press Ctrl + For further information about variables, see Talend Studio |
|
Usage |
This component can be used alone or with other S3 components, e.g. |
|
Log4j |
The activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html. |
|
Limitation |
Due to license incompatibility, one or more JARs required to use this component are not |
In this scenario, tS3List is used to list all the
files in a bucket which have the same prefix.
There are such files in this bucket:

For how to create a bucket and put files into it, see Scenario: Verifing the absence of a bucket, creating it and listing all the S3
buckets and Scenario: File exchanges with Amazon S3 .
-
Drop tS3Connection, tS3List, tIterateToFlow,
tLogRow and tS3Close onto the workspace. -
Link tS3Connection to tS3List using the OnSubjobOk trigger.
-
Link tS3List to tIterateToFlow using the Row >
Iterate connection. -
Link tIterateToFlow to tLogRow using the Row >
Main connection. -
Link tS3List to tS3Close using the OnSubjobOk trigger.
-
Double-click tS3Connection to open its
Basic settings view. -
In the Access Key and Secret Key fields, enter the authentication
credentials. -
Double-click tS3List to open its
Basic settings view. -
Select the Use existing connection check
box to reuse the connection. -
In the Bucket area, click the [+] button to add one line.
-
In the Bucket name and Key prefix fields, enter the bucket name and file
prefix.This way, only files with the specified prefix will be listed.
-
Double-click tIterateToFlow to open its
Basic settings view. -
Click Edit schema to open the schema
editor.Click the [+] button to add one column,
namely file_list of the String
type.Click Ok to validate the setup and close
the schema editor. -
In the Mapping area, press Ctrl + Space in the Value field to choose the variable tS3List_1_CURRENT_KEY.
-
Double-click tLogRow to open its
Basic settings view.Select Table (print values in cells of a
table) for a better display of the results. -
Double-click tS3Close to open its
Basic settings view.There is no need to select a connection component as the only one is
selected by default.