tS3Put

Uploads data onto Amazon S3 from a local file or from cache memory via the
streaming mode.

tS3Put Standard properties

These properties are used to configure tS3Put running in the Standard Job framework.

The Standard
tS3Put component belongs to the Cloud family.

The component in this framework is available in all Talend
products.

Basic settings

Use existing connection	Select this check box and in the Component List click the relevant connection component to reuse the connection details you already defined.
Access Key	The Access Key ID that uniquely identifies an AWS Account. For how to get your Access Key and Access Secret, visit Getting Your AWS Access Keys.
Secret Key	The Secret Access Key, constituting the security credentials in combination with the access Key. To enter the secret key, click the […] button next to the secret key field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.
Inherit credentials from AWS role	Select this check box to obtain AWS security credentials from Amazon EC2 instance metadata. To use this option, the Amazon EC2 instance must be started and your Job must be running on Amazon EC2. For more information, see Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances.
Assume role	If you temporarily need some access permissions associated to an AWS IAM role that is not granted to your user account, select this check box to assume that role. Then specify the values for the following parameters to create a new assumed role session.
Region	Specify the AWS region by selecting a region name from the list or entering a region between double quotation marks (e.g. “us-east-1”) in the list. For more information about the AWS Region, see Regions and Endpoints.
Encrypt	Select this check box and from the Key type drop-down list displayed, select one of the following three options for encrypting the data on the client-side before sending to Amazon S3. For more information, see Protecting Data Using Client-Side Encryption. KMS-managed customer master key: use a KMS-managed customer master key (CMK) for the client-side data encryption. In the Key field, you need to specify the AWS KMS customer master key ID (CMK ID). Symmetric Master Key: use a symmetric master key (256-bit AES secret key) for the client-side data encryption. Algorithm: select the algorithm associated with the key from the list. By default, there is only one algorithm named AES. Encoding: select the encoding type associated with the key from the list, either Base64 or X509. Key or Key file: specify the key or the path to the file that stores the key. Asymmetric Master Key: use an asymmetric master key (a 1024-bit RSA key pair) for the client-side data encryption. Algorithm: select the algorithm associated with the key from the list. By default, there is only one algorithm named RSA. Public key file: specify the path to the public key file. Private key file: specify the path to the private key file.
Bucket	Specify the name of the bucket, namely the top level folder, on the S3 server.
Key	Specify the path to the file that will hold the uploaded data on the S3 server, using slash (/) to separate folders in the path. If: you only specify a file name in this field, the file will be in the bucket; the path contains folders that do not exist, the folders will be created; the file already exists, it will be overwritten. Warning: This field must not start with a slash (/).
File or Stream	The path to the local file to be uploaded or the input stream from which data will be uploaded. For an example of how to use stream, see Reading data from a remote file in streaming mode.
Server-Side Encryption	Select this check box to enable server-side encryption with Amazon S3-Managed Encryption Keys (SSE-S3) and use the 256-bit Advanced Encryption Standard (AES-256) cipher to encrypt your data. For more information about the server-side encryption, see Protecting Data Using Server-Side Encryption.
Use KMS	Select this check box to enable server-side encryption with AWS KMS-Managed Keys (SSE-KMS) instead of Amazon S3-Managed Encryption Keys (SSE-S3). This property is available when the Server-Side Encryption check box is selected.
Customer Master Key	Specify your own customer master key (CMK) that is created in the IAM console using AWS Key Management Service for SSE-KMS encryption. If not specified, the default CMK, which is created the first time you add an SSE-KMS encrypted object to the defined bucket in your region, will be used for SSE-KMS encryption. This property is available when the Use KMS check box is selected.
Die on error	This check box is cleared by default, meaning to skip the row on error and to complete the process for error-free rows.

Advanced settings

Config client	Select this check box to configure client parameters. Client parameter: select client parameters from the list. Value: enter the parameter value. Not available when Use existing connection is selected.
Enable Accelerate Mode	Select this check box to enable fast, easy and secure transfers of files over long distances between your client and an S3 bucket. To take it into account, you should enable this acceleration mode on the S3 bucket in advance.
STS Endpoint	Select this check box and in the field displayed, specify the AWS Security Token Service endpoint, for example, `sts.amazonaws.com`, where session credentials are retrieved from. This check box is available only when the Assume role check box is selected.
Part size (Mb)	Specify the part size (in MB) when uploading an object in parts. Part size is in the range 5MB to 5GB. A value less than `5` uploads the object with the part size of 5MB. For an object that is less than 5GB in size, you can have it uploaded without being divided into parts by setting this option to a large enough value. For information about S3 multipart upload, see Multipart Upload Overview.
Access control	Select this check box to set the access control list (ACL) for the current object. With Access control selected, click the plus button under the Access table to add a new line, where you can add an AWS user account and set permissions for it as follows. Enter the canonical user ID or the Email address of the user in double quotation marks in the Account canonical ID / email column; Select the check box in the Read object column to grant the permission to read the data and metadata of the object in the bucket; Select the check box in the Read object permissions column to grant the permission to read the ACLs of the object in the bucket; Select the check box in the Write object permissions column to grant the permission to write the ACLs of the object in the bucket. Note: The string in the column is considered as an Email address if it contains @. If Access control is not selected, the user who uploads an object becomes the owner of the object and owns all the permissions over the object. If Access control is selected, the user who uploads an object becomes the owner of the object but owns no permission over the object. You can grant permissions to the owner in the Access table in the way mentioned above. AWS group is not supported.
tStatCatcher Statistics	Select this check box to collect log data at the component level.

Global Variables

ERROR_MESSAGE	The error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is selected.

Usage

Usage rule	This component can be used alone or with other S3 components, e.g. tS3Connection.
Dynamic settings	Click the [+] button to add a row in the table and fill the Code field with a context variable to choose your database connection dynamically from multiple connections planned in your Job. This feature is useful when you need to access database tables having the same data structure but in different databases, especially when you are working in an environment where you cannot change your Job settings, for example, when your Job has to be deployed and executed independent of Talend Studio. Once a dynamic parameter is defined, the Component List box in the Basic settings view becomes unusable. For examples on using dynamic parameters, see Reading data from databases through context-based dynamic connections and Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic settings and context variables, see Talend Studio User Guide.

Usage rule

This component can be used alone or with other S3 components,
e.g. tS3Connection.

Dynamic settings

Click the [+] button to add a row in the table
and fill the Code field with a context
variable to choose your database connection dynamically from multiple
connections planned in your Job. This feature is useful when you need to
access database tables having the same data structure but in different
databases, especially when you are working in an environment where you
cannot change your Job settings, for example, when your Job has to be
deployed and executed independent of Talend Studio.

Once a dynamic parameter is defined, the Component List box in the Basic
settings view becomes unusable.

For examples on using dynamic parameters, see Reading data from databases through context-based dynamic connections and Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic
settings and context variables, see Talend Studio
User Guide.

Exchange files with Amazon S3

In this scenario, tS3Put is used to send a
file to the Amazon S3 server while tS3Get is used to get
that file.

Linking the components

Drop tS3Connection, tS3Put and tS3Get onto the
workspace.
Link tS3Connection to tS3Put using the OnSubjobOk trigger.
Link tS3Put to tS3Get using the OnSubjobOk
trigger.

Configuring the components

Double-click tS3Connection to open its
Basic settings view.
In the Access Key and Secret Key fields, enter the authentication
credentials.
Double-click tS3Put to open its Basic settings view.
Select the Use existing connection check
box to reuse the connection.
In the Bucket field, enter the bucket
name.
In the Key field, enter the name of the
file to be saved on the S3 server.
In the File field, enter the file to be
uploaded to the S3 server.
Double-click tS3Get to open its Basic settings view.
Select the Use existing connection check
box to reuse the connection.
In the Bucket field, enter the bucket
name.
In the Key field, enter the name of the
file to be retrieved from the S3 server.
In the File field, enter the local name
of the retrieved file.

Executing the Job

Press Ctrl + S to save the Job.
Press F6 to run the Job.
Go to the S3 web console:

As shown above, the file has been put on to the S3 server.
Refresh the S3 Browser client:

This shows that the S3 Put action was performed successfully.
Go to the folder where the retrieved file is saved locally:

As shown above, the remote file is retrieved to the local disk, proof that
the S3 Get action was performed successfully.

Document get from Talend https://help.talend.com

Thank you for watching.

Docs 7.x

0 Comments

Inline Feedbacks

View all comments

tS3Put – Docs for ESB 7.x