July 30, 2023

cAWSS3 – Docs for ESB ESB 7.x

  2. Talend for ESB Docs
  3. Docs 7.x
  4. cAWSS3 – Docs for ESB ESB 7.x

cAWSS3

Stores and retrieves objects from/to Amazon’s Simple Storage Service
(S3)

cAWSS3 Standard properties

These properties are used to configure cAWSS3 running in the Standard Job framework.

The Standard
cAWSS3 component belongs to the AWS and Connectivity families.

Basic settings

Connection

Select an AWSS3 connection component from the list to reuse the
connection details you already defined.

Bucket Name

Specify the name of the bucket, namely the top level folder, on
Amazon’s S3 service.

The following options are available only when the cAWSS3 is used as a Producer:

  

Storage Class

Select from Standard, Standard – Infrequent Access and
Reduced Redundancy. For more
information about storage classes, see the site http://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html.

Delete After Write

Select this check box to delete the local file object after it is
uploaded to the S3 service.

Multi Part Upload

Select this check box to upload the file with multi part format,
and specify part size in the corresponding field. The default size
is 25M.

Server Side Encryption

Select this option to encrypt the object on the server side using
the AWS-managed keys.

Message Headers

Set the message headers in the corresponding table. Click
[+] as many times as required
to add message headers to the table. Then click the corresponding
Value field and enter a value.
See the site http://camel.apache.org/aws-s3.html for available
headers.

Be sure to set the CamelAwsS3Key header which will be
used as the name of the S3 file.

Use User Defined Headers

Select this check box to set user defined headers in the
corresponding table. Click [+] as
many times as required to add message headers to the table. Then
click the corresponding Value field
and enter a value. Note that user defined headers must start with
x-amz-meta-.

The following options are available only when the cAWSS3 is used as a Consumer:

  

File Name

Specify the name of the file to be consumed.

Max Messages Per Poll

The maximum number of objects that can be retrieved in one
poll.

Prefix

Specify the prefix of files so that only files with that prefix
will be consumed.

Delete After Read

Select this check box to delete the object from the S3 service
after it has been retrieved.

Include Body

Select this check box to include the exchange body in the content
of the file. Otherwise only the headers will be set with the S3
object metadata and the body will be null.

Advanced settings

Advanced

Set the optional parameters in the corresponding table. Click
[+] as many times as required
to add parameters to the table. Then click the corresponding
Value field and enter a value.
See the site http://camel.apache.org/aws-s3.html for available
options.

Usage

Usage rule

cAWSS3 can be a start, middle or
end component in a Route. It has to be used with the cAWSConnection component, which creates a
connection to Amazon Web Services. For more information about
cAWSConnection, see cAWSConnection.

Limitation

 n/a

Sending messages to and receiving messages from Amazon’s S3 service

This scenario applies only to Talend Open Studio for ESB, Talend Data Services Platform and Talend Data Fabric.

This scenario will show you how to use the cAWSS3 component
to send messages to and consume messages from Amazon’s S3 service. To do this, two
Routes are built, a message producer Route, and a consumer Route. Messages are sent to
the Amazon’s S3 service in the producer Route and then consumed in the consumer
Route.

Building the producer Route

Dropping and linking the components

cAWSS3_1.png

  1. From the Palette, drag and drop a cAWSConnection, a cFile,
    and a cAWSS3 component onto the design
    workspace.
  2. Label the components for better identification of their roles and link
    them with the Row > Route connection as shown above.

Configuring the components

  1. Double-click the cAWSConnection component to display its
    Basic settings view in the Component tab.

    cAWSS3_2.png

  2. In the Access Key and Secret
    Key     fields, enter the authentication credentials of your AWS
    account. For how to get your Access Key and Access Secret, see Access keys (access key ID
    and secret access key)    .
  3. Double-click the cFile component to open its Basic settings view in the Component tab.

    cAWSS3_3.png

  4. In the Path field, browse to or enter the input file
    path.

    In the fileName field, enter the name of file to be
    uploaded to the S3 service.
    In this scenario, a TXT file with the name talend
    that contains a simple string Hello World! is
    used.
  5. Double-click the cAWSS3 component to open its Basic settings view in the Component tab.

    cAWSS3_4.png

  6. In the Connection list, select the cAWSConnection component that you have just configured to
    connect to Amazon’s S3 service.

    In the Bucket Name field, enter the name
    of the bucket to upload the file, "talend-s3-demo" in this use
    case.
    Under the Message Headers table, click
    [+] to add two rows in the table. Set
    the header CamelAwsS3Key with the value
    “talend.txt” as the name of the S3 object, and the
    header CamelAwsS3ContentLength with the
    value 5 to define the length of the object.
  7. Press Ctrl+S to save your Route.

Viewing the code and executing the Route

  1. Click the Code tab at the bottom of the
    design workspace to check the generated code.

    cAWSS3_5.png

    As shown above, the message is routed from s3producer_cFile_1 to
    s3producer_cAWSS3_1.
  2. Press F6 to execute the Route. The logs
    of the message exchange are printed in the console.

    cAWSS3_6.png

  3. In the Amazon’s S3 Web Console, you can see that the object
    talend.txt has been created.

    cAWSS3_7.png

Building the consumer Route

Arranging the flow of the message

cAWSS3_8.png

  1. From the Palette, drag and drop a cAWSConnection, a cAWSS3
    and a cProcessor component onto the design
    workspace.
  2. Label the components for better identification of their roles and link
    them with the Row > Route connection as shown above.

Configuring how the message is processed

  1. Configure the cAWSConnection using the same properties
    as in the producer Route.
  2. Double-click the cAWSS3 component to display its
    Basic settings view in the Component tab.

    cAWSS3_9.png

  3. In the Connection list, select the cAWSConnection component to connect to Amazon’s S3 service.

    In the Bucket Name field, enter the name
    of the bucket that contains the file to be consumed,
    "talend-s3-demo" in this use case.
    In the File Name field, enter the name of the file,
    “talend.txt”.
    Clear the Delete After Read check box to
    keep the S3 object file after it is consumed.
  4. Double-click the cProcessor component to
    display its Basic settings view in the
    Component tab.

    cAWSS3_10.png

  5. In the Code box, enter the following code to print the
    file name and its content in the execution
    console:BufferedReader br = new BufferedReader(new InputStreamReader((InputStream) exchange.getIn().getBody()));
    System.out.println("FileName: "+exchange.getIn().getHeader("CamelAwsS3Key")+" Content: " + br.readLine());
    br.close();
  6. Press Ctrl+S to save your Route.

Executing the Route

  1. Click the Code tab at the bottom of the
    design workspace to check the generated code.

    cAWSS3_11.png

    As shown above, the message flow is routed from s3consumer_cAWSS3_1 and
    processed by s3consumer_cProcessor_1.
  2. Press F6 to execute the Route. The logs
    of the message exchange are printed in the console.

    The content of the file is Hello as the length of the object
    is defined to 5 when uploading the file to the S3
    service.
    cAWSS3_12.png

Document get from Talend https://help.talend.com
Thank you for watching.
Docs 7.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x