August 16, 2023

tS3BucketExist properties – Docs for ESB 6.x

  2. Talend for ESB Docs
  3. Docs 6.x
  4. tS3BucketExist properties – Docs for ESB 6.x

tS3BucketExist properties

Verifies if the specified bucket exists on Amazon
S3.

“Bucket” is a term used by AWS for top level folders on S3,
which can contain sub folders and store all your data (objects).

tS3BucketExist Standard properties

These properties are used to configure tS3BucketExist running in the Standard Job framework.

The Standard
tS3BucketExist component belongs to the Cloud family.

The component in this framework is generally available.

Basic settings

Use existing connection

Select this check box and in the Component
List click the relevant connection component to reuse the connection
details you already defined.

Access Key

The Access Key ID that uniquely identifies an AWS Account. For how to
get your Access Key and Access Secret, visit Getting Your AWS Access Keys.

Secret Key

The Secret Access Key, constituting the security credentials in
combination with the access Key.

To enter the secret key, click the […] button next to
the secret key field, and then in the pop-up dialog box enter the password between double
quotes and click OK to save the settings.

Inherit credentials from AWS role

Select this check box to obtain AWS security credentials from Amazon EC2
instance metadata. To use this option, the Amazon EC2 instance must be started and
your Job must be running on Amazon EC2. For more information, see Using an IAM Role to Grant
Permissions to Applications Running on Amazon EC2 Instances.

Assume role

Select this check box and specify the values for the following parameters used to
create a new assumed role session.

  • Role ARN: the Amazon Resource Name (ARN) of the
    role to assume.

  • Role session name: an identifier for the
    assumed role session.

  • Session duration (minutes): the duration (in
    minutes) for which we want to have the assumed role session to be active.

For more information about assuming roles, see AssumeRole.

Region

Specify the AWS region by selecting a region name from the list or entering
a region between double quotation marks (e.g. “us-east-1”) in the
list. For more information about the AWS Region, see Regions and Endpoints.

Bucket

Name of the bucket, namely the top level folder, on the S3
server.

Die on error

This check box is cleared by default, meaning to skip the row on error
and to complete the process for error-free rows.

Advanced settings

Config client

Select this check box to configure client parameters.

Client parameter: select client
parameters from the list.

Value: enter the parameter
value.

Not available when Use existing
connection is selected.

STS Endpoint

Select this check box and in the field displayed, specify the AWS Security Token
Service endpoint where session credentials are retrieved from.

This check box is available only when the Assume
role check box is selected.

tStatCatcher Statistics

Select this check box to collect log data at the component
level.

Global Variables

Global Variables

BUCKET_EXIST: the existence of a specified bucket. This
is a Flow variable and it returns a boolean.

BUCKET_NAME: the name of a specified bucket. This is an
After variable and it returns a string.

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule

This component is usually used with other S3 components, e.g.
tS3BucketCreate.

Dynamic settings

Click the [+] button to add a
row in the table and fill the Code field
with a context variable to choose your database connection dynamically from
multiple connections planned in your Job. This feature is useful when you
need to access database tables having the same data structure but in
different databases, especially when you are working in an environment where
you cannot change your Job settings, for example, when your Job has to be
deployed and executed independent of
Talend Studio
.

Once a dynamic parameter is defined, the Component List box in the Basic
settings view becomes unusable.

For examples on using dynamic parameters, see Scenario: Reading data from databases through context-based dynamic connections and Scenario: Reading data from different MySQL databases using dynamically loaded connection parameters. For more information on Dynamic
settings and context variables, see
Talend Studio User Guide.

Limitation

Due to license incompatibility, one or more JARs required to use this component are not
provided. You can install the missing JARs for this particular component by clicking the
Install button on the Component tab view. You can also find out and add all missing JARs easily on the
Modules tab in the
Integration
 perspective of your
studio. You can find more details about how to install external modules in Talend Help Center (https://help.talend.com).

Scenario: Verifing the absence of a bucket, creating it and listing all the S3
buckets

In this scenario, tS3BucketExist is used to verify
the absence of a bucket, tS3BucketCreate to create that
bucket upon confirmation, and tS3BucketList to list all
the buckets on Amazon S3.

Linking the components

  1. Drop tS3Connection, tS3BucketExist, tS3BucketCreate, tS3BucketList, tIterateToFlow and tLogRow
    onto the workspace.
  2. Link tS3Connection to tS3BucketExist using the OnSubjobOk trigger.
  3. Link tS3BucketExist to tS3BucketCreate using the Run if trigger.
  4. Link tS3BucketCreate to tS3BucketList using the OnSubjobOk trigger.
  5. Link tS3BucketList to tIterateToFlow using the Row > Iterate connection.
  6. Link tIterateToFlow to tLogRow using the Row >
    Main     connection.

    use_case_ts3exist_1.png

Configuring the components

  1. Double-click tS3Connection to open its
    Basic settings view.

    use_case_ts3exist_2.png

  2. In the Access Key and Secret Key fields, enter the authentication
    credentials.
  3. Double-click tS3BucketExist to open its
    Basic settings view.

    use_case_ts3exist_3.png

  4. Select the Use existing connection check
    box to reuse the connection.
  5. In the Bucket field, enter the bucket
    name to check if it exists.
  6. Double-click the If link to define the
    condition.

    use_case_ts3exist_4.png

  7. In the Condition box, enter the
    expression:

    This way, the rest of the Job will be executed if the specified bucket
    does not exist.
  8. Double-click tS3BucketCreate to open its
    Basic settings view.

    use_case_ts3exist_5.png

    Select the Use existing connection check
    box to reuse the connection.
    In the Bucket field, enter the bucket
    name to create.
  9. Double-click tS3BucketList to open its
    Basic settings view.

    use_case_ts3exist_6.png

    Select the Use existing connection check
    box to reuse the connection.
  10. Double-click tIterateToFlow to open its
    Basic settings view.

    use_case_ts3exist_7.png

  11. Click Edit schema to open the schema
    editor.

    use_case_ts3exist_8.png

    Click the [+] button to add one column,
    namely bucket_list of the String
    type.
    Click Ok to validate the setup and close
    the schema editor.
  12. In the Mapping area, press Ctrl + Space in the Value field to choose the variable tS3BucketList_1_CURRENT_BUCKET_NAME.
  13. Double-click tLogRow to open its
    Basic settings view.

    use_case_ts3exist_9.png

    Select Table (print values in cells of a
    table)     for a better display of the results.

Executing the Job

  1. Press Ctrl + S to save the Job.
  2. Press F6 to run the Job.

    use_case_ts3exist_10.png

    As shown above, the bucket is created and all the buckets are
    listed.
  3. Go to the S3 web console:

    use_case_ts3exist_11.png

    As shown above, the bucket has been created on the S3 server.
  4. Refresh the S3 Browser client:

    use_case_ts3exist_12.png

    This shows that the S3 Create action was performed successfully.
Document get from Talend https://help.talend.com
Thank you for watching.
Docs 6.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x