August 16, 2023

tSQSInput – Docs for ESB 6.x

  2. Talend for ESB Docs
  3. Docs 6.x
  4. tSQSInput – Docs for ESB 6.x

tSQSInput

Retrieves one or more messages, with a maximum limit of ten
messages, from an Amazon SQS (Simple Queue Service) queue.

tSQSInput Standard properties

These properties are used to configure tSQSInput running in the Standard Job framework.

The Standard tSQSInput component belongs to
the Cloud family.

The component in this framework is generally available.

Basic settings

Use an existing connection

Select this check box and in the Component
List click the relevant connection component to reuse the connection
details you already defined.
Access Key and Secret
Key

Specify the access keys (the access key ID in the Access
Key field and the secret access key in the Secret
Key field) required to access the Amazon Web Services. For more information
about AWS access keys, see Access keys (access key ID and secret access
key).

To enter the secret key, click the […] button next to
the secret key field, and then in the pop-up dialog box enter the password between double
quotes and click OK to save the settings.
Inherit credentials from AWS role

Select this check box to leverage the instance profile credentials. These
credentials can be used on Amazon EC2 instances, and are delivered through the Amazon EC2
metadata service. To use this option, your Job must be running within Amazon EC2 or other
services that can leverage IAM Roles for access to resources. For more information, see Using an IAM Role to Grant Permissions to
Applications Running on Amazon EC2 Instances.
Assume role
Select this check box and specify the values for the following parameters used to
create a new assumed role session.

  • Role ARN: the Amazon Resource Name (ARN) of the
    role to assume.

  • Role session name: an identifier for the
    assumed role session.

  • Session duration (minutes): the duration (in
    minutes) for which we want to have the assumed role session to be active.

For more information about assuming roles, see AssumeRole.
Region

Specify the AWS region by selecting a region name from the list or entering a
region between double quotation marks (for example “us-east-1”) in the
list. For more information about AWS Regions, see AWS Regions and Endpoints.
Queue (Name or URL)

Specify the name or URL of the queue from which the messages will be
retrieved.
Schema and Edit
schema

A schema is a row description. It defines the number of fields (columns) to
be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

The schema of this component is predefined with the
following four String type message attributes:

  • MessageId: the ID of the
    message.

  • ReceiptHandle: the receipt
    handle of the message.

  • MD5OfBody: the MD5 digest
    of the message body.

  • Body: the message body.

You can click the components-button_three_dot.png button next to Edit
schema to view the schema and edit the schema by
removing the existing attributes or adding the standard SQS attributes
and the customized user attributes.
Read standard attributes

Select this check box to retrieve the standard Amazon SQS
attributes along with each message. For more information about the
Amazon SQS attributes, see ReceiveMessage >
Request Parameters.
Read custom user attributes

Select this check box to retrieve the customized user
attributes along with each message.
Custom visibility
timeout

Select this check box and in the field displayed specify
the visibility timeout (in seconds) for the received messages. If not
specified, the overall visibility timeout for the queue will be used for
the received messages. For more information, see Visibility
Timeout.
Custom wait time

Select this check box and in the field displayed specify
the duration (in seconds) for which the call will wait for the message
to arrive in the queue before returning.
Delete the messages while
streaming

Select this check box to delete the message while
retrieving it from the queue.
Read all messages from the
queue

Select this check box to retrieve all messages from the
queue.

This check box is not available when the Delete the messages while streaming check
box is cleared.
Max number of message to return per
request

Specify the maximum number of messages to return per each request. Valid
values are 1 to 10.
Die on error

Select the check box to stop the execution of the Job when an error
occurs.

Advanced settings

Config client
Select this check box and in the table displayed specify the client configuration
parameters. Click the [+] button below the table to add as many rows as
needed, each row for a client configuration parameter, and set the value of the following
attributes for each parameter:

  • Client Parameter: click the cell and select a
    parameter from the drop-down list.

  • Value: enter the value for the corresponding
    parameter.
STS Endpoint

Select this check box and in the field displayed, specify the AWS Security Token
Service endpoint where session credentials are retrieved from.

This check box is available only when the Assume
role check box is selected.
tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level
as well as at each component level.

Global variables

NB_LINE

The number of messages processed. This is an After variable and it returns an
integer.
ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After
variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

Usage

Usage Rule

This component is usually used as a start component of a Job or Subjob and it
always needs an output link.

Retrieving messages from an Amazon SQS queue

Here’s an example of using Talend components to retrieve all messages
from an existing Amazon SQS (Simple Queue Service) queue and then delete the empty
queue.

Creating a Job for retrieving Amazon SQS messages

Create a Job to open a connection to access Amazon SQS, then retrieve
all messages from an existing Amazon SQS queue, finally delete the queue.

amazon_sqs_job_retrieve_message.png

Prerequisites: You must deliver the following message
creator attribute and message body data to the Amazon SQS queue named
talend in order to replicate this scenario.

  1. Create a new Job and add a tSQSConnection component, a tSQSInput component, a tLogRow
    component, and a tSQSQueueDelete component by
    typing their names in the design workspace or dropping them from the Palette.
  2. Link the tSQSInput component to the
    tLogRow component using a Row > Main connection.
  3. Link the tSQSConnection component to
    the tSQSInput component using a Trigger > OnSubjobOk
    connection.
  4. Link the tSQSInput component to the
    tSQSQueueDelete component using a Trigger > OnSubjobOk
    connection.

Opening a connection to access Amazon SQS

Configure the tSQSConnection
component to open a connection to access Amazon SQS.

  1. Double-click the tSQSConnection
    component to open its Basic settings view.

    amazon_sqs_tsqsconnection_basic.png

  2. In the Access Key and Secret Key fields, specify the authentication information
    required to access Amazon SQS.
  3. Select an AWS region available for Amazon SQS from the Region drop-down list. In this example, it is
    Asia Pacific (Tokyo).

Retrieving messages from an Amazon SQS queue

Configure the tSQSInput component
and the tLogRow component to retrieve all messages from
an existing Amazon SQS queue and display the retrieved messages on the console of

Talend Studio
.

  1. Double-click the tSQSInput component
    to open its Basic settings view.

    amazon_sqs_tsqsinput_basic.png

  2. Specify the connection details required to access Amazon SQS. In this
    example, select the Use an existing connection
    check box and from the Component List drop-down
    list displayed, select the connection component to reuse its connection details you
    have already defined.
  3. In the Queue URL field, enter the name
    for the queue from which the messages will be retrieved. In this example, it is
    talend.
  4. Click the

    components-button_three_dot.png

    button next to Edit schema to
    open the schema dialog box.

    amazon_sqs_tsqsinput_schema.png

    You can see the schema of the tSQSInput component
    is predefined with four columns of String type: MessageId,
    ReceiptHandle, MD5OfBody and
    Body.
  5. Click the

    plus_button.png

    button to add five columns of String type, including four
    Amazon SQS attributes ApproximateFirstReceiveTimestamp,
    ApproximateReceiveCount, SenderId and
    SentTimestamp and one customized attribute
    Creator.

  6. Click OK to save the changes and close
    the schema dialog box.
  7. Select the Read standard attributes and Read custom user attributes check boxes to retrieve the
    value of four Amazon SQS attributes and one customized user attribute along with
    each message.
  8. Double-click the tLogRow component to
    open its Basic settings view, and then select
    Table (print values in cells of a table) in the
    Mode area for better readability of the
    result.

    amazon_sqs_tlogrow_basic.png

Deleting an Amazon SQS queue

Configure the tSQSQueueDelete
component to create an Amazon SQS queue.

  1. Double-click the tSQSQueueDelete
    component to open its Basic settings view.

    amazon_sqs_tsqsqueuedelete_basic.png

  2. Specify the connection details required to access Amazon SQS. In this
    example, select the Use an existing connection
    check box and from the Component List drop-down
    list displayed, select the connection component to reuse its connection details you
    have already defined.
  3. In the Queue field, enter the name for
    the queue to be deleted. In this example, it is talend.

Executing the Job to retrieve Amazon SQS messages

After setting up the Job and configuring the components used in the
Job for retrieving Amazon SQS messages, you can then execute the Job and verify the Job
execution result.

  1. Press Ctrl + S to save the Job and
    then F6 to execute the Job.

    amazon_sqs_job_retrieve_message_result.png

    As shown above, all three messages and their attributes are retrieved and
    displayed on the console.
  2. View the queue details on the AWS console to validate the Job execution
    result.

    amazon_sqs_job_retrieve_message_result_amazon.png

    You can see the talend queue has already been deleted and is
    not on the AWS console any more.
Document get from Talend https://help.talend.com
Thank you for watching.
Docs 6.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x