July 30, 2023

tFTPFileList – Docs for ESB 7.x

tFTPFileList

Lists all files and folders directly under a specified directory based on a
filemask pattern.

tFTPFileList Standard properties

These properties are used to configure tFTPFileList running in the Standard Job framework.

The Standard
tFTPFileList component belongs to the Internet family.

The component in this framework is available in all Talend
products
.

Basic settings

Property Type

Select the way the connection details
will be set.

  • Built-In: The connection details will be set
    locally for this component. You need to specify the values for all
    related connection properties manually.

  • Repository: The connection details stored
    centrally in Repository > Metadata will be reused by this component. You need to click
    the […] button next to it and in the pop-up
    Repository Content dialog box, select the
    connection details to be reused, and all related connection
    properties will be automatically filled in.

Use an existing connection

Select this check box and in the Component List click the relevant connection component to
reuse the connection details you already defined.

Host

The IP address or hostname of the FTP server.

Port

The listening port number of the FTP server.

Username and Password

The user authentication data to access the FTP server.

To enter the password, click the […] button next to the
password field, and then in the pop-up dialog box enter the password between double quotes
and click OK to save the settings.

Remote directory

The remote directory where the files and folders to be listed are located.

Move to the current directory

Select this check box to change the directory to the one specified in the
Remote directory field. The next FTP component in the Job will
take this directory as the root of the remote directory when using the same
connection.

This property is available only when the Use an existing
connection
check box is selected.

File detail

Select this check box to list the details of each file/folder.
The informative details include the file/folder permissions, the name of the
author, the name of the group of users that have read/write permissions, the
file size, and the last modification date.

Files

The names of the files/folders to be listed. You can specify
multiple files/folders in a line by using wildcards or a regular
expression.

SFTP Support

Select this check box to connect to the FTP server via an SFTP connection.

Warning: This option does not work with an HTTP/HTTPS proxy. If you need a
proxy, set a SOCKS proxy in the Advanced settings tab.

Authentication method

Select the SFTP authentication method, either Public key or
Password.

  • Public key: Enter the path to the private key and the
    passphrase for the key in the Private key and Key
    Passphrase
    fields correspondingly.

  • Password: Enter the password required.

This property is available only when the
SFTP Support check box is selected.

Filename encoding

Select this check box to set the encoding used to convert file names from Strings to
bytes. It should be the same encoding used on the SFTP server. If the SFTP server’s
version is greater than 3, the encoding should be UTF-8, or else
an error occurs.

This property is available only when the
SFTP Support check box is selected.

FTPS Support

Select this check box to connect to the FTP server via an FTPS
connection.

If you are using an HTTP proxy, via a tSetProxy component for example,
you need to select this check box and set the connection mode to
Passive.

Keystore File

The path to your keystore file, a password protected file containing several keys and
certificates.

This property is available only when the
FTPS Support check box is selected.

Keystore Password

The password for your keystore file.

This property is available only when the
FTPS Support check box is selected.

Security Mode

Select the security mode from the list, either Implicit or
Explicit.

This property is available only when the
FTPS Support check box is selected.

Connection mode

Select the connection mode from the list, either Passive or
Active.

Encoding

Specify the encoding type by selecting an encoding type from the list or selecting
CUSTOM and enter the encoding type manually.

Advanced settings

Use Socks Proxy

Select this check box if you are using a proxy, and in the Proxy
host
, Proxy port, Proxy user
and Proxy password fields displayed, specify the proxy server
settings information.

Ignore Failure At Quit (FTP)

Select this check box to ignore library closing errors or FTP closing errors.

Data Channel Protection Level

The data channel protection level with which data is transferred between the client and
the server. For more information, see RFC
2228: FTP Security Extensions
.

This property is available only
when the FTPS Support check box is selected.

Protection Buffer Size

The maximum size (in bytes) of the encoded data blocks to be transferred between the
client and the server. For more information, see RFC
2228: FTP Security Extensions
.

This property is available only
when the FTPS Support check box is selected.

Connection timeout

Specify the timeout value (in ms) for the connection. A value of 0
or any negative values will be ignored. In this case, the default value (that is, 60000ms)
will be used.

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level
as well as at each component level.

Global Variables

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After
variable and it returns a string.

CURRENT_FILE

The current file name. This is a Flow variable and it returns a string.

CURRENT_FILEPATH

The current file path. This is a Flow variable and it returns a string.

NB_FILE

The number of the files processed. This is an After variable and it returns an integer.

Usage

Usage rule

This component is typically used as a single-component subJob
but can also be used with other components.

Listing and getting files/folders on an FTP directory

Here is an example of using Talend FTP components to iterate and list
all files and folders on an FTP server directory, and then get only text files on that
directory to a local directory.

Creating a Job for listing and getting files/folders on an FTP directory

Create a Job to connect to an FTP server, iterate and list all files
and folders on an FTP root directory, then get only text files on the FTP root directory to
a local directory, finally close the connection to the server.

tFTPFileList_1.png

Prerequisites: To replicate this
scenario, an FTP server must be started and a couple of files/folders must be put onto
the root directory of the FTP server.

tFTPFileList_2.png

  1. Create a new Job and add a tFTPConnection component, a tFTPFileList component, a tIterateToFlow component, a tLogRow
    component, a tFTPGet component, and a tFTPClose component by typing their names in the design
    workspace or dropping them from the Palette.
  2. Link the tFTPFileList component to the
    tIterateToFlow component using a Row > Iterate
    connection.
  3. Link the tIterateToFlow component to
    the tLogRow component using a Row > Main connection.
  4. Link the tFTPConnection component to
    the tFTPFileList component using a Trigger > OnSubjobOk
    connection.
  5. Do the same to link the tFTPFileList
    component to the tFTPGet component, and the
    tFTPGet component to the tFTPClose component.

Opening a connection to the FTP server

Configure the tFTPConnection component to open a connection to the FTP
server.

  1. Double-click the tFTPConnection
    component to open its Basic settings view.

  2. In the Host and Port fields, enter the FTP server IP address and the listening port
    number respectively.
  3. In the Username and Password fields, enter the authentication details.

Listing all files/folders on the FTP root directory

Configure the tFTPFileList
component, the tIterateToFlow component, and the
tLogRow component to iterate all files and folders on
the FTP root directory and display the names and paths of these files and folders on the
console of
Talend Studio
.

  1. Double-click the tFTPFileList
    component to open its Basic settings view.

    tFTPFileList_3.png

  2. Specify the connection details required to access the FTP server. In
    this example, select the Use an existing connection
    check box and from the Component list drop-down
    list displayed, select the connection component to reuse its connection details you
    have already defined.
  3. In the Remote directory field, specify
    the FTP server directory on which the files and folders will be iterated. In this
    example, it is /, which means the root directory
    of the FTP server.
  4. Clear the Move to the current
    directory
    check box.
  5. Double-click the tIterateToFlow
    component to open its Basic settings view.

    tFTPFileList_4.png

  6. Click the

    tFTPFileList_5.png

    button next to Edit schema to
    open the schema dialog box.

    tFTPFileList_6.png

  7. Click the

    tFTPFileList_7.png

    button to add two String type columns
    filename and filepath
    that will hold the names and paths of the files to be iterated respectively. When
    done, click OK to close the dialog box.

  8. In the Mapping table, set the values
    for the filename and filepath columns. In this example, the global variable ((String)globalMap.get(“tFTPFileList_1_CURRENT_FILE”))
    for filename and the global variable ((String)globalMap.get(“tFTPFileList_1_CURRENT_FILEPATH”)) for
    filepath.

    Note that you can fill the values by pressing Ctrl + Space to access the global variables list and then selecting
    tFTPFileList_1_CURRENT_FILE and
    tFTPFileList_1_CURRENT_FILEPATH from the list.

  9. Double-click the tLogRow component to
    open its Basic settings view, and then select
    Table (print values in cells of a table) in the
    Mode area for better readability of the
    result.

    tFTPFileList_8.png

Getting files on the FTP server directory to a local directory

Configure the tFTPGet
component to get only the text files on the FTP root directory to a local
directory.

  1. Double-click the tFTPGet component to
    open its Basic settings view.

    tFTPFileList_9.png

  2. Specify the connection details required to access the FTP server. In
    this example, select the Use an existing connection
    check box and from the Component list drop-down
    list displayed, select the connection component to reuse its connection details you
    have already defined.
  3. In the Local directory field, specify
    the local directory to which the files and folders will be downloaded. In this
    example, it is D:/FtpDownloads.
  4. In the Remote directory field, specify
    the FTP server directory under which the files and folders will be downloaded. In
    this example, it is /, which means the root
    directory of the FTP server.
  5. In the Files table, click the
    [+] button to add a line and in the Filemask column field, enter *.txt between double quotation marks to get only the text files on
    the FTP directory to the local directory.

Closing the connection to the FTP server

Configure the tFTPClose
component to close the connection to the FTP server.

  1. Double-click the tFTPClose component
    to open its Basic settings view.

    tFTPFileList_10.png

  2. From the Component list drop-down
    list, select the tFTPConnection component that
    opens the connection you need to close. In this example, only one tFTPConnection component is used and it is selected by
    default.

Executing the Job to list and get files/folders on the FTP directory

After setting up the Job and configuring the components used in
the Job for listing and getting files/folders on the FTP directory, you can then execute
the Job and verify the Job execution result.

  1. Press Ctrl + S to save the Job.
  2. Press F6 to execute the Job.

    tFTPFileList_11.png

    As shown above, the names and paths of the files and folders on the FTP server
    root directory are displayed on the console, and only the text files are downloaded
    to the specified local directory.

Document get from Talend https://help.talend.com
Thank you for watching.
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x