August 15, 2023

tCouchDBOutput – Docs for ESB 6.x

  2. Talend for ESB Docs
  3. Docs 6.x
  4. tCouchDBOutput – Docs for ESB 6.x

tCouchDBOutput

Loads JSON documents, writes data into or remove data from them and then saves the
documents back to the database on a CouchDB server.

tCouchDBOutput receives data from
the preceding component, and writes data into CouchDB.

tCouchDBOutput Standard properties

These properties are used to configure tCouchDBOutput running in the Standard Job framework.

The Standard
tCouchDBOutput component belongs to the Big Data and the Databases families.

The component in this framework is available when you are using one of the Talend solutions with Big Data.

Basic settings

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to
be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Click Edit schema to make changes to the schema.
If the current schema is of the Repository type, three
options are available:

  • View schema: choose this option to view the
    schema only.

  • Change to built-in property: choose this
    option to change the schema to Built-in for
    local changes.

  • Update repository connection: choose this
    option to change the schema stored in the repository and decide whether to propagate
    the changes to all the Jobs upon completion. If you just want to propagate the
    changes to the current Job, you can select No
    upon completion and choose this schema metadata again in the [Repository Content] window.

 

Built-In: You create and store the
schema locally for this component only. Related topic: see
Talend Studio

User Guide.

 

Repository: You have already created
the schema and stored it in the Repository. You can reuse it in various projects and
Job designs. Related topic: see
Talend Studio

User Guide.

When the schema to be reused has default values that are integers or
functions, ensure that these default values are not enclosed within quotation marks. If
they are, you must remove the quotation marks manually.

You can find more details about how to verify default
values in retrieved schema in Talend Help Center (https://help.talend.com).

Sync columns

Click this button to retrieve schema from the previous component
connected in the Job.

Use existing connection

Select this check box and in the Component
List click the relevant connection component to reuse the connection
details you already defined.
DB Version

Select the CouchDB version that you are using.

Server

Hostname or IP address of the CouchDB server.

Port

Listening port number of the CouchDB server.

Database

Name of the database that you want to use.

Required authentication

Select this check box to provide credentials for the CouchDB
authentication.

  • Username: Type in the
    user name for the CouchDB authentication.

  • Password: Type in the
    password for the CouchDB authentication.

To enter the password, click the […] button next to the
password field, and then in the pop-up dialog box enter the password between double quotes
and click OK to save the settings.

Use trigger replication

Select this check box to trigger replication between
databases.

Warning:

This check box appears only if you do not select the Use an existing connection check
box.

Target DB name

Specify the target database to which the documents will be copied.
It can be a local database name or a remote database URL.

Continuous

Select this check box to continue a replication upon the server
restart.

Create target DB

Select this check box to create the target database if it does not
exist.

IS Canceled

Select this check box to cancel the existing replication between
the specified source database and target database at the end of the
Job.

Action on data

On the data of interest, you can perform:

  • Insert: Insert data into
    the database.

  • Update: Update data in
    the database.

  • Upsert: Insert the data
    if they do not exist or update the existing data.

  • Delete: Remove the data
    from the database.

Generate JSON Document

Select this check box to generate a JSON document and configure
the desired data structure for it.

Key

Select the key that you want to use from the list.

Configure JSON Tree

Click the […] button to open
the window for JSON tree configuration. For more information, see
Configuring a JSON Tree.

Group by

Customize the input columns based on which you want to group the
data.

Remove root node

Select this check box to remove the root node.

Die on error

This check box is cleared by default, meaning to skip the row on
error and to complete the process for error-free rows.

Advanced settings

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the
Job level as well as at each component level.

Global Variables

Global Variables

NB_LINE: the number of rows read by an input component or
transferred to an output component. This is an After variable and it returns an
integer.

NB_LINE_INSERTED: the number of rows inserted. This is an
After variable and it returns an integer.

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule

This component is used as an output component and it always needs
an incoming link.

Scenario: Replicating data from the source database to the target database

This scenario applies only to a Talend solution with Big Data.

This scenario describes a Job that writes the information of books into a CouchDB
database, replicates the data to the target database, and then displays some replicated
information of interest on the console.

use_case_couchdboutput.png

Linking the components

  1. Drop the following components from the Palette onto the design workspace: tCouchDBConnection, tFileInputDelimited, tCouchDBOutput, tCouchDBInput, tLogRow and
    tCouchDBClose.
  2. Connect tFileInputDelimited to tCouchDBOutput using a Row > Main link.
  3. Do the same to connect tCouchDBInput to
    tLogRow.
  4. Connect tCouchDBConnection to tFileInputDelimited using a Trigger > OnSubjobOk
    link.
  5. Do the same to connect tFileInputDelimited to tCouchDBInput and tCouchDBInput to tCouchDBClose.
  6. Label the components to better identify their functions.

Configuring the components

Opening a CouchDB connection and triggering replication

  1. Double-click the tCouchDBConnection
    component to open its Basic settings view
    in theComponent tab.

    use_case_couchdboutput1.png

  2. In the Server and Port fields, type in the connection details.
  3. In the Database field, type in the name
    of the database you want to use: bookstore_old in this example.
  4. Select the CouchDB version that you are using from the DB Version list.
  5. Select the Use trigger replication check
    box.
  6. In the Replicate target database area,
    click [+] to add one line for database
    replication settings.
  7. Enter the name of the target database name: bookstore_new in this example.
  8. Select the Continuous check box to
    continue the replication upon the server restart.
  9. In this example, the target database does not exist. Select the Create target DB check box to create the target
    database.
  10. Select the Is Canceled check box to
    cancel the replication between bookstore_old and bookstore_new at the end of the Job.

Reading the input data

  1. Double-click the tFileInputDelimited
    component to open its Component
    view.

    use_case_couchdboutput2.png

  2. Click the […] button next to the
    File name/Stream field to browse to the
    file that you want to read data from. In this scenario, it is D:/Input/bookstore.txt. The file contains six
    columns: _id, title, author, category, ISBN, and abstract._id;title;author;category;ISBN;abstract
    001;Computer Networks: A Systems Approach;Larry L. Peterson, Bruce S. Davie;Computer Science;0123850606;This best-selling and classic book teaches you the key principles of computer networks with examples drawn from the real world of network and protocol design.
    002;David Copperfield;Charles Dickens;Language&Literature;1555763227;This adaptation of the original story is presented in the format of a novel study, complete with exercises and vocabulary lists, and is geared to the language arts classes of grades 4 and 5.
    003;Life of Pi;Yann Martel;Language&Literature;0547350651;The son of a zookeeper, Pi Patel has an encyclopedic knowledge of animal behavior and a fervent love of stories.
    004;Les Miserables: Easyread Comfort Edition;Victor Hugo;Language&Literature;1425048250;Expressing the author's ideas about society, religion and politics, it is in the backdrop of Napoleonic Wars and ensuing years that the story unravels. Grace, moral philosophy, law and history of France are discussed.
    005;Computer Security;Dieter Gollmann;Computer Science;0470741155;This text moves away from the 'multi-level' security approach to compare and evaluate design alternatives in computer security.
    006;Advanced Database Systems;Carlo Zaniolo;Database;155860443X;This book, written by a team of leading specialists in their fields, introduces the research issues at the forefront of database technology and supports them with a variety of examples.
  3. In the Header field, type in 1 so that the header of the file will be
    skipped.
  4. Click Edit schema to define the data to
    pass on to the tCouchDBOutput component.

    use_case_couchdboutput3.png

Writing data into one database and replicating the data

  1. Double-click the tCouchDBOutput component
    to open its Basic settings view in the
    Component tab.

    use_case_couchdboutput4.png

  2. Click Sync columns to retrieve the schema
    from the preceding component.
  3. Select the Use an existing connection
    check box. In this example, the replication is triggered when opening the
    CouchDB connection.
  4. Select Upsert from the Action on data drop-down list.

Extracting the replicated data of interest

  1. Double-click the tCouchDBInput component
    to open its Component view.

    use_case_couchdboutput5.png

  2. Click Edit schema to define the data
    structure to be read from the CouchDB database.

    By default, the Include docs check box is
    selected, so the id, key, value and jsonDoc columns are available in the
    schema.
    In this example, we define four columns to be extracted: id, title,
    author and category.
    use_case_couchdboutput6.png

  3. Enter the Server and Port information.
  4. In the Database field, enter the name of
    the database from which the replicated data will be read. In this example,
    it is bookstore_new.
  5. In the Querying options area, type in the
    start key and end key to set the range of the data to be read: “001” and “006” in this example.
  6. Select the Extract JSON field check box
    to extract the desired data.
  7. Select jsonDoc from the JSON field list.
  8. In the Mapping area, click [+] to add items. Select the schema output column
    from the list and then type in the proper XPath query.

Displaying the extracted data

  1. Double-click the tLogRow component to
    open its Component view.
  2. Click Edit schema to define the data
    structure to be displayed on the console. In this example, we need to remove
    the jsonDoc column.
  3. In the Mode area, select Table (print values in cells of a table).

Closing the CouchDB connection

  1. Double-click the tCouchDBClose component
    to open its Component view.

    use_case_couchdboutput7.png

  2. Select the connection to be closed from the Component List.

Saving and executing the Job

  1. Press Ctrl+S to save your Job.
  2. Execute the Job by pressing F6 or
    clicking Run on the Run tab.

    use_case_couchdboutput8.png

    The book information read from the replicated database is shown on the
    console.
Document get from Talend https://help.talend.com
Thank you for watching.
Docs 6.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x