August 17, 2023

tCouchbaseInput – Docs for ESB 5.x

  2. Talend for ESB Docs
  3. Docs 5.x
  4. tCouchbaseInput – Docs for ESB 5.x

tCouchbaseInput

tcouchbaseinput_icon32_white.png

Warning

This component will be available in the Palette of the studio on the condition that you have subscribed to
one of the Talend solutions with Big
Data.

tCouchbaseInput Properties

Component family

Big Data / Couchbase

  

Function

tCouchbaseInput allows you to
fetch your documents from the Couchbase database either by the
unique key or through Views.

Purpose

This component allows you to query the documents from the
Couchbase database.

Basic settings

Schema and Edit
Schema

A schema is a row description. It defines the number of fields to be processed and passed on
to the next component. The schema is either Built-In or
stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are
available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the
current schema is of the Repository type, three options are
available:

  • View schema: choose this option to view the
    schema only.

  • Change to built-in property: choose this option
    to change the schema to Built-in for local
    changes.

  • Update repository connection: choose this option to change
    the schema stored in the repository and decide whether to propagate the changes to
    all the Jobs upon completion. If you just want to propagate the changes to the
    current Job, you can select No upon completion and
    choose this schema metadata again in the [Repository
    Content]     window.
 

Use existing connection

Select this check box and in the Component List click the
relevant connection component to reuse the connection details you already defined.

 

DB Version

List of database versions.

 

Data Bucket

Name of the data bucket in the Couchbase database.

 

Username and Password

Authentication credentials for a data bucket, instead of those for
a server node.

To enter the password, click the […] button next to the
password field, and then in the pop-up dialog box enter the password between double quotes
and click OK to save the settings.

 

URIs

URIs of server nodes in the Couchbase cluster, in the form of
“http://127.0.0.1:8091/pools”
or “http://localhost:8091/pools”.

View

Key

(not in the Query by
view mode)

IDs of the documents stored in the Couchbase database, in the form
of “1”, “2” or “Integration at any scale”, etc.

 Query by view

 Query by view

Select this check box to view the document information according
to the Map/Reduce functions and other settings.

Note

When this check box is selected, the schema has three
pre-defined fields, Id,
Key and Value.

Where, Id holds the
document ID, Key holds the
information specified by the key of the Map
function and Value holds the
information specified by the value of the Map
function.

 

Design document

Name of the design document in the Query by
view mode.

Note

  • This document appears in the Production Views area under the
    View tab in the
    Couchbase web console.

  • Multiple views can be created for each design
    document.

Doc action

Actions below are available:

None: no action is taken on the
document specified in the Design
document field.

Create: create the document
specified in the Design document
field.

Drop and Create: drop and then
create the document specified in the Design
document field.
 

View

Name of the view for extracting the document information.

View action

Actions below are available:

Create: create the view specified
in the View field.
 

Map

Type in the code for the Map function.

Available when Create is selected
in the View action list.
 

Reduce

Type in the code for the Reduce function.

Available when Create is selected
in the View action list.
 

Startkey

Type in the startkey of a key range.
 

Endkey

Type in the endkey of a key range.
 

Startkey docid

Type in the start document ID of a docid range.
 

Endkey docid

Type in the end document ID of a docid range.
 

Use complex key

Select this check box to show the Complexkey field, where you can type in the complex
keys for the view queries.

Note that here the keys refer to the values of the key defined in
the Map function.
 

Key

(in the Query by
view mode)

The values of the key defined in the Map function.

Not available when Use complex
key is selected in the Query by
view mode.
 

Include docs

Select this check box to include the document specified by the Key
in the view results.

Note that the JSONDoc field
appears in the schema once this check box is selected.
 

Inclusive end

Select this check box to include the specified end key in the
result.

 

Descending

Returns the documents in descending order, by key.

 

Stale

Allows the results from a Stale view to be used.

false: forces a view update
before returning data.

ok: allows stale views.

update_after: allows stale view
and updates view after it has been accessed.
 

Limit

Limits the number of the returned documents to the specified
number.

 

Skip

Skips this number of records before starting to return the
results.

JSON Configuration

Extract JSON field

Select this check box to extract data from a JSON field.

 

JSON field

List of JSON fields for data extraction.

 

Loop XPath query

XPath query to specify the loop node of the JSON data.

 

Mapping

Schema output column: output
columns for holding the JSON data extracted.

XPath Query: specifies the JSON
node that holds the desired data.

Get nodes: select this check box to
extract the JSON data of all the nodes specified in the XPath query list or select the check box
next to a specific node to extract its JSON data only.

 

Die on error

This check box is cleared by default, meaning to skip the row on
error and to complete the process for error-free rows.

Advanced settings

Debug

Outputs the debug information.

Available in the Query by view
mode.

 

tStatCatcher Statistics

Select this check box to collect the log data at the component
level.

Global Variables

NB_LINE: the number of rows read by an input component or
transferred to an output component. This is an After variable and it returns an
integer.

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio
User Guide.

Usage

As a start component, tCouchbaseInput reads the documents from the
Couchbase database either by the unique key or through Views.

Log4j

The activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User
Guide.

For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

Limitation

n/a

Scenario 1: Querying JSON documents in the Couchbase database by unique document
IDs

This scenario queries the JSON documents in the Couchbase database by unique document
IDs. Specifically, we use the document IDs as the key to query the documents, whose
contents are JSON data. The documents in the Couchbase database are shown below:

use_case_tcouchbaseinput_1.png

For how to write such documents to the database, see Scenario: Inserting documents to a data bucket in the Couchbase database.

Linking the components

  1. Drop tCouchbaseConnection, tCouchbaseInput, tCouchbaseClose and tLogRow
    onto the workspace.

  2. Link tCouchbaseConnection to tCouchbaseInput using the OnSubjobOk trigger.

  3. Link tCouchbaseInput to tLogRow using a Row > Main
    connection.

  4. Link tCouchbaseInput to tCouchbaseClose using the OnSubjobOk trigger.

    use_case_tcouchbaseinput_2.png

Configuring the components

  1. Double-click tCouchbaseConnection to open
    its Basic settings view.

    use_case_tcouchbaseoutput_2.png

  2. In the Data Bucket field, enter the name
    of the data bucket in the Couchbase database.

  3. In the Password field, enter the password
    for access to the data bucket.

  4. In the URIs table, click the [+] button to add lines as needed, where you can
    enter the URIs of the Couchbase server nodes.

  5. Double-click tCouchbaseInput to open its
    Basic settings view.

    use_case_tcouchbaseinput_3.png

  6. Select the Use existing connection check
    box to reuse the connection.

  7. In the Key field, enter the document IDs,
    for example “ELT Overview”, “Integration at any
    scale”    .

  8. Click the Edit schema button to open the
    schema editor. The schema contains two pre-defined fields, Key and Value.

    use_case_tcouchbaseinput_4.png

  9. Double-click tLogRow to open its
    Basic settings view.

    use_case_tcouchbaseinput_5.png

  10. Select Table (print values in cells of a
    table     for a better display of the results.

Executing the Job

  1. Press Ctrl + S to save the Job.

  2. Press F6 to run the Job.

    use_case_tcouchbaseinput_6.png

    As shown above, the JSON documents with the keys (IDs) of ELT Overview and Integration at any scale are retrieved.

Scenario 2: Querying JSON documents in the Couchbase database through view
queries

This scenario queries the JSON documents through view queries. Specifically, we
combine the View queries and some other filters to retrieve the document information.
For the View part, we define the map function where the Key is specified as the
id field of the documents while the Value is
specified as the title and contents fields. The documents in the Couchbase database are shown
below:

use_case_tcouchbaseinput_1.png

For how to write such documents to the database, see Scenario: Inserting documents to a data bucket in the Couchbase database.

Linking the components

  1. Drop tCouchbaseConnection, tCouchbaseInput, tCouchbaseClose and tLogRow
    onto the workspace.

  2. Link tCouchbaseConnection to tCouchbaseInput using the OnSubjobOk trigger.

  3. Link tCouchbaseInput to tLogRow using a Row > Main
    connection.

  4. Link tCouchbaseInput to tCouchbaseClose using the OnSubjobOk trigger.

    use_case_tcouchbaseinput_2.png

Configuring the components

  1. Double-click tCouchbaseConnection to open
    its Basic settings view.

    use_case_tcouchbaseoutput_2.png

  2. In the Data Bucket field, enter the name
    of the data bucket in the Couchbase database.

  3. In the Password field, enter the password
    for access to the data bucket.

  4. In the URIs table, click the [+] button to add lines as needed, where you can
    enter the URIs of the Couchbase server nodes.

  5. Double-click tCouchbaseInput to open its
    Basic settings view.

    use_case_tcouchbaseinput_7.png

  6. Select the Use existing connection check
    box to reuse the connection.

  7. Select the Query by view check box to
    define the View functions and other filters.

  8. In the Design document field, enter the
    design document name of the View.

    In the Doc action list, select Drop create to remove an existing design document
    and create it again.

    In the View field, enter the name of the
    View.

    In the View action list, select Create to create the View.

  9. In the Map box, enter Map
    function:

    Where, the Key is doc.id, namely the id field of the JSON documents and the Value is
    [doc.title,doc.contents], namely the title and contents fields of the JSON documents.

  10. Click the Edit schema button to open the
    schema editor. The schema contains four pre-defined fields, Id, Key,
    Value and jsonDoc.

    use_case_tcouchbaseinput_8.png

    In this scenario, Id holds the document
    ID, Key holds the id field of the JSON documents, Value holds the title and
    contents fields of the JSON documents
    and jsonDoc holds the entire JSON
    documents.

  11. Select the Include docs check box to
    retrieve the entire documents.

  12. Double-click tLogRow to open its
    Basic settings view.

    use_case_tcouchbaseinput_5.png

  13. Select Table (print values in cells of a
    table     for a better display of the results.

Executing the Job

  1. Press Ctrl + S to save the Job.

  2. Press F6 to run the Job.

    use_case_tcouchbaseinput_9.png

  3. Go to the Couchbase web console and check the View created:

    use_case_tcouchbaseinput_10.png

    As shown above, the View is created and the document information is
    correctly fetched.

Document get from Talend https://help.talend.com
Thank you for watching.
Docs 5.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x