July 30, 2023

tMDMInput – Docs for ESB 7.x

  2. Talend for ESB Docs
  3. Docs 7.x
  4. tMDMInput – Docs for ESB 7.x

tMDMInput

Reads data in an MDM Hub and thus makes it possible to process this
data.

tMDMInput Standard properties

These properties are used to configure tMDMInput running in the Standard Job
framework.

The Standard
tMDMInput component belongs to the Talend MDM family.

The component in this framework is available in all Talend
products.

Basic Settings

Property Type

Either Built in or Repository.

 

Built-in: No property
data stored centrally

 

Repository: Select the
repository file where properties are stored. The fields that follow are
completed automatically using the fetched data

Schema and Edit Schema

A schema is a row description, it defines the
number of fields that will be processed and passed on to the next component.
The schema is either built-in or remote in the Repository.

Click Edit
schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this
    option to view the schema only.

  • Change to built-in property:
    choose this option to change the schema to Built-in for local changes.

  • Update repository connection:
    choose this option to change the schema stored in the repository and decide whether
    to propagate the changes to all the Jobs upon completion. If you just want to
    propagate the changes to the current Job, you can select No upon completion and choose this schema metadata
    again in the Repository Content
    window.

 

Built-in: The schema
will be created and stored for this component only. Related Topic: see
Talend Studio

User Guide.

 

Repository: The schema
already exists and is stored in the repository. You can reuse it in various
projects and jobs. Related Topic: see
Talend Studio

User Guide.

Use an existing connection Select this check box if you want to use a configured
tMDMConnection
component.

MDM version

By default, Server 6.0 is selected. Although it is
recommended to migrate existing jobs for this new version, the Server 5.6 option is available
to ease the process of the migration of your Jobs so as to keep them working
without modification with a 6.0 server. To do so, an option on the server must
be enabled to accept and translate requests from such Jobs.

URL

Type in the URL to access the MDM server.

Username and Password

Type in user authentication data for the MDM
server.

To enter the password, click the […] button next to the
password field, and then in the pop-up dialog box enter the password between double quotes
and click OK to save the settings.

Entity

Type in the name of the business entity that
holds the data you want to read.

Data Container

Type in the name of the data container that
holds the data you want to read.

Type

Select Master or Staging to specify the database on which the
action should be performed.

Use multiple conditions

Select this check box to filter the data using
certain conditions.

  • Xpath: Enter between quotes the path and the XML node to which you want to
    apply the condition.

  • Function: Select an operator to be used from the list. The following
    operators are available:

    • Contains:
      Returns a result which contains the word or words entered. Note that full text search
      does not support special characters, for example, @, #, $.
    • Contains the
      sentence      : Returns one or more results which contain the sentence
      entered.
    • Joins With:
      This operator is reserved for future use.
    • Starts With:
      Returns a result which begins with the string entered.
    • Equal:
      Returns a result which matches the value entered.
    • Not Equal:
      Returns a result of any value other than the null value and the value entered.
    • is greater
      than      : Returns a result which is greater than the numerical value entered.
      Applies to number fields only.
    • is greater or
      equal      : Returns a result which is greater than or equal to the numerical
      value entered. Applies to number fields only.
    • is lower
      than      : Returns a result which is less than the numerical value entered.
      Applies to number fields only.
    • is lower or
      equal      : Returns a result which is less than or equal to the numerical value
      entered. Applies to number fields only.
    • whole content
      contains      : Performs a plain text search using the specified Xpath field in the selected data
      container. If you enter an empty string “” in the Xpath field and select whole content contains from the Function list, searches will be performed in all the
      fields of all entities in the selected data container.
    • is empty or
      null      : Returns an empty field or a null value.

    Before using the conditions, bear in mind the
    following:

    • Depending on the type of field the Xpath points to, only certain operators
      apply. For example, if the field is a boolean, only the Equal or Not Equal
      operators are appropriate.

    • Searches in the master database support both full
      text search and standard SQL search, while searches in the staging database only
      support standard SQL search. For more information about how to manage database search
      modes, see Talend
      Installation and Upgrade Guide      .

    • Standard SQL search considers the search content as
      an integrate phrase to be matched, while full text search interprets the search
      content as a phrase which contains multiple words, and the records that match any of
      the words will be returned. For example, if the search content is “test first name”,
      full text search considers it as three independent words “test”, “first”, and “name”,
      while standard SQL search considers it as one phrase “test first name”.

    • The operator Not
      Equal       does not support multi-occurrence fields or complex type fields.

  • Value: Enter
    between inverted commas the value you want to use. Note that if the value
    contains XML special characters such as /, you must also enter the value in
    single quotes (“‘ABC/XYZ'”) or the value will be considered as an
    XPath.

  • Predicate: Select a predicate from the drop-down list.

    The predicates
    none (default), or, and, and Not are now available for use. The other predicates are
    reserved for future.

    • none
      (default): is interpreted as and by
      default.
    • or: returns a
      record if any of the conditions separated by it is true.
    • and: returns
      a record if all the conditions separated by it are true.
    • Not: returns a record if the
      condition on which it is defined is not true.
    Note: The or predicate is interpreted only if it is defined on all
    conditions, except the last condition. Otherwise the or predicate
    is always interpreted as and. For more information, see the related
    description of behaviors of filters using multiple conditions and predicates in Talend
    MDM on Talend Help Center (https://help.talend.com).

If you clear this check box, you have the
option of selecting particular IDs to be displayed in the ID value column of the
IDS table.

Note:

If you clear the Use multiple conditions check box, the
Batch Size option
in the Advanced
Settings tab will no longer be available

Skip Rows

Enter the number of lines to be ignored.

Max Rows

Maximum number of rows to be processed. If
Limit = 0, no row is read or processed.

Die on error

Select this check box to skip the row in error
and complete the process for error-free rows. If needed, you can retrieve the
rows in error via a Row
> Rejects
link.

Advanced settings

Batch Size

Number of lines in each processed batch.

Note:

This option is not displayed if you have
cleared the Use multiple
conditions check box in the Basic settings view.

Loop XPath query

The XML structure node on which the loop is
based.

Mapping

Column: reflects the
schema as defined in the Edit schema editor.

XPath query: Type in
the name of the fields to extract from the input XML structure.

Get Nodes: Select this
check box to retrieve the Xml node together with the data.

tStatCatcher Statistics

Select this check box to gather the processing
metadata at the Job level as well as at each component level.

Global Variables

Global Variables

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

NB_LINE: the number of rows processed. This is an After
variable and it returns an integer.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule

Use this component as a start component. It
needs an output flow.

If an MDM entity has a large number of foreign key fields, it
may take you a long time to query data records that pertain to the entity using
the tMDMInput component.

For more information about the
workaround, see the article How to read data effectively that pertains to an
MDM entity with many foreign key fields on Talend Help Center (https://help.talend.com).

You can increase the timeout values for a Job using this component
to help process a large number of data records. For more information, see advanced
execution settings for JVM parameters in the article Timeout values for a Job using
MDM components on Talend Help Center (https://help.talend.com).

Reading master data from an MDM hub

This scenario applies only to Talend MDM Platform and Talend Data Fabric.

This scenario describes a two-component Job that fetches master data from an MDM server, and
displays the data in the log console.

tMDMInput_1.png

Prerequisites:

  • Make sure the MDM server is up and running.

  • You have imported the MDM demo project and loaded the sample data into the data container
    Product by running the Job MDM_LoadAll.

  • You have been assigned a role with the appropriate user authorization and access rights to
    the data model Product.

Creating a Job to read master data from MDM

  1. From the Palette, drop tMDMInput and tLogRow onto the design
    workspace.
  2. Link the two components together using a Row > Main
    connection.

Configuring basic settings of tMDMInput to read master data from MDM

  1. Double-click tMDMInput to open the Basic settings view.

    tMDMInput_2.png

  2. In the Property Type list, select Built-In.
  3. In the Schema list, select Built-In and click the … button next to
    Edit schema to open a dialog box and define the structure
    of the master data you want to read from the MDM server.

    In this example, three columns are defined to fetch three elements from the
    Product entity: Name,
    Price, and Colors.
    tMDMInput_3.png

  4. After you have defined the schema, click OK to close
    this dialog box, and then click Yes in the Propagate dialog box to propagate the schema changes to tLogRow.
  5. Enter the user name and password for accessing the MDM server.
  6. In the Entity field, enter Product
    between quotes.
  7. In the Data Container field, enter
    Product between quotes.
  8. Select Master from the Type list.
  9. Define the query conditions in the Operations area.

    In this example, we want to query the product data records whose names include
    Shirt.
    1. Click the [+] button to add a row.
    2. Enter Product/Name between quotes in the Xpath field.

      Apart from elements defined in entities, you can query metadata elements which are
      also known as built-in elements. To query metadata elements from records in master
      database, you need to follow the format of
      metadata:<timestamp|task_id> when defining the path expression to
      select the XML node to run the query on.
    3. Select Contains from the Function list.
    4. Enter Shirt between quotes in the Value field.

Configuring advanced settings of tMDMInput to read master data from MDM

  1. In the Component view, click the Advanced settings tab.

    tMDMInput_4.png

  2. In the Loop XPath query field, enter
    /Product between quotes on which the loop is based.
  3. In the XPath query column of the Mapping table, enter the name of the XML node from which you want to collect
    the master data, next to the corresponding output column name.
  4. Select the Get Nodes check box for the
    Colors row to retrieve the XML node together with its data.

Configuring the data display mode and executing the Job

  1. Double-click the tLogRow component to display its
    Basic settings view.
  2. In the Mode area, select Table
    (print values in cells of a table)     for better readability of the result.
  3. Save the Job and press F6 to run it.

    The Product data records whose names include “Shirt” are displayed
    on the console with the values of three specified columns.
    tMDMInput_5.png

Reading staging data from MDM

This scenario applies only to Talend MDM Platform and Talend Data Fabric.

This scenario describes a two-component Job that reads some staging data from the MDM
staging area, and then displays the data in the log console.

tMDMInput_6.png
Prerequisites:

  • Make sure the MDM server is up and running.

  • You have created a Customer data model with a
    Customer entity, and a Customer data container
    that you have already populated with some staging data records.

  • You have been assigned a role with the appropriate user authorization and access
    rights to the data model Customer.

Creating a Job to read staging data from MDM

  1. From the Palette, drop tMDMInput and tLogRow onto the design
    workspace.
  2. Link the two components together using a Row > Main
    connection.

Configuring the connection to read staging data from MDM

  1. Double-click tMDMInput to open the Basic settings view.

    tMDMInput_7.png

  2. In the Property Type list, select Built-In.
  3. In the Schema list, select Built-In and click the … button next to
    Edit schema to open a dialog box and define the structure
    of the staging data you want to read from the MDM server.

    In this example, three columns are defined to fetch three elements from the
    Customer entity: account_number,
    lastname, and firstname.
    tMDMInput_8.png

  4. After you have defined the schema, click OK to close
    this dialog box, and then click Yes in the Propagate dialog box to propagate the schema changes to tLogRow.
  5. Keep the default Server 6.0 in the MDM version list and keep the default URL in the URL field.
  6. Enter the user name and password for accessing the MDM server.
  7. In the Entity field, enter between quotes the name of
    the business entity from which you want to query the staging data.
  8. In the Data Container field, enter between quotes the
    name of the data container that holds the staging data you want to read.
  9. Select Staging from the Type list.

Configuring the query conditions and advanced settings to query staging data

  1. In the Operations area, click the [+] button to add one new row, where you can define the query conditions.
  2. In the Xpath column, enter
    Customer/metadata:staging_status between quotes.

    In this example, we want to query the staging data that fails to pass the MDM validation
    phase due to a validation issue against the data model, and those staging data has the
    status of 403. For more information, see the section on working with records in the staging
    area in
    Talend MDM Web UI User Guide    .
    Note:

    Unlike elements defined in entities, metadata elements are
    built-in elements. To query metadata elements from records in staging database,
    you need to follow the format of metadata:staging_<source|status|error|blockkey|hastask> when
    defining the path expression to select the XML node to run the query on.

  3. Select Equal from the Function list.
  4. Enter 403 between quotes in the Value column.
  5. Click the Advanced settings tab.
  6. In the Loop XPath query field, enter
    /Customer between quotes on which the loop is based.
  7. In the XPath query column of the Mapping table, enter between quotes the name of the XML node from which you
    want to collect the staging data, next to the corresponding output column name.

    In this example, three elements that pertain to the Customer entity
    will be queried: account_num, lname, and
    fname.
    tMDMInput_9.png

Configuring the query result display and executing the Job

  1. Double-click the tLogRow component to display its
    Basic settings view.
  2. In the Mode area, select Table
    (print values in cells of a table)     for better readability of the result.
  3. Save the Job and press F6 to run it.

    The staging data that matches the query conditions is displayed on the console.
    tMDMInput_10.png

Document get from Talend https://help.talend.com
Thank you for watching.
Docs 7.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x