August 16, 2023

tNLPPredict – Docs for ESB 6.x

  2. Talend for ESB Docs
  3. Docs 6.x
  4. tNLPPredict – Docs for ESB 6.x

tNLPPredict

Uses a classifier model generated by tNLPModel to predict and
label the input text.

tNLPPredict outputs labeled text and the labels for tokens in two
separate columns.

This component can run only with Spark 1.6.

tNLPPredict properties for Apache Spark Batch

These properties are used to configure tNLPPredict running in
the Spark Batch Job framework.

The Spark Batch
tNLPPredict component belongs to the Natural Language Processing family.

The component in this framework is available when you have subscribed to any Talend Platform product with Big Data or Talend Data
Fabric.

Basic settings

Schema and Edit Schema

A schema is a row description. It defines the number of fields (columns) to
be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Click Sync columns to retrieve the schema from
the previous component connected in the Job.

Click Edit schema to make changes to the schema.
If the current schema is of the Repository type, three
options are available:

  • View schema: choose this option to view the
    schema only.

  • Change to built-in property: choose this
    option to change the schema to Built-in for
    local changes.

  • Update repository connection: choose this
    option to change the schema stored in the repository and decide whether to propagate
    the changes to all the Jobs upon completion. If you just want to propagate the
    changes to the current Job, you can select No
    upon completion and choose this schema metadata again in the [Repository Content] window.

Read-only columns are added to the output schema:

  • outputsent: This column holds the labeled
    text.

  • outputlabel: This column holds the
    labels.

 

Built-In: You create and store the
schema locally for this component only. Related topic: see
Talend Studio

User Guide.

 

Repository: You have already created
the schema and stored it in the Repository. You can reuse it in various projects and
Job designs. Related topic: see
Talend Studio

User Guide.

Define a storage configuration component

Select the configuration component to be used to provide the configuration
information for the connection to the target file system such as HDFS.

If you leave this check box clear, the target file system is the local
system.

The configuration component to be used must be present in the same Job. For
example, if you have dropped a tHDFSConfiguration component in the Job, you can select it to write
the result in a given HDFS system.

Original text column

Select the column to be labeled in the input schema.

Token column

Select the column used for feature construction and prediction.

Additional Features

Select this check box to add additional features to the
Additional feature template.

When you add features, the order must be the same as the additional
features used in the TNLPModel component to generate
the model file.

NLP model path

Set the path to the local folder from where you want to retrieve the
model file.

If you want to store the model in a specific file system, for example S3
or HDFS, you must use the corresponding component in the Job and select
the Define a storage configuration component
check box in the component basic settings.

The button for browsing does not work with the Spark Local mode; if you are using the Spark Yarn or the Spark Standalone mode,
ensure that you have properly configured the connection in a configuration component in
the same Job, such as tHDFSConfiguration.

Usage

Usage rule

This component is used as an intermediate step.

This component, along with the Spark Batch component Palette it belongs to, appears only
when you are creating a Spark Batch Job.

Spark Batch Connection

You need to use the Spark Configuration tab in
the Run view to define the connection to a given
Spark cluster for the whole Job. In addition, since the Job expects its dependent jar
files for execution, you must specify the directory in the file system to which these
jar files are transferred so that Spark can access these files:

  • Yarn mode: when using Google
    Dataproc, specify a bucket in the Google Storage staging
    bucket     field in the Spark
    configuration     tab; when using other distributions, use a
    tHDFSConfiguration
    component to specify the directory.

  • Standalone mode: you need to choose
    the configuration component depending on the file system you are using, such
    as tHDFSConfiguration
    or tS3Configuration.

This connection is effective on a per-Job basis.

Scenario: Extracting named entities using a classification model

This scenario applies only to a subscription-based Talend Platform solution with Big data or Talend Data Fabric.

The tNLPPredict component predicts and automatically labels named
entities using a classification model generated by the tNLPModel
component.

You can find more information about natural language processing on
Talend Help Center (https://help.talend.com).

Creating a Job to extract named entities from text data

In this Job, tNLPPredict predicts named entities and
automatically labels text data divided into tokens, using a classification
model.

  1. Drop the following components from the Palette onto the
    design workspace: tMySQLInput,
    tNLPPredict, tFilterColumns
    and tFileOutputDelimited.
  2. Connect the components using Row > Main connections.
use_case_tnlppredict.png

Configuring the input component

The tokenized text in CoNLL format to be labeled is stored in a MySQL
database.

  1. Double click the tMysqlInput component to open its
    Basic settings view and define its properties.

    use_case_tnlppredict2.png

  2. From the Property Type list, select
    Repository if you have already stored the connection
    to database in the Metadata node of the Repository tree view. The property
    fields that follow are automatically filled in.
  3. Set the Schema as Built-in and click Edit
    schema     to define the desired schema.

    In this example, three columns are added to the schema:
    text, tokens and
    labels.

  4. Set the Query Type as Built-In.
    In the Query box, enter the query required to retrieve
    the desired columns from the table.

Extracting named entities from text data

In this Job, the tNLPPredict component predicts named entities
and automatically labels text data, using a classification model generated by the
tNLPModel component.

  1. Double click the tNLPPredict component to open its
    Basic settings view and define its properties.

    use_case_tnlppredict3.png


    1. Click Sync columns to retrieve the
      schema from the previous component connected in the Job.
    2. From the Original text column list, select the
      column that holds the text to be labeled, which is
      text in this example.
    3. From the Token column list, select the column
      used for feature construction and prediction, which is
      tokens in this example
    4. From the NLP Library list, select the same
      library you used for generating the model.
    5. Specify the path to the named entity recognition model in the
      NLP model path.
  2. Double click the tFilterColumns component to open its
    Basic settings view and define its properties.


    1. Click Sync columns to retrieve the
      schema from the previous component connected in the Job.
    2. Set the Schema as Built-in and click
      Edit schema to keep only the columns that
      hold the original text, the labeled text and the labels.

      use_case_tnlppredict4.png

  3. Double click the tFileOutputDelimited component to open
    its Basic settings view and define its properties.

    use_case_tnlppredict5.png


    1. Click Sync columns to retrieve the
      schema from the previous component connected in the Job.
    2. Specify the path to the folder where you want to store the labeled text
      and the labels, in the Folder field.

    3. Enter "
      "       in the Row
      separator       field and ";" in the
      Field separator field.

  4. Press F6 to save and execute the
    Job.

The output files contain the original text, the labeled text and the labels. The
named entity recognition task was performed correctly, since person names were
extracted from the original text.

use_case_tnlppredict6.png

Document get from Talend https://help.talend.com
Thank you for watching.
Docs 6.x
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x