tHMap
Executes transformations (called maps) between different sources and destinations
by harnessing the capabilities of Talend Data Mapper , available in the Mapping perspective.
tHMap transforms data from a wide range of sources to a wide range of
destinations, including support for multiple inputs and multiple outputs. If you want to
use multiple inputs, you must use Talend Data Mapper I/O functions. For more information, see Talend Data Mapper User Guide.
tHMap Standard properties
These properties are used to configure tHMap running in the Standard Job framework.
The Standard
tHMap component belongs to the Processing family.
This component is available in Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
Basic settings
Open Map Editor |
Click the […]
If Talend Studio detects If neither input nor output connection exists, the Structure Selection |
Map Path |
Specifies the map to be executed. If the map was automatically created using the wizard described above, If you want to use an existing map, click the […] button next to the Map |
Read Input As |
Select the radio button which corresponds to how you want the input to
|
Write Output As |
Select the radio button which corresponds to how you want
|
Advanced settings
Map Variable |
In this field, enter a context variable that contains a In the Contexts Note that all maps that might be referenced by the For further information on working with |
Alternate contextual Map(s) |
From the drop-down list, select a map or several maps that can be |
Map each row (disable virtual |
Select this check box to have tHMap process the input as a single output row. This This can be useful, for example, when you use the |
Exception Threshold |
Talend Data Mapper returns an execution status with an severity From the drop-down list, select the severity level at
Note that, in order to help you diagnose problems with |
tStatCatcher Statistics |
Select this check box to gather the Job processing |
Global Variables
Global Variables |
ERROR_MESSAGE: the error message generated by the
MapExecutionStatus: returns an object of the
EXECUTION_SEVERITY: the Overall A Flow variable functions during the execution of a component while an After variable To fill up a field or expression with a variable, press Ctrl + For further information about variables, see |
Usage
Usage rule |
tHMap is used for Jobs that The input and output connections can use When you open the Map Editor for the This component can be used in several ways:
|
Advanced use case |
|
Limitations |
|
-
For further information about performing
transformations using Talend Data Mapper , see Talend Data Mapper User Guide. - For tHMap using representations other than Hashmap,
CSV, Avro, XML, and Cobol: Running tHMap may create
temporary folders, which are not automatically removed after completing
the Eclipse runtime. If you frequently run Talend Data Mapper, these files may
pile up and eventually lead to a disk full error. When the error occurs,
the runtime fails. To avoid such scenario, ensure that you perform
periodical cleaning.
Using Talend Data Mapper metadata
This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
The following scenario creates a three-component Job, reading data from an input file
that is transformed using a map that was previously created in the
Mapping
perspective and then outputting the transformed data in a new file. It works with
Talend Data Mapper
metadata.
Copying an editable version of the example files
-
In the
Mapping
perspective, in
the Data Mapper view, expand the Hierarchical Mapper node and the Other Projects folder, right-click Examples and then select Copy in the contextual menu. -
In the Data Mapper view, right-click at the
root of the Hierarchical Mapper node, and then
select Paste in the contextual menu.This copies an editable version of all the read-only example files to your
local workspace.
Adding and linking the components
-
In the
Integration
perspective,
create a new Job and call it tdm_to_tdm. -
Click the point in the design workspace where you want to add the first
component, start typing tFileInputRaw, and then
click the name of the component when it appears in the list proposed in order to
select it. -
Do the same to add a tHMap component and a
tFileOutputRaw component as well. -
Connect the tFileInputRaw component to the
tHMap component using a Row > Main link and rename it input, then connect the tHMap
component to the tFileOutputRaw component using
a Row > Main link and name it output. When you are asked if you want to get the
schema of the target component, click Yes.
Defining the properties of tFileInputRaw
-
Select the tFileInputRaw component to define
its properties. -
In the Basic settings tab, click the
[…] button next to the Filename field then browse to the location on your
file system where the input file is stored, or enter the path manually between
double quotes. For this example, use <PATH_TO_WORKSPACE>/<PROJECT_NAME>/Sample
Data/CSV/PurchaseOrderPayPal/PayPalPO.csv. -
Set the Mode as Read
the file as a string, and leave all the other parameters
unchanged.
Defining the properties of tFileOutputRaw
-
Select the tFileOutputRaw component to define
its properties. -
In the Basic settings tab, click the
[…] button then browse to the location on
your file system where the output file is to be stored, or enter the path
manually between double quotes. Leave the other parameters unchanged.
Defining the properties of tHMap
-
Select the tHMap component to define its
properties. -
Click the […] button next to the Map Path field to open the picker and select the map
to use, Maps/CSV/POPayPalCsv_PO2, then click
OK. This map transforms a CSV file into an
XML file. - Check that Read Input As is set to Single Column.
- Check that Write Output As is set to String (single column).
Saving and executing the Job
- Press Ctrl+S to save your Job.
- In the Run tab, click Run to execute the Job.
-
Browse to the location on your file system where the output file is stored to
check that an XML file has been created containing the same data as the input
CSV file.
Using Talend Data Integration metadata
This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
The following scenario creates a three-component Job, reading data from an input file
that is transformed using a map that you create in the
Mapping
perspective and then outputting the transformed data in a new file. It works with
Talend Data Integration
metadata.
Copying an editable version of the example files
-
In the
Mapping
perspective, in the Data Mapper view, expand the Hierarchical Mapper node and the Other
Projects folder, right-click Examples
and then select Copy in the contextual menu. -
In the Data Mapper view, right-click at the root
of the Hierarchical Mapper node, and then select
Paste in the contextual menu.This copies an editable version of all the read-only example files to your local
workspace.
Adding and linking the components
-
In the
Integration
perspective, create a new
Standard Job and call it di_to_di. -
Click the point in the design workspace where you want to add the first component,
start typing tFileInputDelimited, and then click
the name of the component when it appears in the list proposed in order to select
it. -
Do the same to add a tHMap component and a
tFileOutputXML component as well. -
Connect the tFileInputDelimited component to the
tHMap component using a Row
> Main link, then connect the tHMap
component to the tFileOutputXML component using a
Row > Main link.
Defining the properties of tFileInputDelimited
-
Select the tFileInputDelimited component to
define its properties. -
In the Basic settings tab, click the […] button next to the File
name/Stream field then browse to the location on your file system where
the input Excel file is stored, or enter the path manually between double quotes.
For this example, use <PATH_TO_WORKSPACE>/<PROJECT_NAME>/Sample
Data/CSV/PurchaseOrderPayPal/PayPalPO.csv. - Select the CSV options check box.
-
Change the Field Separator to a comma, between
double quotes (“,”). - Change the value of Header to 1.
- Click the […] button next to Edit schema to define the schema.
-
Add three columns and rename them txn_id,
payment_date and first_name (which correspond to the names of the first three columns in
the input file, and is sufficient for the purposes of this example), and then click
OK. - Leave all the other parameters unchanged.
Defining the properties of tFileOutputXML
-
Select the tFileOutputXML component to define its
properties. -
In the Basic settings tab, click the […] button next to the File
Name field then browse to the location on your file system where the
output file will be stored, or enter the path manually between double quotes. - Click the […] button next to Edit schema to define the schema.
-
Add three columns to the input schema on the left and rename them id, date and name, copy them to the output schema on the right, and then
click OK. - Leave the other elements unchanged.
Defining the properties of tHMap
-
Select the tHMap component to define its
properties. -
Click the […] button next to the Open Map Editor field to create a new map based on the input
and output of tHMap. -
In the tHMap Structure Generate/Select dialog box
that opens, select Generate hierarchical mapper structure
based on the schema and then click Next
to generate the input structure. - Do the same for the output structure.
-
In the Map editor that opens, drag the txn_id
element of Input (map) to the id element of Output (map). Do the same
to map payment_date to date and first_name to name, and then save your changes.
Saving and executing the Job
- Press Ctrl+S to save your Job.
- In the Run tab, click Run to execute the Job.
-
Browse to the location on your file system where the output file is stored to
check that an XML file has been created containing the same data as the input CSV
file.
Transforming from a Data Integration schema to a complex content
schema
This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
The following scenario creates a three-component Job, generating some random data from
an input component, transforming this data using a map which was previously created in
the
Mapping
perspective, and then outputting the transformed data in a JSON file. It works with
Talend Data Integration
metadata for the input and
Talend Data Mapper
metadata for the output.
Creating a new structure in the
Mapping
perspective
-
In the
Mapping
perspective, in
the Data Mapper view, expand the Hierarchical Mapper node, right-click Structures and then select New > Structure. -
In the New Structure dialog box that opens,
select Create a new structure where you manually enter
elements, and then click Next. -
Name your structure JSON_structure, and
then click Next. -
In the Select Representation dialog box,
select JSON from the list of available
representations, and then click Next. -
Select Don’t select a sample document for
now, and then click Finish.
Entering the elements for your new structure
-
In the
Mapping
perspective, in
the Data Mapper view, expand the Hierarchical Mapper node and the Structures node, and then open the JSON_structure structure you created
earlier. -
In the JSON_structure, right-click to add a
new element, click New element and name the new
element Root. -
Follow the same steps to create a new element called people under the Root
element, a person element under people, and four new elements under the person element: firstname, lastname,
address and city. -
For the person element, change the
Occurs Max value to -1 (unlimited). - Press Ctrl + S to save your changes.
Adding and linking the components
-
In the Integration perspective, in the Repository, right-click Job
Designs, and then click Create Standard
Job to create a Job named di_to_json. Add a Purpose and
Description if you wish, and then click
Finish. -
Click the point in the design workspace where you want to add the first
component, start typing tRowGenerator, and then
click the name of the component when it is displayed in the list proposed in
order to select it. -
Do the same to add a tHMap component, and a
tFileOutputRaw component as well. -
Connect the tRowGenerator component to the
tHMap component using a Row > Main link, then connect the tHMap component to the tFileOutputRaw component using a Row >
Main link. When you are asked if you want to get the schema of
the target component, click Yes.
Defining the properties of tRowGenerator
-
Select the tRowGenerator component to define
its properties. -
In the Basic settings tab, click the
[…] button next to RowGenerator Editor to define the rows to be generated. -
In the dialog box that opens, click four times the [+] button to add four new columns to the schema, and name them
firstname, lastname, address and
city. -
For each of the columns you just added, change the function to match what is
shown in the table below by clicking in the Functions column and scrolling through the list of available
functions until you find the one you want, and then click OK when you’re done.firstname
TalendDataGenerator.getFirstName
lastname
TalendDataGenerator.getFirstName
address
TalendDataGenerator.getUsStreet
city
TalendDataGenerator.getUsCity
Defining the properties of tFileOutputRaw
-
Select the tFileOutputRaw component to define
its properties. -
In the Basic settings tab, click the
[…] button then browse to the location on
your file system where the output file is to be stored, or enter the path
manually between double quotes, and call the output file output.json
Leave the other parameters unchanged.
Defining the properties of tHMap
-
Select the tHMap component to define its
properties. - Click the […] button next to the Open Map Editor field to create a new map.
-
In the tHMap Structure Generate/Select
dialog box that opens, select Generate hierarchical mapper
structure based on the schema for the input structure, and then
click Next and then Finish. This means that
Talend Data Mapper
will
automatically generate a structure for you, based on the schema of the input
component (tRowGenerator in this case). -
For the output structure, select Select an existing
hierarchical mapper structure, and then click Next. -
Select the JSON_structure structure that
you created earlier, and then click Next and
then Finish. -
In the Map editor that opens, drag row from
Input (Map) to person in Output (JSON) to map
each of the input elements to its corresponding output element. -
Double click SimpleLoop in Loop tab and, in the properties box that opens, check
Stream Input and then click OK. -
Press Ctrl+S to save your changes to the
map.
Saving and executing the Job
- Press Ctrl+S to save your Job.
- In the Run tab, click Run to execute the Job.
-
Browse to the location on your file system where the output file is stored to
check that a JSON file showing the expected data has been successfully
created.
Handling errors
This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
This following scenario creates a six-component Job that shows how to handle error
conditions using the tHMap component.
When tHMap executes a
Talend Data Mapper
map, an ExecutionStatus object is always returned. A pointer to this object is
stored in the globalMap as EXECUTION_STATUS. In
addition, the Overall Severity numeric value (the
constants are defined in ExecutionStatus) is also
stored in the globalMap as EXECUTION_SEVERITY.
Finally, a parameter called Execution Threshold is
defined that specifies the severity on which to throw an exception, thus triggering
Job-related or component-related error processing. The default value for this parameter
is Fatal.
To consult the Javadoc for the ExecutionStatus
object, see the Camel Runtime documentation included in the
Talend Data Mapper
runtime kits and the
Talend Data Mapper User Guide.
Adding and linking the components
-
In the
Integration
perspective,
create a new Standard Job and call it error_handling. -
Click the point in the design workspace where you want to add the first
component, start typing tFixedFlowInput, and
then click the name of the component when it appears in the list proposed in
order to select it. -
Do the same to add a tHMap component, a
tJavaRow component, and three tJava components as well. -
Connect the tFixedFlowInput component to the tHMap component using a Row
> Main link, then connect the tHMap component to the tJavaRow
component using a Row > Main link and rename
the link out. If you are asked whether you
want to get the schema of the target component, click Yes. -
Connect the tFixedFlowInput component to the
first tJava component using an OnSubjobError trigger and to the second tJava component using an OnSubjobOk trigger, then connect the tHMap component to the third tJava component using a RunIf
trigger.
Defining the properties of the RunIf trigger
-
Select the RunIf trigger to define its
properties. -
Add the following code in the Condition section:
!((org.talend.transform.common.MapExecutionStatus)
globalMap.get("tHMap_1_EXECUTION_STATUS")).isOK()
Defining the properties of tFixedFlowInput
-
Select the tFixedFlowInput component to
define its properties. -
In the Basic settings tab, click the
[…] button next to Edit schema to define the schema. -
Add one column, of type
String
, rename it inputKey, and then click OK. -
In the Mode area. select the Use Inline Table radio button, then add three values
to the inputKey column by clicking the
[+] Â button and then entering the value
between double quotes in each case. In this example, the values used are
value1, value2 and value3.
Defining the properties of tJavaRow
-
Select the tJavaRow component to define its
properties. -
In the Basic settings tab, click the
[…] button next to Edit schema to define the schema. -
Click the [+] Â button to add a column to the
input schema on the left, of typeString
, rename it outputKey, copy it to the output schema on the
right, and then click OK. -
Add the following code in the Code
section.String actualOutput = out.outputKey;
System.out.println("======" + actualOutput + "======");
Defining the properties of tHMap
-
Select the tHMap component to define its
properties. -
Click the […] button next to the Open Map Editor field to create a new map based on
the input and output of tHMap. -
In the tHMap Structure Generate/Select dialog
box that opens, select Generate hierarchical mapper
structure based on the schema and then click Next to generate the input structure. - Do the same for the output structure.
-
In the Map editor that opens, drag the inputKey element
of Input (map) to the outputKey element of Output
(map), and then save your changes. -
Back in the Job, in the tHMap component, leave the value
of the Exception Threshold drop-down list (in
the Advanced settings tab) as Fatal. -
In the Basic settings tab, check that Read Input As is set to Data
Integration columns. - Check that Write Output As is set to Data Integration columns.
Defining the properties of the first tJava component (OnSubjobError)
-
Select the first tJava component to define
its properties. This component displays information in the console when the Job
contains an error. -
Add the following code in the Code
section.System.out.println("tJava_1: Subjob ERROR");
org.talend.transform.runtime.api.ExecutionStatus es = (org.talend.transform.runtime.api.ExecutionStatus)globalMap.get("tHMap_1_EXECUTION_STATUS");
System.out.println("Execution result:" + es.getOverallSeverity());
// ExecutionStatus object
System.out.println(es.toString());// XML version of ExecutionStatus object
java.io.StringWriter sw = new java.io.StringWriter();
es.exportToXml(sw);
System.out.println("ExecutionStatus as XML");
System.out.println(sw.toString());
Defining the properties of the second tJava component (OnSubjobOk)
-
Select the second tJava component to define
its properties. This component displays information in the console when the Job
runs successfully. -
Add the following code in the Code
section.System.out.println("tJava_2: Subjob OK");
org.talend.transform.runtime.api.ExecutionStatus es = (org.talend.transform.runtime.api.ExecutionStatus)globalMap.get("tHMap_1_EXECUTION_STATUS");
System.out.println("Execution result:" + es.getOverallSeverity());
// ExecutionStatus object
System.out.println(es.toString());// XML version of ExecutionStatus object
java.io.StringWriter sw = new java.io.StringWriter();
es.exportToXml(sw);
System.out.println("ExecutionStatus as XML");
System.out.println(sw.toString());
Defining the properties of the third tJava component (RunIf)
-
Select the third tJava component to define
its properties. This component runs if any errors occur within the map itself
(higher than informational status). -
Add the following code in the Code
section.System.out.println("tJava_3: Run If");
org.talend.transform.runtime.api.ExecutionStatus es = (org.talend.transform.runtime.api.ExecutionStatus)globalMap.get("tHMap_1_EXECUTION_STATUS");
System.out.println("Execution result:" + es.getOverallSeverity());
// ExecutionStatus object
System.out.println(es.toString());// XML version of ExecutionStatus object
java.io.StringWriter sw = new java.io.StringWriter();
es.exportToXml(sw);
System.out.println("ExecutionStatus as XML");
System.out.println(sw.toString());
Running the Job under different conditions
- Press Ctrl+S to save your Job.
-
In the Run tab, click Run to execute the Job.
In this case, no errors occur, so the Job triggers the second tJava component only.
-
Double-click the tHMap component to open the Map
editor. -
Right-click the outputKey element and click
Go to Structure Element. -
Change the tHMap_1_output structure that
opens from Read Only to Editable and then change the Data
Type for outputkey to
Integer (32). This means that this element
can only be anInteger
, and since this does not match the input, an
error will occur. -
In the Run tab, click Run to execute the Job again.
In this case, the Job still triggers the second tJava component even though there is an error, because the
threshold above which an exception should be thrown (thus enabling the
downstream components to detect the error) is set as Fatal. However, the third tJava
component is also triggered since there is an error in the execution of the
map. -
In the tHMap component, change the value of the Exception Threshold drop-down list (on the Advanced Settings tab) to Error. This causes an exception to be thrown when the map has a
severity of Error or higher. -
In the Run tab, click Run to execute the Job for a third time.
In this case, the Job detects the error and triggers the first tJava component (OnSubjobError).
Generating the Output Using tHMap with Multiple Schema Inputs
tHMap supports multiple inputs coming from either schema-based
input connections or other raw file inputs. In this scenario, you can see how you can
generate outputs using schema-based inputs with Main connections to
the tHMap component.
This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
Creating the Schema tHMap Job
The Schema tHMap Job includes three types of components –
tRowGenerator, tHMap, and
tLogRow.
- Create a Standard Job and name it thm_schemaInOut.
-
Add the following components:
- two tRowGenerator
- tHMap
- two tLogRow
- Connect the components using the Row > Main connection as shown in the previous image.
-
When prompted for the output name, enter aa and
bb respectively.
Configuring tRowGenerator
tRowGenerator creates an input flow in the Schema tHMap Job. In
this scenario, add two tRowGenerator components.
-
Double-click the first tRowGenerator to display the
Schema of tRowGenerator window. -
Add three columns with the following details:
- Type: String
- Functions:
TalendDataGenerator.getFirstName()
-
Double-click the second tRowGenerator to open the Schema of tRowGenerator
window. -
Add two columns with the following details:
- Type: String
- Functions: TalendDataGenerator.getLastName()
Configuring tLogRow
tLogRow displays data or results in the Run console. In this
scenario, use two tLogRow components to connect to tHMap.
-
Select the first tLogRow component to display the
Component tab. -
In the Component tab, click Edit
schema to open the Schema of tLogRow editor. -
Add two columns in both the Input and Output tables as shown in the following
image: -
Select the second tLogRow to the
Component tab. -
In the Component tab, click Edit
schema to open the Schema of tLogRow editor. -
Add three columns in both the Input and Output tables as shown in the following
image:
Configuring tHMap
tHMap executes transformations between sources and
destinations.
-
Double-click tHMap to display the tHMap
Structure Generate/Select window. -
Select Generate hierarchical mapper structure based on the
schema and click Next. - Click Next twice.
- Click Finish to open the Map Editor.
-
In the Map Editor, connect the following entries from the Input Map to the
Output Map:- row1 to bb
- row2 to aa
- Save your changes.
Running the Schema tHMap Job
Now that you have configured the components and added the connections, you can run
the Job to see the output in the Run console.
- Save the changes you made in the Job.
-
Click Run to execute the Job. The Run console displays
the following result:
Generating the Output using tHMap with Multiple Payload Inputs
tHMap allows you to connect multiple payload inputs and generates
the output in a single result. In this scenario, you will learn how to create a Job with a
tHMap connected to a multiple inputs.
This scenario applies only to Talend Data Management Platform, Talend Big Data Platform, Talend Real Time Big Data Platform, Talend Data Services Platform, Talend MDM Platform and Talend Data Fabric.
Prerequisites
- Download the attached sample payload input file. See the
Download tab and save the file in your local
machine. - Download the sample payload structure and import it to the Hierarchical Mapper > Structure list. See the Download tab and save the
file in your local machine.
Creating the Payload tHMap Job
The Payload tHMap Job contains three types of components – tFileInputRaw, tHMap, and
tLogRow.
-
Create a Standard Job and name it
thm_payload. -
Add the following components:
- two FileInputRaw
- tHMap
- tLogRow
- Connect the components using the Row > Main connection as shown in the previous image.
- When prompted for the output name, enter aa.
Configuring tFileInputRaw
tFileInputRaw reads all data in a raw file and sends the data to a
single output column for subsequent processing by another component.
-
Double-click the first tFileInputRaw to display the
Component tab. -
Specify the file path of the payload input file you downloaded earlier in the
Filename field as shown below: -
Repeat steps 1-2 for the second tFileInputRaw
component.
Configuring tLogRow
tLogRow displays data or results in the Run console. In this
scenario, use the tLogRow component to connect to
tHMap.
-
Select the tLogRow component to display the
Component tab. -
Click Edit schema to display the Schema of
tLogRow window. -
Add two columns in both the Input and Output tables as shown below:
Configuring tHMap
tHMap executes transformations between sources and destinations. In this scenario,
use an existing hierarchical mapper structure, test_payload, which is attached in the
Download tab.
-
Double-click tHMap to display the tHMap
Structure Generate/Select window. -
Click Select an existing hierarchical mapper structure
for row1 input connection. Click Next. -
Select the test_payload structure that you have
downloaded and imported previously from Hierarchical Mapper > Structures. Click Next. -
Repeat steps 2-3 for the row2 input connection. Click
Next twice. -
Select Generate hierarchical mapper structure based on the
schema option for the aa ouput connection. Click
Next. - Click Finish to display the tHMap mapping.
-
From the Input Map, drag and drop root/row1/root/x(1:*)/a
to root/aa/aaRecord(0:*)/newColumn in the Output Map to
create a connection. -
Select root/aa/aaRecord(0:*)/newColumn1 to display the
Value tab. -
Drag and drop AgConcat function in the Value
tab. -
From the Input Map, drag and drop root/row2/root/x(1:*)/b
to the AgConcat function as shown below: - Save the changes you made.
Running the Payload tHMap Job
After configuring the components and adding the connections, you can now execute the
Job to see the results in the Run console.
- Save the changes you made in the Job.
-
Click Run to execute the Job. The
Run console displays the following result:
Migrating from DatabaseLookup to tHMap
If you have Jobs that use the deprecated DatabaseLookup functionality, you can
migrate these to use the multi-input feature of tHMap.
Prerequisites
This procedure provides you an example of how you import Oracle databases to Talend Studio and create input connections
to the tHMap component.
- You have a Talend Data Mapper Structure
created from a database connection. - You have a Talend Data Mapper Map,
where the output element is the output of a DatabaseLookup using the database
tables imported in Talend Data Mapper.
See the following example: - If you wish to use the attached sample project from the
Downloads tab, you need to create databases using the
sample database scripts in your local server.
Importing the Database Tables
Manually perform the migration by importing the database tables to Talend
Studio.
- Create a new database connection in Metadata > Db Connections.
-
Select Retrieve Schema from the new DB
Connection menu. -
Select the table schemas to import.
Configuring the tHMap
Create a new Job that includes the tHMap component, where you can connect the
imported database tables.
-
Create a new Job.
-
Drag-and-drop the database tables from the Db
Connections and select
tOracleInput. -
Add the following components:
- tHMap
- tLogRow
-
Connect the following components using the Row > Main connection:
- tOracleInputdatabase connections to the
tHMap - tHMap to the tLogRow
- tOracleInputdatabase connections to the
- Enter xmlout when prompted for an output name.
-
Double-click tHMap to display the Structure
Generate/Select window. -
Select the Generate hierarchical mapper structure based on the
schema option to automatically generate Structures.
Migrating the DatabaseLookup to AgConcat
Now that you have successfully configured tHMap, Talend Data Mapperautomatically generates a new map with
multiple records that represent the database input connections.
-
Select tHMap to show the Input Map and Output XML panes.
See that the Input pane contains the imported database tables: -
Use the AgConcat function to replace the deprecated
DatabaseLookup function:-
Drag the row2 from the Input (Map) column to the
AgConcat expression. -
Set SimpleLoop on the Loop
Expression. -
Use the same filter values. If you have specified filter values in your
Job using the deprecated DatabaseLookup function, re-use these filter
values.
-
Drag the row2 from the Input (Map) column to the
Specifying a map variable and alternate map in tHMap
The Map Variable parameter in tHMap allows you to specify a
contextual variable that contains a path to a map.
This example shows you how you can build a Job using the tHMap component, where you
specify a map variable and an alternate contextual map.
Prerequisites
- Ensure that you have created several contextual maps. In this example scenario,
use three alternate contextual maps. -
In your local machine, create a text file with the following content, where
my_map
refers to the Map Variable name,
job_with_tContextLoad
refers to the Job name and
thMap_1_c
refers to the Alternate Contextual Map
name:1my_map;Jobs/job_with_tContextLoad/thMap_1_cEnsure that you replace the values according to the names of the Map
variable, the Job, and the Alternate Contextual Map you created.
Creating the Job
Create a Job that includes tHMap, tFileInputRaw, tFileInputDelimited, tContextLoad,
and tLogRow.
-
Create a Standard Job to add the following components:
- tFileInputRaw
- tHMap
- tLogRow
- tFileInputDelimited
- tContextLoad
Note: tContextLoad allows the contextual variable value to change at runtime.
This component uses a predefined schema with two columns
(key, value), where
key refers to the name of the contextual variable and
value refers to the new value of the corresponding
variable. -
Connect the components using the Row > Main connection.
- Enter out when prompted for the output name.
Configuring the Components
Before executing the Job, configure the components to add the required configuration
details.
-
Add three columns in the tRowGenerator component:
-
Add three columns in the out(Input) and
tLogRow(Output) tables in the
tLogRow component: -
Specify the text file that you have previously created earlier in the
File name/Stream field of the
tFileInputDelimited component. See the Prerequisite for more information. -
Enter details in the Map Variable and
Alternate contextual Map(s) fields in the
Advanced settings menu of
tHMap. -
Double-click tHMap to display the tHMap
Structure Generate/Select window. For both the Input and Output
Structures, select Generate hierarchical mapper structure based on
the schema. -
Map the input rows to the output rows in the tHMap
Structure:
Executing the Job
Running the Job automatically exports the referenced contextual maps.
Migrating IO Database functions to tHMap Multi-Output Database Connections
tHMap supports multiple output connections. If you have previously created a Job
using the IO Database functions, you can migrate these into using the multi-output
feature.
Prerequisites
- You have previously created Talend Data Mapper Structures from a
database connection. - You have a Talend Data Mapper Map that
uses a wrapped multi-table structure with a DatabaseInsert or DatabaseUpdate
IO/Database output function for each table. - If you wish to use the attached sample project from the
Downloads tab, you need to create
databases using the sample database scripts in your local server.
Updating the Map
To start the migration process, you need to change the output representation of the
Map.
- Double-click tHMap to display the Input and Output tables.
-
Replace each DatabaseInsert or
DatabaseUpdate function with a
WriteURL function:
Migrating the Job
Add the databases to the tHMap as output connections.
-
Right-click Metadata > Db Connections and select Create connection to add a new
database connection. -
Right-click the newly created database connection and select
Retrieve Schema. Import the necessary schemas. -
In the existing Job, drag-and-drop the database connection and select the
connection component. In this example, use
tOracleConnection and connect using the Trigger > On Subjob OK connection: -
Drag-and-drop the database tables in the Job and select its corresponding
output component. In this example, use
tOracleOutput. -
Connect each output component using the Row > Main connection. For the output name, enter the same database table
name from the Talend Data Mapper Map’s Output Structure.When you open the tHMap Map Editor, note that there are no synchronization
checks or proposals provided even after adding the output connections. To
synchronize the database table’s schema update on the Structure, see Syncing Database Table
Schemas. -
Add tOracleCommit and connect it to the
tHMap using the Trigger > OnComponentOk. - Run the Job and check the data of the database tables accordingly.
Syncing Database Table Schemas
Reload the changes made in the database table schemas.
- Right-click the database connection you created in Metadata > Db Connections and select Retrieve Schema.
-
Double-click the database output component,
tOracleOutput, in the Job to display the
Component tab.- Select Edit Schema.
- Sync the columns accordingly.
-
Go to the Mapping view and access Hierarchical Mapper > Databases.
- Right-click on the database you created.
- Select [Re]Load Table Definitions.
-
Access Hierarchical Mapper > Maps.
- Select the appropriate Job.
-
Right-click on the tHMap and select
Synchronize with Structures.