August 15, 2023

Scenario: Joining data about road conditions in a Pig process – Docs for ESB 6.x

Scenario: Joining data about road conditions in a Pig process

This scenario applies only to a Talend solution with Big Data.

The Job in this scenario uses two tPigLoad components
to read data about the traffic conditions and the related events on given roads from a
given Hadoop distribution, joins and filters the data using tPigMap, and writes the results into that Hadoop distribution using
two tPigStoreResult.


The Hadoop distribution to be used is keeping the data about traffic situation such as
normal or jam and the data about the traffic-related events such as road work, rain and
even no event. In this example, the data to be used reads as follows:

  1. The traffic situation data stored in the directory /user/ychen/tpigmap/date&traffic:
  2. The event data stored in the directory /user/ychen/tpigmap/date&event:

For any given moment shown in the timestamps in the data, one row is logged to reflect
the traffic situation and another row to reflect the traffic-related event. You need to
join the data into one table in order to easily detect how the events on a given road
are impacting the traffic.


The data used in this example is a sample with limited size.

To replicate this scenario, ensure that the Studio to be used has the appropriate
right to read and write data in that Hadoop distribution and then proceed as

Document get from Talend
Thank you for watching.
Notify of
Inline Feedbacks
View all comments
Would love your thoughts, please comment.x