August 15, 2023

tBigQueryOutputBulk – Docs for ESB 6.x

tBigQueryOutputBulk

Creates a .txt or .csv file for the data
of large size so that you can process it according to your needs before transferring it to
Google BigQuery.

The tBigQueryOutputBulk and tBigQueryBulkExec components are generally used together as parts of a two step
process. In the first step, an output file is generated. In the second step, this file is
used to feed a dataset. These two steps are fused together in the tBigQueryOutput component, detailed in a separate section. The advantage of
using two separate components is that the data can be transformed before it is loaded in the
dataset.

This component writes given data into a .txt or
.csv file, ready to be transferred to Google BigQuery.

tBigQueryOutputBulk Standard properties

These properties are used to configure tBigQueryOutputBulk running in the Standard Job framework.

The Standard
tBigQueryOutputBulk component belongs to the Big Data family.

The component in this framework is generally available.

Basic settings

Schema and Edit
Schema

A schema is a row description. It defines the number of fields (columns) to
be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Click Edit schema to make changes to the schema.
If the current schema is of the Repository type, three
options are available:

  • View schema: choose this option to view the
    schema only.

  • Change to built-in property: choose this
    option to change the schema to Built-in for
    local changes.

  • Update repository connection: choose this
    option to change the schema stored in the repository and decide whether to propagate
    the changes to all the Jobs upon completion. If you just want to propagate the
    changes to the current Job, you can select No
    upon completion and choose this schema metadata again in the [Repository Content] window.

Property type

Built-In: You create and store the
schema locally for this component only. Related topic: see
Talend Studio

User Guide.

 

Repository: You have already created
the schema and stored it in the Repository. You can reuse it in various projects and
Job designs. Related topic: see
Talend Studio

User Guide.

File name

Browse, or enter the path to the .txt or .csv file
you need to generate.

Append

Select the check box to write new data at the end of the existing
data. Otherwise, the existing data will be overwritten.

Advanced settings

Field Separator

Enter character, string or regular expression to separate fields for the transferred
data.

Create directory if not exists

Select this check box to create the directory you defined in the
File field for Google Cloud Storage, if it
does not exist.

Custom the flush buffer size

Enter the number of rows to be processed before the memory is freed.

Check disk space

Select the this check box to throw an exception during execution
if the disk is full.

Encoding

Select the encoding from the list or select Custom and define it manually. This field is compulsory for database
data handling.

tStatCatcher Statistics

Select this check box to collect the log data at the component
level/

Global Variables

Global Variables

NB_LINE: the number of rows read by an input component or
transferred to an output component. This is an After variable and it returns an
integer.

ERROR_MESSAGE: the error message generated by the
component when an error occurs. This is an After variable and it returns a string. This
variable functions only if the Die on error check box is
cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable
functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl +
Space
to access the variable list and choose the variable to use from it.

For further information about variables, see
Talend Studio

User Guide.

Usage

Usage rule

This is an output component which needs the data provided by its
preceding component.

Related Scenario

For related topic, see Scenario: Writing data in BigQuery


Document get from Talend https://help.talend.com
Thank you for watching.
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x