Converts records into byte arrays.
strings or byte arrays to be used by the output components, such as tKafkaOutput requiring serialized data while tJMSOutput requiring strings. tWritePositionalFields embeds the incoming data into a single column based on
the format you defined for each column of the input flow.
tWritePositionalFields properties for Apache Spark Streaming
These properties are used to configure tWritePositionalFields running in the Spark Streaming Job framework.
The Spark Streaming
tWritePositionalFields component belongs to the Processing family.
Select the type of the data to be outputted into the target file. The data is
Schema and Edit
A schema is a row description. It defines the number of fields
The schema of this component is read-only. You can click
When the output type is String, the read-only single column is messageContent. This column is used to provide strings to the output components
When the output type is byte, the read-only single column is serializedValue. This column is used to provide byte arrays to the output
The output schema and its read-only column can be seen by
Select this check box to include the column header to the file.
You may encounter encoding issues when you process the stored data. In that
Select the encoding from the list or select Custom
Customize the data format of the positional file for each column
Advanced separator (for number)
Select this check box to change the separator used for numbers. By default, the thousands separator is a comma (,) and the decimal separator is a period (.).
This component is used as an intermediate step.
This component, along with the Spark Streaming component Palette it belongs to, appears
Note that in this documentation, unless otherwise explicitly stated, a scenario presents
In the Spark
Configuration tab in the Run
view, define the connection to a given Spark cluster for the whole Job. In
addition, since the Job expects its dependent jar files for execution, you must
specify the directory in the file system to which these jar files are
transferred so that Spark can access these files:
This connection is effective on a per-Job basis.
No scenario is available for the Spark Streaming version of this component