Host and Port

Enter the hostname and the port of the machine used as the sink (the data output point
bound to the channel of a Flume agent) to receive data from Flume.

• If you select As Receiver from the Type drop-down list, this machine must be one of the machines on which a
  Spark worker runs and the hostname must be the same as the one used by the resource
  manager of the Spark cluster to be used.

• If you select As Sink from the Type drop-down list, this machine must be a sink in a Flume agent and be
  accessible to the Spark cluster.

Type

Select the approach to read data from Flume.

• As Receiver: this is the Push-based approach typically
  employed by Flume. In this approach, a machine from the Spark cluster is set up as
  an agent to receive data pushed by Flume and the Spark Streaming Job you are
  designing reads data from this agent.

• As Sink: this is the Pull-based approach. In this approach, a
  machine is set up as sink to buffer data pushed by Flume and the Spark Streaming Job
  you are designing pulls data from this sink.

For further information about these two approaches, see https://spark.apache.org/docs/1.3.1/streaming-flume-integration.html.

Schema and Edit
Schema

A schema is a row description. It defines the number of fields (columns) to
be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Built-In: You create and store the
schema locally for this component only. Related topic: see
Talend Studio

User Guide.

Repository: You have already created
the schema and stored it in the Repository. You can reuse it in various projects and
Job designs. Related topic: see
Talend Studio

User Guide.

This read-only line column is used by tFlumeInput to automatically extract the body of an input Flume event
and construct an RDD along with the other columns used to store the header of the same
event.

Encoding

Select the encoding from the list or select Custom and define it manually.

This encoding is used by tFlumeInput to decode the input
event arrays.

Usage rule

This component is used as a start component and requires an output link.

At runtime, the tFlumeInput component keeps listening to
the sink and reads new events once they are buffered in this sink.

This component, along with the Spark Streaming component Palette it belongs to, appears
only when you are creating a Spark Streaming Job.

Note that in this documentation, unless otherwise explicitly stated, a scenario presents
only Standard Jobs, that is to say traditional
Talend
data
integration Jobs.

Spark Connection

This connection is effective on a per-Job basis.

Limitation

Due to license incompatibility, one or more JARs required to use this component are not
provided. You can install the missing JARs for this particular component by clicking the
Install button on the Component tab view. You can also find out and add all missing JARs easily on the
Modules tab in the
Integration
perspective of your
studio. You can find more details about how to install external modules in Talend Help Center (https://help.talend.com).