Skip to main content
info
This documentation is for version v2.5 of the product.
For the latest version(v2.7) documentation click here
Version: v2.5 print this page

PipePython

This node runs any given Python code. It pipes the incoming DataFrame through pipe to the Python Script. Output back to Spark has to be written out using print.

Input

It pipes the incoming DataFrame through pipe to the Python Script. It also passes the Schema of the DataFrame to the Python script through the command line argument - argv[1]

Output

Output back to Spark has to be written out using print.

Type

transform

Fields

NameTitleDescription
codePipe PythonPython code to be run. It receives each record as a string and outputs records back as a string.
outputColNamesOutput Column NamesOutput Schema of Pipe Python Processor
outputColTypesOutput Column TypesData Type of the Output Columns
outputColFormatsOutput Column FormatsFormat of the Output Columns