Version: v2.4
RemoveDuplicateRows
This node take an array of fields, compare rows on those fields. If they full match then its a match. From the matches it would randomly take one row and drop the rest.
Input
It accepts a DataFrame as input from the previous Node
Output
The output Dataframe is the same as the input Dataframe with the duplicate rows removed
Type
transform
Fields
Name | Title | Description |
---|---|---|
order | Order | Whether to take the first or last matching record when removing duplicates |
inputCols | Columns | The columns to be selected for match |