CODEWITHSUNDEEP
kettlepentaho-spoonpentaho-data-integration
Philipp
Philipp

Reputation: 916

Parse CSV in transformation

I have an SSH step that executes a command which outputs CSV. This CSV should be parsed for further processing, however in Spoon I only found steps for parsing CSV data in files. How can I parse the stdOut field of the SSH step as CSV without writing it to a file first?

Upvotes: 1

Views: 61

Answers (1)

Ganesh Chandrasekaran
Ganesh Chandrasekaran

Reputation: 1936

If you are using MAC / Linux or windows 10 WSL try using AWK before parsing it via Kettle.

It can parse CSV very easily.

Example : Code like this will parse the csv filter the rows and extract necessary columns which are needed.

$ awk -F ',' '{if($4 == "Online" && $5 =="L") {print $1,$2,$4}}' sales_100.csv

More info : https://medium.com/analytics-vidhya/use-awk-to-save-time-and-money-in-data-science-eb4ea0b7523f

Upvotes: 1

Related Questions