CODEWITHSUNDEEP
apache-hop
JustLudo
JustLudo

Reputation: 1790

Pipeline unable to read field of plain text file

Using Apache Hop latest version I'm trying to read in a plain text file. This text file is old and basically only structured by its lines (it has no delimiter, no seperator, no enclosure, etc.). I would like to read and process the lines of this file as rows in my transformation.

I use the "Text file input" transformation to read the file. Apparently reading it works, but I seem have no field available when trying to retrieve the fields. It simply states that no fields were found.

When I run the "preview records" I do get empty records equal to the number if lines in the file, so that is good. However there is no data shown as there is no field detected.

Curiously enough, when I press "Show file content" I DO get the desired content, nicely structured in the rows as desired, so I know the file is being read correctly.

Does anyone know how to best read these kind of files?

PS: The files can be anywhere from 10 to 100000 lines.

Upvotes: 0

Views: 511

Answers (2)

HansVA
HansVA

Reputation: 106

When there is no header row with field names or Hop is not able to detect any fields you can also create a field in the fields tab and it will put content in there.

As we just use a position based approach and split the content using the specified delimiter everything should go in "field1" when no delimiter is found in the data.

Upvotes: 1

JustLudo
JustLudo

Reputation: 1790

Figured it out. The naming is a bit misleading, but you can use the "CSV File input" and then set a TAB as delimited. Then use preview on your file and you should find that the lines are actually being parsed.

Upvotes: 0

Related Questions