Reputation: 2489
I am importing data from large files (fwf and csv) that were once stored as tapes, so they may have errors created when writing-reading the tape.
The old files are in fixed width file (fwf) format The newer files are in .csv format (with ";" as separator)
The erros could be something like:
for fwf files:
for csv files:
Is there a way to import this into R skipping the error lines, but keeping a log of the errors so that they can be checked manually afterwards?
Or should I use another tool, external to R? In this case, which tool?
I have about ~100 files very large files (90GB each) so I would prefer some data.table::fread based, or some other fast, solution for this.
Upvotes: 3
Views: 255