LucasMation
LucasMation

Reputation: 2489

fix errors in csv and fwf files (corrupted characters) when importing to R

I am importing data from large files (fwf and csv) that were once stored as tapes, so they may have errors created when writing-reading the tape.

The old files are in fixed width file (fwf) format The newer files are in .csv format (with ";" as separator)

The erros could be something like:

for fwf files:

for csv files:

Is there a way to import this into R skipping the error lines, but keeping a log of the errors so that they can be checked manually afterwards?

Or should I use another tool, external to R? In this case, which tool?

I have about ~100 files very large files (90GB each) so I would prefer some data.table::fread based, or some other fast, solution for this.

Upvotes: 3

Views: 255

Answers (0)

Related Questions