CODEWITHSUNDEEP
runix
user702432
user702432

Reputation: 12178

Piping unix commands

This is a follow-up to this question. I have this fixed-width file, where the columns-widths are 34 2 3 2 2 1 2 2 1 1 2 2 1 2 2 2 and 65. I want to read this into R after dropping the first and last columns, where the second column has the value "07". Thanks to Mat, if I just want to convert the txt file to csv, I can do it with this code:

awk -v FIELDWIDTHS='34 2 3 2 2 1 2 2 1 1 2 2 1 2 2 2 65' -v OFS=',' '($2=='07'){ $1=$1 ''; print }' </filepath/pipe.txt | cut --delimiter=',' --fields=2- > /filepath/parsed.csv

Of course, I can then read the parsed.csv file in. But I am trying to do it one shot with pipe(). when I try to pipe this in R, it hangs:

a = read.csv(pipe("awk -v FIELDWIDTHS='34 2 3 2 2 1 2 2 1 1 2 2 1 2 2 2 65' -v OFS=',' '($2=='07'){ $1=$1 ''; print }' 
                  </filepath/pipe.txt 
                  | cut --delimiter=',' --fields=2-")
             , header=F, colClasses="character")

Upvotes: 2

Views: 1406

Answers (2)

user702432
user702432

Reputation: 12178

jigr (and other readers)- I found the strangest quirk in R/RStudio. The code in my question works perfectly, provided there is no line-break:

a = read.csv(pipe("awk -v FIELDWIDTHS='34 2 3 2 2 1 2 2 1 1 2 2 1 2 2 2 65' -v OFS=',''($2=='07'){ $1=$1 ''; print }' </filepath/pipe.txt | cut --delimiter=',' --fields=2-"), header=F, colClasses="character")

I don't have the slightest idea why this is happening.

Upvotes: 0

johannes
johannes

Reputation: 14453

I am not sure about pipe() but I would do it with system().

library(stringr)
txt <- system("awk -v FIELDWIDTHS='34 2 3 2 2 1 2 2 1 1 2 2 1 2 2 2 65' -v OFS=',' '($2=='07'){ $1=$1 ''; print }' < pipe.txt | cut --delimiter=',' --fields=2-", intern=T)
do.call(rbind, str_split(txt, ","))

Upvotes: 3

Related Questions