CODEWITHSUNDEEP
bash
Village
Village

Reputation: 24423

How to delete all lines containing more than three characters in the second column of a CSV file?

How can I delete all of the lines in a CSV file which contain more than 3 characters in the second column? E.g.:

cave,ape,1
tree,monkey,2

The second line contains more than 3 characters in the second column, so it will be deleted.

Upvotes: 3

Views: 2380

Answers (4)

pizza
pizza

Reputation: 7640

This is a filter script for your type of data. It assumes your data is in utf8

#!/bin/bash
function px {
 local a="$@"
 local i=0
 while [ $i -lt ${#a}  ]
  do
   printf \\x${a:$i:2}
   i=$(($i+2))
  done
}
(iconv -f UTF8 -t UTF16 | od -x |  cut -b 9- | xargs -n 1) |
if read utf16header
then
 px $utf16header
 cnt=0
 out=''
 st=0
 while read line
  do
   if [ "$st" -eq 1 ] ; then
     cnt=$(($cnt+1))
   fi
   if [ "$line" == "002c" ] ; then
     st=$(($st+1))
   fi
   if [ "$line" == "000a" ]
    then
     out=$out$line
     if [[ $cnt -le 3+1 ]] ; then
        px $out
     fi
     cnt=0
     out=''
     st=0
   else
    out=$out$line
   fi
  done
fi | iconv -f UTF16 -t UTF8

Upvotes: 2

sorpigal
sorpigal

Reputation: 26096

No one has supplied a sed answer yet, so here it is:

sed -e '/^[^,]*,[^,]\{4\}/d' animal.csv

And here's some test data. 

>animal.csv cat <<'.'      
cave,ape,0
,cat,1
,orangutan,2
large,wolf,3
,dog,4,happy
tree,monkey,5,sad
.

And now to test:

sed -i'' -e '/^[^,]*,[^,]\{4\}/d' animal.csv
cat animal.csv

Only ape, cat and dog should appear in the output.

Upvotes: 2

Peter
Peter

Reputation: 3417

You can use this command:

grep -vE "^[^,]+,[^,]{4,}," test.csv > filtered.csv

Breakdown of the grep syntax:

-v = remove lines matching
-E = extended regular expression syntax (also -P is perl syntax)

bash stuff:

> filename = overwrite/create a file and fill it with the standard out

Breakdown of the regex syntax:

"^[^,]+,[^,]{4,},"

^ = beginning of line
[^,] = anything except commas
[^,]+ = 1 or more of anything except commas
, = comma
[^,]{4,} = 4 or more of anything except commas

And please note that the above is simplified and would not work if the first 2 columns contained commas in the data. (it does not know the difference between escaped commas and raw ones)

Upvotes: 2

kev
kev

Reputation: 161864

awk -F, 'length($2)<=3' input.txt

Upvotes: 9

Related Questions