CODEWITHSUNDEEP
shellunix
user3901666
user3901666

Reputation: 409

Remove blank lines at the end of MS CSV files

I have around 300 csv files places at a path. These files names have 3 different categories i.e. "_PROD.csv", "_DEV.csv" and *_UAT.csv"

These files have many blank lines(around 1000) which got inserted at the end of the records when they were copied to the path by a process.

I want to remove the blank lines from all theses files. I have to perform a merge on these files after the blank spaces are removed.

I have tried the below:

sed -i '/^$/d' ${File_Path}*_PROD.csv
sed -i '/^$/d' ${File_Path}*_DEV.csv
sed -i '/^$/d' ${File_Path}*_UAT.csv

sed -i -e '/^[[:space:]]*$/d' \
    "${File_Path}"*_{PROD,DEV,UAT}.csv

But it's not working.

-sh-4.2$  echo sed -i '/^$/d'  Sample_data.csv
sed -i /^$/d Sample_data.csv

Sample data:

ID,Name ,DB
100,Amit,TD
200,Sumit,Oracle
300,Anshu,MS SQL
,,
,,
500,Jaya,MONGO
600,Sameer,HADOOP

Output: enter image description here

Upvotes: 0

Views: 65

Answers (1)

Yuji
Yuji

Reputation: 525

Please use '/^,*$/d' instead of '/^$/d'.

sed '/^,*$/d' ${File_Path}*_PROD.csv > ${File_Path}*_PROD_new.csv

Upvotes: 1

Related Questions