CODEWITHSUNDEEP
bashshellcsvunix
Jonas Van Bogaert
Jonas Van Bogaert

Reputation: 339

How to read CSV file stored in variable

I want to read a CSV file using Shell, But for some reason it doesn't work.

I use this to locate the latest added csv file in my csv folder

lastCSV=$(ls -t csv-output/ | head -1)

and this to count the lines.

wc -l $lastCSV

Output

wc: drupal_site_livinglab.csv: No such file or directory

If I echo the file it says: drupal_site_livinglab.csv

Upvotes: 0

Views: 276

Answers (1)

Tom Fenech
Tom Fenech

Reputation: 74695

Your issue is that you're one directory up from the path you are trying to read. The quick fix would be wc -l "csv-output/$lastCSV".

Bear in mind that parsing ls -t though convenient, isn't completely robust, so you should consider something like this to protect you from awkward file names:

last_csv=$(find csv-output/ -mindepth 1 -maxdepth 1 -printf '%T@\t%p\0' |
  sort -znr | head -zn1 | cut -zf2-)
wc -l "$last_csv"
  • GNU find lists all files along with their last modification time, separating the output using null bytes to avoid problems with awkward filenames.
    • if you remove -maxdepth 1, this will become a recursive search
  • GNU sort arranges the files from newest to oldest, with -z to accept null byte-delimited input.
  • GNU head -z returns the first record from the sorted list.
  • GNU cut -z at the end discards the timestamp, leaving you with only the filename.

You can also replace find with stat (again, this assumes that you have GNU coreutils):

last_csv=$(stat csv-output/* --printf '%Y\t%n\0' | sort -znr | head -zn1 | cut -zf2-)

Upvotes: 1

Related Questions