CODEWITHSUNDEEP
bashcsv
yuqli
yuqli

Reputation: 5161

write the name of all files in a directory, and their absolute path, to a csv file in bash

This is harder than I expected, but I have a folder with ~100 datasets in .csv format.

I would like to create a .csv file with the following fields:

I would like to do this with bash commands. But so far I have only be able to do :

ls >> out.csv

which will write all file names into a txt file... I see some people using a for loop, but manipulating lines in .csv file seems forbidding, and I don't know what to put inside the for loop...

Am I better off just using Python? Any help is appreciated... Thanks!

Upvotes: 0

Views: 2532

Answers (1)

yuqli
yuqli

Reputation: 5161

Thanks for the advice of gurus above, I came up with this Python program that 1) extracts file names and 2) extract field names in each file. Any comments are welcomed. Thanks!

import os
import csv
info = {}    # store all information into a Python dictionary
for filename in os.listdir(os.getcwd()):
    with open(filename, newline='') as f:
        reader = csv.reader(f)
        row1 = next(reader)
        info[filename] = row1

path = os.getcwd()
header = 'field, dataset, path'
write_file = "output.csv"
with open(write_file, "w") as output:
    output.write(header + '\n')
    for key, value in info.items():
        for elem in value:
            curr_path = path + key
            line = '{0}, {1}, {2}'.format(elem, key, curr_path)
            output.write(line + '\n')

Upvotes: 1

Related Questions