CODEWITHSUNDEEP
pythonpython-2.7
ireinstein
ireinstein

Reputation: 63

How to join names of subdirectories and files for processing - python

I am trying to loop through a directory and read the files of each subdirectory. However, I need to keep track of the name of the subdirectory as it is a time value. I managed to create a dictionary that looks as follows:

dict = {'time1/dir1':['file1.ext', 'file2.ext'], 'time2/dir2':['name1.ext', 'name2.ext'}

But I have not been able to find the correct way to pass the complete name of the file to a function.

As I am trying to use np.fromfile(), I need to recursively join the name of the directory/time with each of the files in the list and store them in a way such that I have:

dict2 = {'time1/dir1':[value1, value2], 'time2/dir2':[value1, value2], }

I also read the directory as a pandas DataFrame, but I still need to read the files in such a way that the time is consistent.

I have tried using and mixing os.walk(), os.path.join(), os.listdir(), glob.glob() and others but my logic might be wrong when using these functions.

I know there might be a more robust, simple way to loop directly and maintaining the timestamp/directory name instead of creating a great number of directories and lists.

Upvotes: 0

Views: 2033

Answers (1)

Bharel
Bharel

Reputation: 26901

Is that what you're looking for?

import os
import os.path
base_path = "my/base/path"
directory_generator = os.walk(base_path)
next(directory_generator)
path_tree = {}
for root_path, directories, files in directory_generator:
    path_tree[os.path.basename(root_path)] = [
        os.path.join(root_path, file_path) for file_path in files]

The result is this:

{
    "dir1": [
        "my/full/path/dir1/file1.ext",
        "my/full/path/dir1/file2.ext"
    ],
    "dir2": [
        "my/full/path/dir2/anotherfile1.ext",
        "my/full/path/dir2/anotherfile2.ext"
    ],
}

Upvotes: 1

Related Questions